A point vortex is introduced into a weak background vorticity gradient at finite Reynolds number. As the vortex spreads viscously so the background vorticity becomes wrapped around it, leading to enhanced diffusion of vorticity, but also giving a feedback on the vortex and causing it to move. This is investigated in the linear approximation, using a similarity solution for the advection of weak vorticity around the vortex, at finite and infinite Reynolds number. A logarithmic divergence in the far field requires the introduction of an outer length scale $L$ and asymptotic matching. In this way results are obtained for the motion of a vortex in a weak vorticity field modulated on the large scale $L$ and these are confirmed by means of numerical simulations.