With the cheap kg to orbit heavy lift systems coming online, these near misses are becoming opportunities for vast riches if they can be captured and mined. Less massive making the orbit easier to change.
you have to have delta v regardless. enough for your mined material, or the entire mass of the asteroid. i seriously doubt we'd ever approach ability of completely mining the whole of the asteroid.
however, if you removed enough mass from the asteroid, would you end up screwing yourself by now changing the orbit so that the next time it swings by it's more of a direct trajectory?