Couple of things I've experienced, not sure if either matches your symptoms, but might help in your process of elimination...
When I got my Soarer it was quite a pig to drive due to worn lower front wishbone bushes, tramlining quite badly. But it was under heavy braking that it felt the worst, as the forces involved then pushed the wheels back, adding toe-out, making the car meander around. So I'd imagine that under braking is when one might first notice that bushes are wearing.
Another issue I've had with braking is when the wheel bearing went - gradually I noticed more and more vibration under braking, but felt through the car not the wheel or brake pedal. Even just before I got it sorted, it really wasn't noticable at low speeds or light braking, and there wasn't any detectable play in the bearing either. It was worse when braking on a bend. This happened after the car had been stood for a couple of months (mechanic thought that wasn't just coincidence).
Hope you get it sorted whatever it is, braking's the one thing you want working 100% on any car!!