When it rattles, that's a sign of the bushing allowing too much free play side to side.

​

The solution is to replace it or while you're at it, replace the whole SSV.