When the blades are out of balance the fan will wobble. A wobbling fan may also be a noisy fan. The wobbling will cause things to loosen up.
Not your problem? See the articles 'Ceiling Fan Troubleshooting' and 'Electrical Repair' for all the electrical articles.
The first thing to check would be the housing. Is it mounted securely to the bracket. Tighten anything that is loose. Does the mounting plate seem loose? Remove the housing and tighten the mounting plate.
The primary reason a fan will wobble is the blades. The blades are attached to mounting brackets. The brackets are attached to the flywheel. All of the blades need to be evenly spaced, at the same pitch, in alignment with one another, weigh the same and not have bent brackets. I've got to be kidding right? Not really.
Watch the fan on slow speed both from below and from the side. You should be able to tell if something is out of kilter. Do you have a warped fan blade? You can try bending it or get a replacement set of blades. Does one of the brackets seem bent? You can try bending it as well, be careful, they can break.
The same would be true if you see a blade that is pitched differently than the others. Try bending it, so they are all even. Be careful or your ceiling fan troubleshooting project will turn into a ceiling fan replacement project.
You usually get a balancing kit with a new fan. Some weights with adhesive on them. Taping a quarter or metal washer on the back will also work. Try adding a weight to the blades one at a time and see if things improve. This is not an exact science and you may need to experiment.