The quick way of determining if the breaker is at fault (before buying a new one) is to temporarily move the wire from the breaker that is slow-to-trip to the breaker next to it. If you get the same results as with the first breaker then the problem is between the breaker and the fan.
I had a similar problem recently with lights in the upstairs bath. Everythign pointed to a bad breaker. Turns out that the neutral wire (white) was loose at the breaker box.
Just want to caution you here:
Before doing what you describe, make sure that the breaker next to the suspect one has the same rating!
You don't want to be putting a 15A circuit onto a 20A breaker!