Need help with bathroom exhaust fan/light box
well most likely what you have is an incoming two wire, an outgoing two wire and a two wire to the switch, what jims saying is that you need to hook into the "switch leg" from the switch, a picture will probably help but i assume that there's three black and three whites in the box from your description. Most likely the two "always on" blacks get hooked together with the white from the switch(should be identified as black) and the other two whites get wire nutted together with a pigtail out to the neutral for the light fan combo, the black comming from the switch gets hooked to the black for the light fan combo. This is how it should have been wired previously, if it wasn't then you may need to identify where the other two wires go before going any further.
Sarcasm is my friend
I'm here to learn too, i do mostly commercial/industrial/new construction and this place is a great way to pick up tips on residential from some good electrical minds. Excuse the spelling, my phone has a mind of it's own.