Cold air coming in around microwave

So I installed a new microwave above the cooktop. It is vented through the wall and up to the roof. There is a damper on the back of the microwave but it is not air tight (just a piece of metal that swings freely.

I now notice that cold air is coming back into the kitchen from the microwave vent. Is there a better way to do this so cold air doesn't leak back in? Would it be better to put an electric fan on the roof and turn it off/on as needed? Do they have built in dampers?

Thanks,
Rut