[Gymschu]

You don't have to get all of the dust off, just most of it. The primer will encapsulate what remains. Once the primer is dry, I sand it fairly aggressively with an round pole sander and sometimes my power orbital sander with 80 grit. For really fussy customers, I will HAND sand (ugh) with 80 grit paper.

Even if you would get all the dust off, the primer tends to create a slight bit of grit when it dries, so, you usually have to sand again. So, save yourself some time and don't get too fussy with removing dust.

A soft broom works well, a vacuum with a HEPA filter works better but you really need one of those soft bristled attachments to keep from damaging the new skim coat. I've even used an electric blower, opened up windows and doors and blew a lot of the dust outside, but, that may leave a layer of white dust on your windows, siding, and anything else within striking distance.

 

[Gymschu]

Not a big fan of leaving the dust on the walls. Now, maybe a little dust is no big deal, but if there’s enough dust left on the walls, you CAN have adhesion issues with the primer.