Very bad indeed. Mold will be your next visitor. That's probably an insulation and ventilation problem. The cold is getting into and through your wall. I am guessing it's a ranch house so make sure your attic insulation is pushed up tight to the walls. Also, your indoor humidity may be a bit high for the current outside air temp. You may not have adequate bathroom and kitchen fans, or you are not running them enough.
I would also check the outside of the house on the north side and see if there is any places that need to be caulked to give you a better seal. Also would agree on the mold issue too. How long has this been going on? If it has been for a while you might think of cutting a section out where it frosts and see what is behind the wall, just to be on the safe side. You definitely do want to let that get out of hand if all possible. Let us know what you find out or how you get it fixed.