Is the house owned, or rented? Is it freezing temperatures outside until April or so, as is typical for 80% of the country, where crazy people live? (jk)
You have lots of options, form simple to complex and costly. I suggest however, that all are money savers. You are paying a steep price as it is, just to the utility company, as you well know, once a month!
Replace the windows, caulk and seal the windows, insulate the outside walls, and more can all be done. The caulk and seal drill is going to be included in all answers. Generally, it cannot be done under 40F.