Have you called the state agency that issued the license for this contractor? They may have suggestions on how to resolve this issue (quite a bit of time has passed - but I'd call anyhow, especially if you notified the contractor in writing of your concerns).
Was this a contractor recommended by Milgard? If yes, I'd also contact Milgard. If you still need a contractor to fix the problems, I'd go with one recommended by Milgard - you can find this information on their website.
This will only get resolved if you are proactive in resolving it. What have you done so far? From your May response, I'm wondering if you have done anything other than talk to the contractor?
The state licensing agency likely records complaints against contractors.