Your box will need to be supported other than with the conduit. like a 4X4 post.
As far as the height, although this isn't a rule, I would install it a little higher than your average snowfall. Around here I usually stay about 12" above the ground, even though we rarely get that much snow
You can also purchase a small yard post that's made out of 2" PVC in various colors, and used to mount light fixtures and receptacles.
RAB lighting calls the "mighty post" and I think Arlington makes a square post for a receptacle. Here's an example
http://www.prolighting.com/rabmpmimipo1.html
Also, Don't forget to use an "in use" cover.