Exactly what it takes to get service turned on depends on your local building inspector... we just had another guy through here with a similar experience, except they wouldn't turn on his power until he made his house habitable.
With your existing box, you might be able to make it work. Cover up that hole (I would screw in a 4" metal box cover) and use the knockout on either corner - the conduit end looks like it already has the correct fitting for it. Make sure you have enough cable to pull into the box to do this - if not you will have to replace the cable.
Haven't seen the ground connection yet - this is of equal importance... no, it cant just be connected to a water pipe either!