With a smaller pipe you could use the pipe as a water drill, I did this once with 2" and went under a street - was a bear to do and took a jack hammer as modivation to the pipe. The city would not allow the street to be be dugup or even closed. I don't think you could do the water drill thing with 4" pipe.
Digging under half way from either side of the sidewalk would be my suggestion. Not easy I know, but should not be too bad.