Sorry to say, it sounds like you are a victim of shoddy work. It would appear as though the base was not prepared properly. Were I you, I would make every effort to track the contractor down and have him rectify the problem as it is not a small one. To repair this properly you will have to remove everything and start over with a solid base.
Info on how it should be done is everywhere, Google 'installing pavers' for lots of articles.