It's DC. Not 120. But yes it does matter. The DC coolers use a thermalpile. When a current is applied to it, one side gets cold one side gets hot. If you install the wires backwards the current flow will reverse (obviously since its DC) and the inside will warm and the outside will cool.
Cheers