[HotRodx10]

I have a black wire, white wire and the bare ground wire going directly to the panel.

Only if you're sure the white is a neutral that goes to the panel. Is the white connected to the switch? If so, it's not a neutral; it's a switched hot wire.