Don't worry about the wire from panel. It likely goes several places before the light such as receptacles. You have the neutral at the light already. Light to switch is your only concern.
You may possibly can run a cable from a nearby receptacle to switch.