Insulating window weight pockets
Just be careful when using great stuff around windows (or doors) as it is an expansion foam and can affect the performance of your window. If you do need to use it, use the 'low-expansion' version. As far as the fiberglass goes, fill the void, but you do not need to 'pack' it. Packing it tightly will prevent the fg from breathing and allowing moisture/condensation to dissipate. If you have removed the old wood windows and weights to install a replacement (vinyl, aluminum etc) then you could leave the weight access cover in place and just fill from the top via the hole where the pulley was. Otherwise, I think you idea of feeding it up from below will work well.
Man of many hats