SHARP flat tool to get what you can off. Plastic razor blades in a holder or something like them but larger would be a good choice. Plastic razor blades will come in handy for lots of things and they don't rust like metal ones.
Then try some rubber cement thinner. I think you can still get it in gallons at an art supply store. Work in small areas at a time as it evaporates quickly.
Products for latex paint removal should also work on latex rug backing residue---Goo Gone and Oops come to mind. Krud Kutter is another possibility but I know it comes with a warning that it may discolor hardwood floor finishes.
You should test for colorfastness and other damage to the hardwood floor surface in a small area no matter what you choose.
USE ANY OF THESE PRODUCTS ONLY WITH ADEQUATE VENTILATION!