Is there a small grub screw at the bottom of this hole? If its old it may be covered in grime so try turn it with a small screwdriver (flat headed) If there is no screw then this knob may have rusted itself on over the years due to its exposed condition. You may have to resort to brute force if a liberal application of WD40 doesn't work!
The trick is to apply force evenly around the knob, try cutting two wedges out of softwood that just fit behind it, then with two hammers strike downward until it loosens off.
I would have thought that taking the inside one off would enable the whole unit to be slid out but its hard to say without taking a look!