Help with 1950s bathroom fixture update.

Hello! I have a home built in the early 1950's with plaster walls. I'm trying to get the recessed metal toilet paper and soap holders out to replace them. I've cut away the grout on the edges but both are being held in from the back by something (Plaster?) and I'm not sure how to go about getting them out without damaging the surrounding tile. Any help is greatly appreciated!