I just did a lot of this, and yeah a putty knife and some effort worked wonders. I bought some silicone caulk remover and it didn't help at all. Scraping pretty darn hard did not scratch or damage the fiberglass surround at all, but I can't comment on different materials.