Tile repair question--looks like the backing behind the tiles has some issues

Hi All,

We've got a bathtub/shower where some tiles were coming loose. I pulled out the loose tiles, ready to scrape the old mortar out, and put new mortar in to re-attach the files.

Well, this is what I found. You can see that the concrete backer board (where it exists) is crumbling to pieces, then there's plywood off to the right, and then there are spots where is no backing at all! The backing is completely missing in the corner (on the righthand side), and between the concrete backer board and the plywood.

By the way, the plywood was black with mold--I sanded a lot of it away. But I thought I would pause and come here for advice. Can I mortar this up and put the tiles back, or am I just asking for heartache later by doing that? Do I have to (gulp) rip all the tiles out in this quadrant of the shower, put up new backerboard, and then re-tile it?

Thanks for any insights!

-Josh