** update **
First of all, thank you all for giving me the stones to take this problem on head-on. I took a little bit from everybody's suggestions.
I stopped off at Lowe's on the way home from work and got ABS materials I would need. The men that worked there really didn't have any good ideas as to how I would attach my 1.5" pipe into the stack. After describing the situation, I was told that it was common practice to use lead to seal joins like that, and I was probably best to leave it alone. I figured my best course of action at this point is to get a 1.5" Fernco, leave about 4" of the original pipe, and this way I could either attach the new ABS or the old brass drain.
I got out my saw and cut the drain pipe, and when it started leaking, I let it drip and came back 10 minutes later (there's a laundry sink right under it). I continued to cut. As I was cutting deeper and deeper, I noticed that the water coming out was more and more brown.
After cutting it through, I grabbed my flashlight and snake. Once again, the snake went nowhere, even though it was only going in about 8 inches. I did a little bit of scraping with auger, coat hanger, and screwdriver. I got nowhere, and decided I had to cut it flush and worry about how I'll attach it later.
After doing so, I was horrified to see that the cast-iron Wye was so clogged that I could barely poke my screwdriver through it. I can tell you that SOME of it was rust, but most of it was some kind of gunk that I had to remove with the screwdriver and hammer. Whatever it was, it was dark gray inside, and hard as rock. The good news is that under this crap, the pipe seemed undamaged (like I'm an expert!)
Using screwdriver, hammer, long round file, and crowbar (hey, it's curved like the Wye!), I was careful to remove all of it that I could, and not let it fall in. Once that was done, I cut and glued all but one of the ABS pieces. This I left unglued so that I have wiggle room to "properly" attach the pipe to the stack.
I feel so much better. I guess my main concern now is, since I still have access to the Wye, is there anything I can apply to it, like a foam, that would help "digest" this? I can't use Drain Care for obvious reasons, because it's no longer clogged, so it would just run down. I thought about mixing up a batch of it and squirting it on the inside, but I don't know if that will help.
Once again, thank you all for your help and support.