here's what I'd do:
Drywall/caulking to stair treads is your quickest way out of the forest, no doubt, but that is a flawed solution for more reasons than I care to get into, and I think you know this. The good news, as I see it, you've got some sweet looking steps already and there's no safety issue at this point, so you can take your swet time on this project ...if you want to prevent the certain cosmetic and maintanance woes you are facing as is.
That said, here's what I would do:
Whomever pointed out the "blocking" issue was correct, i think, - at least it appears that there is insufficient support for the sheetrock between furring strips.(The following suggestions are meant to apply to both sides...)
First, snap a line on the wall from top to bottom a few inches over the steps. This line should be below where you would have your finished baseboard height .neatly remove all the good drywall work you've done below that line (and that was good work with the drywall, I must say) it will go back in as a furring layer, so save it... The goal here is to install continuous nailing for the wall and trim, but as long as you've got access to the profile of the staircase, why not take this opportunity to make a perfect cutting template for the baseboard you want to install, eh? See where IK'm going with this?...Ya pickin up what I'm puttin down here, chum?
If possible, slide a continuous 12" high strip of thin (yet semi rigid)
material like doorskin, inexpensive veneer or formica - whatever- along the entire stringer between the stairs and the block wall. Ideally, you want to trace it as one unbroken length, no bending, or in as few pieces as possible (a hot glue gun is a wonderfull asset for template making btw) and as accurately as possible and transfer the whole shebang onto primed MDF base shoe. This will become your custom fit, one piece baseboard to be installed after you get your blocking and drywall back together. Why MDF? because it's cheap, easy to cut, patch and repair, and you will be painting this anyway because it's probably going to look like frankenstein's underwear by the time you have it all installed. ("one easy piece" is the plan, but you know...) You can finish it with a decorative strip of "stain grade" cap moulding to match the stairs.
On the baseboard, don't try for an airtight fit , -shoot for 1/8" consitent caulking joint all around and things will go way easier, trust me
It's a lot of work, I know.
My other suggestion was to unscrew the lightbulb in the stairwell and go drink a beer.