Hi Harp,
I've been hoping someone would offer a nice solution, but you are still waiting. So, here are some thoughts.
You want to build a wall or two to enclose that utility area but you are concerned about how to insulate that area and or isolate it from the basement so it doesn't make the basement cold. Correct me if I don't have the correct objective.
The difference I see is what will happen with that space is you omit the insulation and isolate it from the heat in the basement. it will become almost as cold as the outside temps and that is not good. Basement wall are always a source of moisture, mostly vapor that can be easily managed inside the conditioned basement.
Here's one approach. Picture frame everything mounted to the wall area where you cannot get rigid insulation and drywall behind it. Use pressure treated wood against the concrete and match the thickness of the rigid insulation. Then cover all exposed concrete within those frames, cut as needed and seal all seams, caulk, can foam, or foil tape. With the framing in place you will then be able to install a covering layer of 1/2" drywall as a thermal barrier.
You would still want to address the rim joist, but with the bulk of the area insulated you could then add some heat to that space to eliminate all moisture issues and omit the need to insulate the new walls. A louvered door might also be a good option.
The areas that are not insulated will remain warm enough to not be a condensation problem and the cost of the heat loss would be minimal, think $10 per year at most.
Bud