[TarheelTerp]

I have a 240V 200amp main service with an 8/4 run of about 50ft to a sub-panel in my garage on a 2-pole 40amp breaker.

Remember when you were advised to use a heavier wire... just in case?

(now) I want to extend from this panel to a pool house

How much more power do you need?

Ideally 50a, but 40a should be fine.
Thankfully I put it in a 3" conduit...

Do a complete load calculation this time so you KNOW how much power
is needed and then upsize that by 20% or more.
---

The short answer is to pull back the #8 wires and re-feed the garage panel.
You might be able to reuse the #8 for the sub-sub feed out to the pool.