This may be a really basic question, but I can't get my mind around it for some reason. If I am using a 3 way dimable switch, do both switches need to be dimmers?
No. Most 3-way dimmers work with a regular 3-way switch at the other end. There are smart dimmers on the market which allow for both sides to be dimmers, but this probably isn't the case in your setup.
I guess I have just been over-thinking the whole thing. I assume you would want the dimmer on the first switch...but then would the second switch just turn the lights on to the level left by the dimmer or would it override the dimmer?
The dimmer portion of a 3-way dimmer is in line with the common terminal. The switch portion is a regular 3-way switch. When you dim the load, that's it's "on" position and if you dim down to zero, there is no way the other 3-way can turn the light on, because it's choices are off or fully dimmed.