How can I tell if 4 tandems and adding a 100amp breaker isn't overloading that panel?
Put a 100 amp panel with main breaker in your workshop. It needs to have a main breaker to serve as the required service rated disconnect. It can be a 100 amp panel with 'main breaker' or it can have the backfed main option where you can install a double pole breaker with hold down kit. Many times the big box will have a value pack 100 amp panel with a main breaker very reasonable and comes with branch circuit breakers. This type panel will have a 'main breaker' like the one in your garage. The back fed type allows you to install your own main breaker which can be any double pole breaker designed for that panel. So if your running a 60 amp feeder from a 60 amp double pole breaker in your house panel you can install a 60 amp double pole breaker in the workshop panel. The idea is that the breaker in the workshop serves as a required disconnect for the detached workshop and the breaker in your house panel serves as the 'protection' breaker based on your feeder size. You cannot install a breaker that is more than
the rating of your workshop panel.
You need 4 wires feeding the workshop panel set it up just like your house panel is being fed from the main breaker on the outside of your home. Notice in that panel there is a ground bar fastened directly to the metal of the enclosure and all the ground wires go to it. Then notice the neutral bar is separate from the ground bar and has all the white wires going to it. Next thing you will see is that the green 'bonding screw' is not installed in the neutral bar. That's the empty hole you see next to where the big gray feeder neutral connects into the lug of the neutral bar. With that bonding screw not installed this isolates the neutral bar from the ground bar and prevents neutral current from getting the the ground bar via the panel metal.Your going to use the same principle in your workshop panel.
When you buy your new panel just repost here and we can tell you how to separate neutral and ground.
I highly recommend using pvc conduit and thhn/thwn wire. A 60 amp feeder like K buz said will be adequate for your needs. Using a conduit size that will accept a 100 amp feeder will allow you to upgrade from the 60 amp feeder if you want in the future. You just use the wires existing to pull in the new wires.
Use the drawing below as your guide and I would strongly suggest you have your work inspected by a qualified person. Understand that the electrical work your doing requires a permit at a small cost. Do not be afraid of inspection it will give you peace of mind.
The drawing is a little different than your situation. The SER cable would be the cable that is coming from your main disconnect to your sub-panel with main breaker in the garage and your panel would be inside not outside other than that the drawing is accurate. The drawing shows a pvc conduit installation. All conduit above ground needs to be sch 80 ... below ground can be sch.40. You can also direct bury but the cost difference is minimal.