Either way is ok. If you want to cover the walls inside, I'd recommend plywood of some sort rather than drywall. Makes for a sturdier job and you can hang all sorts of stuff on plywood easier than drywall....
You sound like my parents. I hate that thinking, you can NEVER find anything because every square inch gets filled with crap! I like things best clean and organized and if it don't fit, you'll probably never need it anyways.
I cover the walls of sheds/workshops/garages with the 1/4" thick 4x8 sheets of pegboard. Makes for lots of storage options and looks nice (for a shed / workshop). You can get it already painted white - so no painting needed (I hate painting...). Quick and easy to put up. Screw it to the studs with drywall screws, rather than nail - makes it easy to remove as may be needed to add electric or something else.
A forum community dedicated to Do it yourself-ers and home improvement enthusiasts. Come join the discussion about tools, projects, builds, styles, scales, reviews, accessories, classifieds, and more! Helping You to Do It Yourself!