Giving you guys an update.
HRG - water pressure gauge read 90 lbs. Shall I proceed to replacing the pressure regulator and ball valve?
You could try adjusting your existing PR but considering that it's about 35+ years old, I would definitely just replace it without even trying to adjust it. I bought a Watts brand PR for our home. It looks like the one that you currently have (except mine has a union coupling on its input side) so yours might be a Watts PR. The new PR should be adjusted for 50-60 psi output.
If you replace the PR, I would also replace the ball valve since they are relatively inexpensive and I guarantee that you will feel much better after the job is done. There are two types of ball valves. One type is soldered in on both sides, the other type is threaded on both sides. I personally buy the threaded type since I don't want to overheat the ball valve during the soldering process. In the threaded type, the male fittings are soldered on to the copper pipes first and then the ball valve is screwed on. --- But in your case, you may have to install a soldered in type of ball valve due to minimal space from the ground to the horizontal pipe.
I would remove the white pressure relief valve that is installed right next to the wall. Your water heater should have a pressure relief valve on it so that one in your photo is not really needed. By removing that pressure relief valve, you can install a union coupler in its place. Then make sure that the pressure regulator you buy has a union coupling on its input side (the side connected to the ball valve). This will allow you to remove the pressure regulator and hose bib assembly in case you ever have to replace the PR in the future. Otherwise you will have to have enough clearance to the wall to spin the PR to remove it in the future.
Be sure to measure the diameter of your existing pipes before you go shopping so you can buy the correct size PR, ball valve, copper pipe, fittings, etc. Don't try to reuse your old pipes or fittings. It's just too much work with more chances of leaking. Don't forget to measure the distance from the ground to the horizontal pipe so you can buy a PR and ball valve that can fit in that limited space.