I would seal all the returns. My energy auditor had this as #1 or #2 on a prioritized list that was conducted earlier this spring. Situations are different for everybody and we're not aware of the layout of your house. But as for me, I have a 2 story and the suggestion made sense to me. Hard for a system to work correctly, pulling air from those rooms upstairs if there are several leaks in the basement. You're going to lose that air current upstairs when it's primarily taking it from the basement. IMO, it's the path of least resistance working here.
Once again, I'd go for the sealing all you can. You can use cheap caulk to seal this joints just as well. A very inexpensive fix that I feel is "bang for your buck."
To each their own though.