Slakker, thanks for your reply!
It is indeed not closing all the way.
I don't know if the pictures below will show it clear enough. The bottom beige thing is the window sill. I have made 2 color block to show how far the window would go inside the bottom thing (I don't know how to call it).
The left picture is the good window. You can see the window goes in nicely, so the height is small. The right picture is the "bad" one. It doesn't close all the way, so the height is bigger. I don't know if I explain it well.
Since it is easy to check if there is anything on the bottom that is hindering the closure, I had checked and found nothing, so I am suspecting something might be in the side rails, but how can I check that? Can the side be opened?
Any suggestion would be appreciated. I have lived in an apartment for 25 years, and in an apartment you would just called the maintenance guy, so I never paid attention, and I never had a window problem in an apartment either.