That would do it, try latex caulking, apply a small amount and wipe with you finger then use a damp sponge for a smooth finish.
The key is small amount, there's no need for any of it to be on the walls, just in the crack.
The key is small amount, there's no need for any of it to be on the walls, just in the crack.