Only one sword?
This is second hand information (as in I haven't read the books), but in the books Geralt doesn't typically roll around with both swords on his back. He keeps his silver sword on his horse since he only needs it for specific situations versus the general usage of the steel sword.
In the game of course it was different because not always carrying both swords would have been a pain.