I have collection of many songs , some are 96kbps bit rate , some are 128, some 256kbps bitrate but whenver I hear 320kbps bitrate mp3 song, they are generally better and have deeper quality. I want to convert low bit rate song to high bit rate of 320kbps.

I need to know:
- Is there any good free software to do this , may be some open source?
- Will it actually increase quality of sound of the songs
- What is difference between 256kbps and 320 kbps in normal layman's language because I find it sometime hard to distinguish between 256kbps song and 320kbps song..

well, obviously no. You can't increase bitrate of any song.
You might want to download the same songs again in 320kbps.

How much song information is stored per second is the bitrate, so 96kbps means, it will store 96kb data per second, and 320kbps bitrate means, it will store 320kb of data per second, now when you convert from 320kbps to 96kbps, rest of the data or song detail is lost. And if you again convert to 320kbps from 96kbps, you will not gain anything in terms of quality but only you will get bigger size mp3 files.