I tried going to Shell integration and click on "Check Now" and its still opening torrents in RT torrent program which is suck please help

My guess is that you'll have to go into your RT program and find whatever setting there is for making RT your default torrent program and unchecking/disabling "make rt your default torrent program" first.

what operating system?

If windows, there are 2 places that will affect this. In firefox, under options/applications there will be a setting for how you want ff to handle .torrent files when you download one. Set it to ASK each time or to always use Tixati.

The other place is in windows explorer, under tools/folder options/file types/ scroll down the list and select TORRENT then change settings under  section to

I don't see .torrents as a file type in Firefox's  Options/Applications mask.

Is there a way to add it?

I currently have to tell windows 7 to use tixati to open torrent files (also have to find it in the directory with every torrent)