If you can find 64bit drivers for ALL your components, maybe not... but you won't get any performance benefit from 64bit vs 32bit because as far as I know, there aren't any 64 bit versions of any games.
If you're running game SERVERS, or doing stuff like CAD, binary compiling, or using other professional applications that have 64 bit versions, then it may be worth using 64bit, but otherwise it'll be useless to you.
I have been using a 64 bit version of Windows XP for over a year. I have had a few problems with getting hold of device drivers, but for the most part this hasn't been too bad. Most major manufacturers supply 64 bit drivers, its only old hardware or obscure cheap web cams that I have had trouble with.
Virtually all older 32 bit programs will work fine. The only time you could run into trouble is if you try and run very old 16 bit or DOS games that are no longer supported. Even then the are ways round the problem by using emulators or dual booting. I run an awful lot of software and games and its very rare to find one that won't work.
DreamThrall is mistaken, some games do have 64 bit modes. Both Half-life 2, (and probably other source games) and Far Cry run natively in 64 bit. I suspect Crysis and future Valve products will support 64 bit. I don't know about other titles but they may also support 64 bit
The reason I switched is that in the long run 64 bit is likely to become more widespread and i like to take full advantage of my hardware. You may not get an improvement in performance for a lot of things, but the are a few programs that could get full benefit.
I have the orange box so if I install Vista 64 Ultimate will STEAM automatically download updates to make use of the 64bit OS? Also will I see any performance increase when playing games that utilise 64 bit?