QUOTE
An earlier build of Firefox 0.8 that I used had issues with the Tabbrowser Extension and didn't open up any popup windows correctly from javascript buttons, including those in the MT interface. Is it possible that you have that extension running? Now I'm using it successfully - this is my info: "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040206 Firefox/0.8".
I am running TBE, however it seems to be opening the popup windows fine (after I unblock my site

) so I am not sure if this is the problem.
I will try later with a fresh install of Fx without TBE