After looking at your source, it appears that you do have the right code for it being a popup on your page. But when I click the Trackback link, I get a fast javascript error as it goes to the next page (so fast that I can't click it.) I'm wondering if the script in your head before the function OpenTrackback isn't quite closed and so its not letting the Trackback open correctly?
Maybe try moving the 2nd } up above function OpenTrackback? I'm not very up on my javascript, so I'm not sure, but like I said, everything looks like it should work as far as the function and the link code

If that doesn't do it, post back and someone else smart will look at it

Kristine