I'm having a serious issue and all help is appreciated.

NOTES
-- RackSpace server running Linux, Apache, MySQL and PHP (dedicated and managed server worked great with MT 3.3)
-- Changed PHP .ini settings to allow PHP Memory to 32MB
-- Mt4.1 Commercial with Pro Pack 1.1
-- Plugins
-- Better File Uploader
-- Cache Block (currently disabled)
-- Fast Search Pro
-- Pagination Pro
-- Template Exporter
-- Template Installer
-- Visitor Stats Pro
-- 10 blogs set at Subdomains (subdomains.maindomain.com) all subdomains access the same MT @ maindomain/cgi-bin/path to mt/mt.cgi


I bought MT 4.1 Commercial w/ Pro Pack 1.1 and played with it in a test environment for a month or so (fresh install). Everything appeared to be working so I went for the upgrade.

I upgraded to my MySQL database (5,000 entries / 8,000 comments) and everything appeared OK at first.

I started tweaking templates more (on 1 Blog) using Include Modules and Widget Sets. Mostly the MT templates were kept "stock" utilizing <MTIF> statements to include columns. widget sets,etc on different template pages (other blogs kept old design but accessed MT4 to update and post). The Dashboard (UI) appeared to get slower over time - taking longer and longer to switch between blogs and rebuilds.

FIRST SIGN OF TROUBLE
I suspected "build issues" and bought Cache Block plugin. After Cache Block - Pagination failed to work properly - but rebuild times seemed much better. I removed Pagination code from my templates - but Cache Block was giving some strange results so, I disabled Cache Block. After reinstalling Pagination code - it would only work on Category pages but NOT Main Index.

I began to wonder if the 'old MT" code on my other blogs might be affecting this in some way. Plus I began noticing problems since some blogs were accessing /oldmt/mt-comments.cgi and things like that. Plus scheduled entries would not post (new MT CRON JOB) - so I decided to take the plunge and start getting the new templates on the old blogs.

After updating 3 blogs with the new template pages things quickly got worse in the UI / Dashboard. 500 internal server errors when trying to change to another blog or open templates.

Within hours the Apache load became so great that i can no longer login to mt.cgi. I have restarted Apache and MySQL but as soon as we do, the Apache load grows faster than I can get into the templates (I Was able to login to the UI for just a second).

With the help and suggestions of Jay Allen I suspect that I have some template issues. I use to have all my 'parts' used on templates using SSI/PHP. I think if I can clean the templates (or at least get them back closer to default stock) things will be better.

However - I can't get into my mt.cgi.

Does anyone have suggestions?

I thought about editing the templates via the database (not happy about that - but I can access the DB). If I could pull that off... how would I rebuild the site?

Or how can I install default templates to 'certain' blogs and rebuild sites without mt.cgi access?

Any and all help is apprecited

You have most likely considered both of these, just adding my 2 cents.
maybe:

Export the database, and if you have an editor that can handle large files (I have found cutehtml does) perhaps doing a find/replace, might be able to disable plugins that way as well?
or;
Install a new db and fresh MT install in another folder and build it up gradually, then replace them/swap/whatever.
Just off the top of my head.
I'd definitely go with index files and php includes.
Til someone here has a better idea, which should happen...
.

Thanks for the thoughts Bruce. I deleted the installation and did a fresh install. I was able to dummy down the templates to work... sort of...

mt.cgi and mt-search and mt-comments are killing Apache.

I installed the Fast Search from MT-Hacks.com to eliminate the mt-search - that helped

Then went after Comments - I just disabled that for now (users love me!)

Apache is still having problems - So I enabled all the cache' - background publishing stuff... and the site itself seems stable.

Admin area is still full of 500 Server Error messages. Rackspace thought it might be a permission problem - so we reassigned Author permissions and deleted custom permissions picked up from upgrade. Nothing changed too much

Background Publishing doesn't appear to be working. What else??? Oh I met with Rackspace today... we're looking at upgrading the server some more (mind you this one worked well under MT 3.36). I'm also looking into putting the application on its own server - once I can come up with a good story to justify why I should do this I'll probably drop some hard earned cash on a new server...

I still trying to remember why I upgraded.... what a money pit this has turned into for me!