How do I set it up so that instead of having my blogs as individual files, I would have them as folders, then my links would display it as a folder and not /index.php

Your individual entries will have to be pages. The catch is that if the page is named as an index file (this defintion varies from host to host), then you don't need to specify the index file, and can access it via directory instead. Assuming your host supports index.php as an index (ie, it shows that file if you access just a directory), then simply change your individual archive templates so that they go to a folder name. For instance:

Would put each entry into it's own directory. You also might want to look at something like Mark Pilgrim's "cruft free" URLs post, though it will require a bit more technical expertise. That may or may not be a bad thing.