The title pretty much says it all. I'm just trying to figure out a way to create a list of all the entries under a specific category that will link to the actual entry. Thought it would be simple (of course) but haven't had any luck.

Modify your Entries loop with the categories argument.

there is your answer :

if you use the code below , you can show last n entries belong to a specific category.

of course you have to use your own category name and number n