Peninsula Main Gate
Talk to Gate Captain Kipp to learn about the prison break quest. He'll tell you to see Sedos Sebile at the Military Headquarters, located in the southeast of the peninsula.
Once you get past the safety of the guards, you'll be set upon by gangs of escaped prisoners. You'll also find a few unlit plague-victim pyres throughout the neighborhood. Equip your torch and set them on fire for a bit of experience.
Master Johns and Ms. Dulcimae
These two citizens are trapped in the streets and need your help getting to the gate. Escort them for some meager experience points. Be sure to talk to Master Johns before doing so--he has some interesting information about the Tanglebrook Estate, which is one of the possible entrances to the prison proper.
Arcane Laboratory
Near Ms. Dulcimae, in the northeast corner, is a locked building marked with a sundial. If you've accepted the Cloaktower Membership quest, you'll have the key for the door. Inside, you'll find the rare earth clay in a pile of stones.
Never's Tomb Excavation Site
Talk to Briley to learn about the excavation site. If you have the letter from Oleff, you can take the ceremonial arrow, ceremonial shield, and ceremonial sword from the nearby chest. These items will serve as keys to the other three tombs. You'll also find the Halueth dig logbook, which describes where to find the other tombs and describes the riddles that unlock them.
Militia HQ
Once you've visited all the other locations of interest, talk to Sedos Sebile in the militia headquarters. She'll ask you to find a way into the prison. She'll give you one lead: A gang of prisoners has been leading raids from the sewers and may hold a key to the prison's front door.
Near the headquarters is the Mizzenmast Mercantile, where you can stock up on supplies if you need anything. There is also a Priest of Helm nearby. Seems like these guys are everywhere and really aren't much help.
The Prison
Before you head into the prison itself, you'll want to quickly explore the prison barracks, located to the west of the large prison building. There isn't much here, but you can pick up some experience points and some loot.
There are two ways into the prison. You'll need a key to get in the front doors, but there is also an alternate, and more interesting, means of entrance into the Tanglebrook Estate. The latter path will give you some cryptic background about the problems in the prison, as well as an item needed for a henchman quest.