You've finally found Herbert the Hermit but in order to get your hands on the last piece of the compass, you're going to have to run an errand for Herbert. You can get this mission after completing the mission "The Hermit".
True Vault Hunter
Gear: Grenade or SMG
From Herbert's place, drop off the ledge to the lower level. Continue past where you originally found the anchor and you'll go under the large rusty boat. You'll come out to another section of the Rustyards called the Coobie Gorge. Fight your way through the various Spiderants here and you'll eventually find a bridge that will lead out over the gorge to your left. Take this bridge and then continue along this ledge until you find another bridge to the left. This bridge will lead out to the middle of the gorge where there is a ship. Board the ship and look around to find the tapes… as they get scattered around the area.
You'll have to find the four tapes in this area. You can head back across the two bridges and turn left to find the first tape by some chests on the ground. On the ledge between the two bridges, you'll see a large ship. You can enter this ship if you follow the path on this ledge to the large gate. As you approach, you'll encounter two Anchormen coming through the gate. Kill them and the raiders that come through just behind them.
Now that you are inside the ship, check nearby to find a "Message in a Bottle". To get it, you'll have to climb on the desk in the room or go over the roof to come down on top of it.
Keep going through the ship and the next tape is going to be on one of the top levels. As you approach the waypoint, you'll see a Gun Turret on the top floor. The tape is going to be near that but you'll want to make sure you take it out first. From this tape, leave the ship by taking the nearby bridge to the south.
This bridge will take you to another part of the Rustyards called the Dreg Scrapyards. The third tape will be part of a scrap pile near where you enter the area. You'll have to climb up on one of the shipping crates to get to it but it will be it will be easily accessible once you get up there.
From here, go around the west side of the Scrapyards and try to find a scrap pile and ramp that will lead you up to the next ledge. This will run around the side of the cliff until you are hanging out just above the large battleship below. Grab the last tape and then take them back to Herbert.
He'll tell you that the compass is in the chest nearby. Blast the lock off and open up the chest to find the last piece of the compass… smashed. Talk to Herbert to find out what to do next.