There is, in fact, a corner stone in the basement that bears the initials of George Washington's older half-brother Lawrence. In the movie, the center of the corner stone has an arrowhead, while in reality it resembles more of a heart shape. Of course, in the movie, the stone is at the end of a short tunnel, and the Secret Service destroys it with sledge hammers in order to find the newly "kidnapped" modern day president.
The last location is a small beachhead found near the wharf. Shown in the dark, here is where Ben Gates emerges from the Potomac in his wet suit before crashing the party for the president's birthday.