Good places for swimming/cliff jumping/hiking around albany?

i stopped by here yesterday on the bike, all the rocks sketched me out a little bit…its a nice place though. Do people do the bridge or the lower points?