We went swimming and snorkeling at Horseshoe Bay Beach and also the Fairmont Private Beach Club in Bermuda. Both have beautiful pink sand. There were some fishes in the water, but not much. These beaches were great for taking in some sun and enjoying the natural scenery. You can read the detailed account here.
The Crystal and Fantasy Caves are very beautiful. We had a great time exploring these caves in a guided tour. You can just do one of the caves if you like, but might as well see both since they are next to each other. We took a public bus, and walked to get there. You can read the detailed account here.