There are dozens of places you can visit in Sydney from a Circular Quay ferry, and one quick trip is Cremorne point. This peninsula is walkable to North Sydney, popular with expats, and has beautiful views of Sydney harbor. There is a park running the circumference of the peninsula where New Years eve spectators gather to watch the fireworks show. If you time your visit right, you may catch the harbor at low tide and be able to walk straight across to Kirbilli.
When we were applying for apartments, one of of top three choices was a mansionette in Cremorne. This is a large home that has been sub-developed into four or five apartments. Milson road is lined with them and it is a rare opportunity should something become available. The 2km stroll along Sydney Harbor is well paved, passes Sirius playground, and ends in the ferry trip back to circular quay.