Cockatoo Island/Wareamah, a former penal colony and ... Plus, there's waterfront camping to fit all budgets (just be sure to follow these tips for booking your campsite in advance).