The time ball at the top of the tower at the front of the Observatory was the reason for the establishment of the Observatory and so is of great significance.It is still in its original condition apart from the addition of an electrical motor and some improvements made by HC Russell in the 1870s.

Before radio the only way to convey accurate time was through a visual signal. Accurate time was especially important to ships setting out on a long voyage as they needed to adjust and calibrate their chronometers, special clocks for navigation that were designed to work on a moving ship. In the 1850s Sydney was becoming a busy port with ships arriving from, and departing for, a variety of distant places. There was an urgent need for a time ball to provide accurate time for this shipping traffic. Initially only a time ball was contemplated, but soon it was realised a time ball without an observatory and an astronomer was useless. The astronomer was needed to observe the sun by day and stars at night to ensure that the ball was dropped at exactly the right time.

Built by the well known engineering firm of Maudslay Sons & Field in England, the time ball was installed in the tower of the new Observatory by June 1858 as it was first dropped on the 5th of that month at twelve noon. A few months later Rev Scott, the Government Astronomer, changed the time to 1pm for at 12 noon he was busy observing the sun crossing the north-south meridian and could not be upstairs in the tower dropping the ball.