Scheduled Monuments are nationally important archaeological sites or historic buildings. Scheduling is the UK's oldest form of heritage protection, going back to the Ancient Monuments Protection Act 1882. The current defining legislation is the Ancient Monuments and Archaeological Areas Act 1979.
A scheduled monument can be any building or structure, cave or excavation on the land or on the seabed within UK territorial waters, or the remains of one. It must be a human-created site; purely natural sites are designated under other legislation.
From a legal perspective, what matters is the inclusion of a site in the Schedule of monuments - hence the word "scheduled". Not all ancient sites are scheduled, even though they may be monuments.
In practice, the terms tend to be interchangeable. The original schedule of ancient monuments comprised almost exclusively prehistoric sites, and the legislation uses the word "ancient" in the title. However, there are scheduled monuments which are much more recent, including WWII military fortifications and abandoned collieries.
Scheduling is the oldest form of heritage protection in the UK, and applies only to sites of national importance. Listing is more recent, and can be applied to structures which are only significant locally.
Listed buildings have to currently exist in something at least resembling their original form. They cannot be completely ruined, or invisible. Scheduled monuments, on the other hand, can be total ruins, and do not even have to be visible - they can be subsurface remains.
Scheduled monuments cannot be buildings which are currently occupied for residential purposes (unless the residential use is purely ancillary to their heritage use, for example a caretaker's apartment at a ruined castle). But currently occupied residential buildings can be (and often are) listed.
Some structures are both Scheduled Monuments and Listed Buildings. Typically, these tend to be intact (or mostly intact) early religious or military structures, such as churches and castles.