Adam Gifford was born at Edinburgh, Scotland on 29 February 1820, the eldest son of James Gifford and his wife Catherine Ann West. His father, rising from a comparatively humble position, became treasurer and master of the Merchant Company, an elder in the Secession church, and a zealous Sunday-school teacher. His mother was the only teacher of her sons Adam and John, till Adam was eight years old, when the boys were sent to learn Latin and Greek at a small school kept by John Lawrie. Later Adam Gifford was a pupil at the Edinburgh Institution.