In 1876 he resigned the presidency of Seton Hall College.
In the west end of the town is Pinkie House, formerly a seat of the abbot of Dunfermline, but transformed in 1613 by Lord Seton.
On the site of St Mary's (1837-1839), also Gothic, stood the small chapel raised by Christiana, sister of Robert Bruce, to the memory of her husband, Sir Christopher Seton, who had been executed on the spot by Edward I.
There are harbours at Morrison's Haven to the west and at Cockenzie and Port Seton to the north-east, which practically form one village, with a population of 1687.
On the Sunday following, Mary left Edinburgh for Seton Palace, 12 miles from the capital, where scandal asserted that she passed the time merrily in shooting-matches with Bothwell for her partner against Lords Seton and Huntly; other accounts represent Huntly and Bothwell as left at Holyrood in charge of the infant prince.