At the centre of every great port is the organisation by which the government attempts to keep control of trade, and in London this was centred on the Custom House.
& 1275:  The first known Custom House was built.

Custom House on right, 1560 (Agas Map 1633)

FitzStephen (12th century) -

To this City, Merchants bring in wares by Ships from every Nation under heaven. The Arabian sends his Gold, the Sabean his Frankincense and Spices, the Scythian Arms, Oil of Palms from the plentiful Wood: Babylon her fat Soil, and Nilus his precious Stones: the Seres send purple Garments: they and Norway and Russia Trouts, Furs and Sables: and the French their Wines.

1375: (from 'Rambles among the Rivers' Charles Mackay 1835) -

... one of the first comptrollers of the customs for the port of London, probably the very first, was no less a personage than Geoffry Chaucer.
This office, a very lucrative one, was bestowed upon him by King Edward III in the year 1375. The articles chiefly under the superintendence of the poet were wool and hides, with a proviso that he should personally execute the office, and keep the accounts of it with his own hand.
In the year after Chaucer's appointment great peculation was discovered in other branches of the customs, and many of the offenders were discovered and prosecuted.
Not a word of complaint, however, was ever breathed against the father of English poetry.

1378:  Custom House rebuilt
1559:  Rebuilt

The Custom House in the time of Elizabeth I, as imagined by The Saturday Magazine, 1834 -

Custom House
The Custom House in the time of Elizabeth I,
as seen by The Saturday Magazine, 1834

1657: Henry Belasye -

The commodity of the river and boates, the prodigious bridge, the due and dayly visit of the ebbing and flowing of the sea in the Thames, which, visiting London duly once a day, either bringeth to it or carryeth from it, all merchandize the world can afford it or it the world. The greatest ships that ride upon the sea come and unload in London in the very harte of the town.

1666:  The Custom House was destroyed in the Great Fire of London
1669-71:  A new Custom House was built by Sir Christopher Wren

1705: Custom House in Prospect of London, Johann Baptist Homann -

Custom House, Homman 1705
Custom House in Prospect of London, Johann Baptist Homann, 1705

Custom House, Wren 1669-71
1714: Christopher Wren's Custom House.
To the Right Honourable the Comissioners of His Majesties Customs
This prospect of the Custom House
is humbly presented by your  Hons. Most humble servant

1714:  The Custom House severely damaged by a gunpowder explosion
1717-25:  Rebuilt on Wren's foundations, Architect Thomas Ripley.

1724-1726: Daniel Defoe, "A Tour through England and Wales" -

The whole river, in a word, from London Bridge to Blackwall, is one great arsenal, nothing in the world can be like it: The great building yards at Schedam near Amsterdam, are said to out-do them in the number of ships which are built there, and they tell us, that there are more ships generally seen at Amsterdam, than in the Thames.
As to the building part, I will not say, but that there may be more vessels built at Schedam, and the parts adjacent; but ... you do not see more ships, nor near so many ships of force, at Amsterdam as at London.
That part of the river of Thames which is properly the harbour, and where the ships usually deliver or unload their cargoes, is called the Pool, and begins at the turning of the Limehouse Reach, and extends to the Custom House keys: In this compass I have had the curiousity to count the ships as well as i could, en passant, and have found above two thousand sail of all sorts, not reckoning barges, lighters or pleasure boats, and yachts; but of vessels that really go to sea.
It is true, the river or Pool, seemed at that time, to be pretty full of ships; it is true also, that I included the ships which lay in Deptford and Black-wall Reaches, and in the wet docks wheoreof, there are no less than three; but 'tis as true, that we did not include the men of war at the King's yard and in the wet dock at Deptford, which are not very few.

1755:  Print by John Stow, but I think this is the Wren building before 1714, which is why I have placed this print before the Maurer copy of 1753

The Custom House, 1755, John Stow’s Survey of London
[I think this is Wren's building before 1714]

1753: A copy of Maurer's print of the Custom House in London on Thames by G H Birch. This must be Thomas Ripley's building -

'A view of the Custom House with part of the Tower,
taken from ye River Thames, London, after Maurer, 1753

Paul Hentzner -

The wealth of the world is wafted to London by the Thames, swelled by the tide; and navigable to merchant ships through safe and deep channel, for sixty miles, from its mouth to the City; its banks are everywhere beautified with fine country seats, woods and farms

1799: Print of Thomas Ripley's Custom House -

Custom House 1792 © MOTCO
Malton’'s Picturesque Tour 1792, The Custom House. May 29th 1799

1808: Print

Custom House 1808 © MOTCO
Ackermann's Microcosm Of London (1808) Custom House,
from the River Thames. J. Bluck Aquat. 1 Augt 1808

1813:  a New Custom House was proposed

To obviate the great inconvenience arising from the inadequate size of the former building; and to concentrate various departments of this branch of the revenue which before were, for want of room, necessarily distributed in remote situations. The astonishing and rapid increase of the commerce of London, and the country in general, had long since called for the adoption of this measure, to afford the requisite facilities to the business of the revenue, and to accommodate the immense concourse of commercial men of all nations who are congregated at this spot; also to confer a suitable dignity on so important a branch of the public service, and keep pace with the rise of national opulence.  After much deliberation on the expediency of altering and enlarging the old custom-house, the project was abandoned as impracticable, to the extent required; and the present structure, as designed by Mr. Laing the architect, was ordered to be erected on the adjacent ground towards Billingsgate dock, which was then covered with numerous streets, quays, and warehouses, chiefly belonging to the crown. It was thus proposed to have removed the business from the old building to the present one, with scarcely any interruption;

1813:  October 25th

The first stone of the new building was laid (being the 53rd anniversary of his majesty's accession to the throne) on which occasion Lord Liverpool officiated, attended by some of his colleagues in the administration, and the Commissioners of the Board of Customs.  In the stone was deposited a glass urn, containing the several current coins of the realm; various medals, illustrative of the great events and personages of the present era; and one engraved with an elevation of the building, inscribed on the reverse with the names of the commissioners, secretary, and architect. On a brass plate inserted in the stone, was also an inscription of the date, with the names of the founders, &c.

1814: There was a terrible fire on February 21st.
However in March, S Owen drew the old building as it was before the fire -

Custom House 1822 © MOTCO
Cooke's Views On The Thames (1822)
The Custom House [before it was] Destroyed by Fire Feby 12th 1814
Drawn by S. Owen. March 31, 1814.

1814: February 21st, The dreadful fire destroyed the old building -

Custom House Fire 1814
The Custom House Fire, 1814, Calvert

1814: The fire frustrated plans for an orderly transfer of business from the old to the new.  Many of the trading records of the Port of London were lost in this fire.  The new building was completed in haste - which was to have consequences later.

1816:  Print of the New building

Custom House 1816 © MOTCO
Papworth's "Select Views Of London" (1816)The New Custom House.
Built by David Laing, 1813-17. July 1,1816, published in the Repository of Arts

1814: The New building was described, the writer particularly emphasizing "the unqualified approbation that the architect ... so well merited" ... -

Custom House 1817 plan
Laing's Custom House Plan 1817

The general character of this building is that of plainness and solidity, being chiefly designed for the convenience of business, which it so extensively comprises;
but from its great magnitude, and the simplicity and just proportions of its parts, the effect is grand and imposing. ...
The extensive and complicated combinations which this edifice comprises, must obtain for the architect
that unqualified approbation which he has so well merited.

1820: New Custom House by Robert Havell, showing Laing's frontage -

Custom House 1820, Havell
The Custom House, Robert Havell, 1820

1825: New Custom House still showing Laing's frontage -

Custom House 1825
The Custom House, still showing Laing's frontage, 1825

On the 26th of January, 1825, a portion of the floor of the Long Room, about forty feet across, and twenty lengthwise to the apartment, gave way between the hours of nine and ten in the morning, and fell into the King's Warehouse beneath.
This accident, by which, very fortunately, no person was injured, was occasioned by the arches of brickwork supporting the floor not being sufficiently strong, and also weakened by the loosening of the earth by the previous high tides.
Notwithstanding that the actual damage was confined to an inconsiderable portion of the entire area, it was deemed expedient to rebuild the Long Room, and basement beneath it; and Mr. (now Sir Robert) Smirke was entrusted with the execution of the repairs necessary to put the building into a state of perfect security.
He did not, however, think proper to adhere to the design of his predecessor; in consequence of which the centre compartment of the river facade has become altogether different ...

Oh dear! Laing was professionally ruined, and the reconstruction work was undertaken by Sir Robert Smirke: the Portland Stone façade of a six-columned Classical portico, found on the wings of the riverside face, was repeated in the centre, projecting forward to the Thames
[See where "unqualified approbation" gets you! - I wonder if David Laing couldn't stand the look on Sir Robert's face (smirk?) ]


Custom House 1836
Custom House, 1836

1837: The penny Magazine of the Society for the diffusion of Useful Knowledge -

Custom House 1837
Custom House, 1837. Coping with wash!

1842: The Custom House in Cruchley's Picture of London -

Custom House 1842
Custom House, Cruchley's Picture of London, 1842

The Custom house is 480 feet in length, and 100 feet in breadth: it affords accommodation to between 600 and 700 clerks and officers, besides 1000 tide-waiters and servants.
In its architecture, which is of the Ionic order, there is nothing particularly striking or remarkable. The interior, which may be freely visited every day from nine to three, will furnish much to gratify the stranger in the number of offices, all in active employment, and the apparent facility with which business of importance is transacted.
The ground floor is principally occupied by Her Majesty's stores; and on the first floor is the long room, 186 feet in length, which is an object of universal curiosity. At the east end of the ground floor is the searcher's office, with accommodation for the examination of foreigners and their baggage.
Aliens arriving in London, or at any other port, are, by the Act lately passed, entitled, "An Act for the Registration of Aliens," simply required to present themselves before the officer of the Customs, appointed for that purpose, at the Custom house of the port at which they land, that their names may be registered, when a certificate of arrival is granted them, which certificate they retain during their stay in the United Kingdom of Great Britain and Ireland ; on which certificate is a note, directing them to deliver it to the said officer on departure. By this plain and simple arrangement, all the objections and difficulties formerly connected with an Alien Office, and so much complained of by foreigners, are completely removed.

1842: The Custom House and the Tower from London Bridge -

Custom House 1842
The Pool. From London Bridge. Morning
Published by Henry Brooks, 319, Regent Street, Portland Place and 87, New Bond Street, April 15th, 1841. Printed by C. Hullmandel
Laden paddle steamers against a background of the Custom House and the Tower of London.
Towers of St Dunstan in the East on the right and St Olave, Tooley Street on the left.

1845: The Custom House in the twelve foot wide "Grand Panorama of London from the Thames" -

Custom House 1845
Custom House in the "Grand Panorama of London from the Thames", 1845

1896: The Custom House -

Custom House 1896
Custom House, 1896

1918: A Dictionary of London -

Custom House On the south side of Lower Thames Street. In Tower Ward. Erected on this site 1814-17. Architect, David Laing, original centre taken down and present front erected, 1828. Architect, Rob. Smirke. -The original Custom House further east, was on the site of the house erected by John Churchman in 1382, 6 Rich. II., on the key called "Wool wharf," between tenement of Paule Salisberie east and the lane called the watergate west, the King granting that the tronage of wools should be kept in this house and a counting place for customers, etc. Burnt in the Fire and rebuilt by Wren, burnt again 1714-15 and rebuilt. Another fire occurred in 1814, when the new building was erected on the present site. The present building occupies the site of a number of wharves and keys, etc.: Bear Key Stairs ; Bear Quay ; Crown Key ; Dice Key ; Horners Key.

John Burns -

One hundred thousand men, dockers, stevedores, lightermen, sailors, and kindred calling depend upon the Port of London; and all of them subsist and owe their livelihood to the bountiful favour of Father Thames.

1930s: Lure and Lore of London's River by A G Linney -

For London is the lodestone which draws towards itself the wealth of the world in all manner of forms: London River [ie downstream of the Pool of London], the River of London [ie the river in the City], and the River Thames just beyond [upstream of] London provide a peerless highway for the conveyance of that wealth; and the bringing, receiving, storing and carriage elsewhere keep busy the hands of the men of the Port. ...
Because London is the lodestone of the world, there is hardly a country of the world, no matter how distant, which is not continually putting forth effort to fulfil some of her needs.
In answer to London's behests, men are toiling far down in the bowels of the earth and fathoms deep below the surface of the sea; they are speeding through the air faster than arrows; they are subduing deserts and felling forests; they are living lonely as hermits or crowded as ants; they are flirting with death or dozing where the lotus blows; they are scooping fish from the sea, stalking savage beasts in the jungle, snaring birds from the trees; councils of wiseacres are scheming, masses of simple primitives are fetching and carrying, and thousands of thoughtful and prudent folks are giving honest labour - all so that the ships may bear to the Port of London those things which she requires.
At the moment the trapper beyond the Arctic Circle is moving along the crackling white carpet on his snowshoes; the trader on the South Pacific Isle is weighing copra in his store; the cattle man in the Argentine is watching his herds; camel trains in the Gobi desert are bearing strange things in bales across the sands; hunters are tracking elephants in the forests of the Congo; the pearling fleet is out from Thursday Island; the shearers are busy in the Queensland sheds; the pulping mills are full of movement in Newfoundland; they are scoring rubber trees in Malaya; patient hand weavers toil at their looms in Benares; in Amsterdam the craftsmen transmute to beauty the stones dug by the kaffirs from Rand mines. These and a thousnad more manifestations of human energy find their fulfillment for the Port of London.
Crazy dugouts slip down streams in savage lands to bring their packages to tidewater; schooners drift between South Pacific Isles; canoes paddle down great West African rivers to the sea; dingy tramp steamers battle across the waters of the world; solid freighters and elegant passenger and mail boats defy Neptune's threats - all because of London's commands.
So the mighty magnet draws them home with their freights from every port of the Seven Seas, draws them to that furrowed estuary facing the North Sea where the well dug channel gives sure passage to London the Lodestone.

1940:  During World War II, the Custom House East Wing was heavily damaged by bombing
1962-66: the East wing frontage was rebuilt in replica.

2020:Plans to turn the historic Grade 1 listed Custom House building into a hotel. HMRC, the present users, plan to leave in 2021. The external changes look to be small.

Custom House Plan 2020
Custom House, Planning Application, 2020