As time passes, does our understanding of a historical event get better or worse? It seems intuitive that something is harder to understand the longer ago it was. You probably have a relatively easy time understanding events from the presidency of Barack Obama, while contextualizing events and tracking down sources from the presidency of Woodrow Wilson will require more work. Events from the reign of Henry VIII will be even more foreign, and they often suffer from large gaps in the historical record.

This isn’t just a problem for individual researchers—entire societies can forget history. The march of entropy destroys witnesses, artifacts, scholarly records, and all other evidence of the past. Entire civilizations are lost. For thousands of years, scholars assumed the Hittites mentioned in the Old Testament were a minor power at best, until nineteenth-century archaeology revealed them as one of the great Bronze Age empires, who were nevertheless forgotten by Late Antiquity. Sometimes knowledge is rescued from oblivion by lucky finds, but usually it is not. We’re far from finding everything worth finding. We never will. Evidence doesn’t wait for us patiently—it is slowly destroyed by human activity and the forces of nature.

Many excellent historians, however, hold the opposite view. It’s very common to see historians implicitly or explicitly assert that knowledge in their field increases over time. For example, in his 1962 masterpiece Medieval Technology and Social Change, Lynn White Jr. assumes greater clarity from archaeological discoveries are yet to come: “Despite prodigious labours by Hungarian archaeologists, the stratification of Avar materials is not yet clear…[Avars] may well have been the first people of Europe to use the stirrup, but the time of its arrival is still uncertain.” Meanwhile, in a more recent article, nonprofit founder Jason Crawford writes, “I note at the outset that this is an old book, published 1925 and revised 1940. Probably a lot has been learned in the last 80 years and the following has already undergone revision, which I’ll uncover when I read more modern sources.”

The historian’s optimism rests on three promises. The first, expressed by White above, is that there are lost artifacts that can be recovered. Secret government records can be declassified, new construction will dig up an ancient tomb, a statesman’s grandchildren will find old letters in the attic and give them to a university, or archaeologists will find the ruins of an ancient temple complex. Such finds improve our understanding of the past, sometimes dramatically.

The second reason for optimism is that historians make better analyses of existing data as time goes on, as Crawford mentions above. After they make inferences from the available material, subsequent historians can take their best arguments and build on them while discarding flawed ideas which do not stand up to scrutiny. By standing on the shoulders of giants, the field will climb higher and higher, like in hard sciences such as physics or biology.

The third reason for optimism is the continued unfolding of history. After all, it is harder to see how an event fits into ongoing trends before those trends have had a chance to play out—time gives us perspective, and hindsight is 20/20. However, while the passage of time may give us a better understanding of a historical event’s effects on the future, it does not improve our knowledge of the event itself. Despite this limitation, knowing what happens next can make it easier to understand which events were important and why.

Archaeologists and Historians Can’t Defeat Entropy

If these three promises are met, then our knowledge of history is steadily increasing. How, then, could past events be so hazy today? Shouldn’t centuries of new finds, ongoing analysis, and knowledge of subsequent history mean that scholars of Henry VIII’s reign know what happened during that period far, far better than scholars of more recent events like the 2008 financial crash or the two world wars? Of course, we usually see the opposite.

These optimistic historians are writing epistemic checks that cannot be cashed. What the three promises leave out is that information is often lost. Firsthand witnesses and expert historians die after passing down only a fraction of their knowledge. If you investigate the 2008 financial crash today, chances are you can still interview someone who worked in finance or government who will give you information that has never been recorded. The information stored in people’s minds is still fundamentally accessible—for now. In a century, much of this information will be irretrievably lost. 

In addition to people, books and artifacts are also lost to entropy in a hundred different ways. The cumulative effect of this destruction is immense, as illustrated by the records of classical civilization. “[T]oday we possess written fragments from only 13% of the ~2,000 ancient Greek authors known to us by name. This does not account for the authors we do not know, and only a small portion of the 13% figure consists of complete works.”

Preserving the ever-growing mass of historical material is too expensive to be practical, so when budgets run thin, even major libraries and archives will discard books and records by the hundreds of thousands. For example, the Manchester Central Library’s recent culling destroyed 210,000 to 500,000 “literary, commercial, educational and political records going back 150 years” with “no subject specialists involved in the process.” This is a standard library practice.

Artifacts are also lost in accidents like the 2018 fire that destroyed 92.5 percent of the 20 million items stored in the National Museum of Brazil, including the only recordings of now-extinct languages. Another example is the 1986 Los Angeles Central Library fire that destroyed 20 percent of the collection and damaged much of the remainder.

In recent decades, digital information has fared no better than paper. Between link rot and changes in software standards, tremendous amounts of digital information become inaccessible over the course of a single decade. The long-term preservation of digital archives remains a hope rather than a guaranteed fact. Even in optimistic scenarios, it would require ongoing effort and maintenance on par with the curation of printed information. As the development of the printing press illustrates, much better ways of recording information can often have only modest effects on how much information gets preserved centuries later.

Preservation is not only a struggle against entropy. Political actors will often deliberately destroy knowledge of their foes’ history and culture, such as the shelling and destruction of the National and University Library of Bosnia and Herzegovina during the siege of Sarajevo, the Chinese Emperor’s “burning of books and burying of scholars” in 212 BC, or the Taliban’s 2001 demolition of the ancient Buddhas of Bamiyan monuments.

Most worrisome of all, knowledge and artifacts are often quietly forgotten. Naturally this abandonment usually leaves no trace, but one dramatic demonstration was uncovered a century ago. Around 500 BC, the Neo-Babylonian princess Ennigaldi-Nanna’ created a museum of then-ancient artifacts dating back as far as 2000 BC. When the city of Ur was abandoned, the museum and its contents were also forsaken. It was not until 1925 that modern scholars rediscovered the museum’s remains. Archeologists shouldn’t just delight in such finds, but reflect on the temporary nature of their own work.

Most losses, of course, are never recovered. Buildings and cities are razed, rebuilt, and abandoned. Rome wasn’t destroyed just by invading barbarians such as the Visigoths in 410 AD, but by the Romans themselves. Emperors would melt down bronze statues and strip buildings of adornments to pay soldiers’ wages. The later Romans built churches by quarrying the material from abandoned pagan temples, amphitheaters, and civic buildings of their fathers.

Indeed, archaeologists’ “new” discoveries are more properly re-discoveries. Lost letters were once just letters, and lost cities were once just cities. Perhaps historians of the year 3500 will excavate the sunken ruins of New Orleans, and this will surely teach them a great deal about the city—but far less than can be learned by any Mardi Gras tourist today. 

The Limits Of Scholarly Progress

It is not just the primary sources that fade over time, but also historical analyses. A glance at the bibliography of any history book will show that scholars overwhelmingly prefer to rely on sources published within their own adult lifetime—most analyses will never be read or cited after the deaths of those who read its initial publication. An especially influential work might be cited by one or two further generations of scholars before fading from the discourse. 

For example, consider Edward Gibbon’s The History of the Decline and Fall of the Roman Empire, published in 1776. It is probably the most influential work of history written in English so far, and throughout most of the nineteenth century it was part of the intellectual bedrock of historical analysis. Any educated historian was expected to be familiar with its arguments about how civilizations fall. Today, Gibbon’s masterpiece is of only niche interest. A scholar of Roman history will certainly have heard of the work, and a large minority have even read parts of it, but its arguments have mostly passed out of the historical discourse. By the year 2200, Gibbon’s footprint will be smaller still.

Works that last longer are scarce indeed. Nearly the only way for a work to gain prominence more than 30 years after its author’s death is if nearly all of their contemporaries’ work is lost, leaving them as our only window into a particular period, as with e.g. Polybius’ history of the Punic Wars, or The Secret Histories Of The Mongols—good for individual reputations, perhaps, but bad for society’s knowledge of history.

Might it be that old works going unread does not represent a loss of knowledge, because more recent works incorporate all the important information and arguments of older works, with improvements besides? This is a decent approximation of what happens in mathematics and the hard sciences, so one might expect to see the same pattern in historical analysis.

Unfortunately, comparing historical analyses of the same topics or events published in different centuries will show that this is not the case. Older works often contain important insights and key facts which are not repeated in newer works, not because they have been debunked or superseded, but because no historian has the time to read and integrate everything which has gone before. 

In our experience, there is no improvement in quality from working with newer sources, either.  If anything, older works that survive tend to be especially good, since mediocre retreads are less likely to endure for a century or three. There are also a few times and places where a field is so strong that selection effects alone are insufficient to explain the high quality of surviving materials—for example, scholarship of the Middle Ages written in the early 20th century is usually far better than anything published in the century before or after.

Historians of each generation prefer their own—but remarkably, the historians of each generation do not prefer the historians of a century prior over those of two centuries ago. We have heard people argue that Gibbon’s eighteenth-century work is inferior to contemporary historians because of the advancing march of knowledge, but never that Gibbon’s work is inferior to the historians of the nineteenth century. Odd, if there really is a discernible increase in quality from generation to generation.

Perhaps this is because history lacks the definitive logical proofs of mathematics and the experimental confirmations of physics, or perhaps this is because history is more frequently weaponized for propaganda. It is only a small exaggeration to say that the last fifty years of scholarship on the American Civil War has been entirely about fighting contemporary narrative battles, and has done nothing to advance our understanding of the period.

Whatever the reason, the “shoulders of giants” effect is mostly absent. New arguments are made, and old arguments are lost. There are local fluctuations in understanding, sometimes lasting for several generations, but no consistent upward trend is visible across timescales of, say, half a thousand years.

The exceptions, of course, are the truly significant archaeological finds. The rediscovery of the Rosetta Stone in 1799 and the excavation of Gobekli Tepe in 1995, for example, led to the recovery of crucial information and massive leaps in our understanding of ancient history. However, it is important to remember that the loss of such crucial information is invisible by its nature. The Rosetta Stone’s rediscovery led us to regain the ability to read hieroglyphics, which is still trumpeted centuries later, but the language’s loss in the early Dark Ages was likely a much quieter affair. 

The Future’s Understanding Of The Past

What is the most important piece of historical knowledge lost in the past century? It is impossible to say because if we knew the answer, the knowledge would probably not be lost.

What does all this mean for the overall trajectory of our knowledge of history? Unfortunately, on average our understanding of any given period will slowly get worse over time. Our direct evidence will gradually decay, with some temporary reversals as lost material is rediscovered. But over the long term, it is impossible for rediscoveries to outpace losses. Meanwhile, the quality of historians’ arguments and interpretations will vary somewhat, sometimes getting better and sometimes getting worse. To take a grossly oversimplified mathematical analogy, imagine a sine wave of analytical quality fluctuating around an exponential decay curve of surviving evidence.

How, then, does the work of historians fit into this? If the field is not a triumphant march towards ever more perfect understanding, is it a futile attempt to hold back an inevitable tide of ignorance? Not at all. Historians slow the decay tremendously, extending the lifespan of useful knowledge by decades at worst and millennia at best. Surely the ancient princess Ennigaldi-Nanna’ would be thrilled to hear that her lost museum was rediscovered 2400 years after her death.

More practically, while two centuries of analysis of the Industrial Revolution still leaves us with an understanding of the period’s economics far worse than that of any industrialist who actually lived through it, we would know much, much less than we do without the work of archivists who preserved the materials, as well as historians who described the important trends in ways that their contemporaries can understand. 

Regardless of how much of this knowledge might evaporate in a thousand years, nations like South Korea and China have already used it to guide their own industrialization, spreading prosperity to billions of people. Historians can carry lessons forward for much longer than this, such as when America’s founders based much of their new nation’s design on the successes and failures of Republican Rome. Historians fight a rearguard battle against the forces of entropy, buying time for their society to learn the lessons of our ancestors’ experience.