The internet is often thought of as a permanent record of information, but a new study reveals that a substantial portion of web pages from the past decade has vanished. Researchers at Pew Research Center found that about a quarter of web pages that existed between 2013 and 2023 are no longer accessible. This phenomenon, known as “digital decay,” results in broken links across various websites, including news media, Wikipedia, and government pages. The long-term cost of losing such a vast amount of web content raises concerns about the information we may be missing. To determine the extent of the problem, researchers analyzed a random sample of nearly one million web pages from Common Crawl, an internet archive service. They discovered that 38% of pages from 2013 were inaccessible, while 15% from 2022 and 8% from 2023 were no longer reachable. Pew defined these inaccessible sites as those that no longer exist on their host servers, often resulting in “404 not found” errors. The research also highlighted the presence of broken links in various types of web content. It found that 54% of Wikipedia’s reference sections had at least one broken link, and 23% of news webpages contained dead links.
Digital decay and missing information
Local government websites were particularly affected, with many city government pages exhibiting a high number of broken links. Social media platforms are not immune to digital decay either. Nearly 20% of posts on X, formerly known as Twitter, are no longer publicly visible. Of these, 60% were from accounts that have been made private, suspended, or deleted, while 40% were posts deleted from still-active accounts. “The internet is an unimaginably vast repository of modern life, with hundreds of billions of indexed webpages,” the study authors wrote. “But even as users across the world rely on the web to access books, images, news articles, and other resources, this content sometimes disappears from view.”
While some digital losses, like embarrassing photos from MySpace, might not be missed, the overall trend of digital decay threatens to give us an incomplete picture of the web’s evolution. In the past, research involved searching physical libraries or microfilm archives. Today, people assume information will always be available online. However, this assumption is becoming increasingly inaccurate, suggesting that we may have much less information at our fingertips than we realize.
