Menu

Post image 1
Post image 2
Post image 3
Post image 4
Post image 5
Post image 6
1 / 6
0

Internet archival sites struggling to preserve the internet because of skyrocketing hard drive prices due to the AI boom — Wayback Machine and Wikimedia punished by stratospheric storage pricing and stricter anti-scraping measures blocking the wrong bots

Reading 0:00
15s threshold

The internet is getting harder to archive because the AI boom has caused a storage crisis, with both NAND and mechanical drives facing shortages. The same large-capacity HDDs now cost up to 3x more due to shriveled production capacities that have otherwise been entirely booked out by hyperscalers . These rising prices have made it difficult to preserve data at the usual rate across the industry, as reported by 404 Media . The Internet Archive, whose mission is to provide "universal access to all knowledge," is one of the organizations affected by this crisis. It holds around 210 petabytes of archives, with another 100 terabytes added every day to collections like the Wayback Machine. Amidst the AI boom, maintaining it has become "a very real issue costing us time and money," founder Brewster Kahle told 404 Media . Archiving the internet shares the same first step — it needs to extract information in order to preserve it, but website operators have been increasingly blocking such efforts .…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More