41 The end is nigh
How Excel almost sabotaged Wiki-Leaks once
When Wiki-Leaks founder Julian Assange handed a file with over 92.000 field reports from the Afghanistan war over to journalists from The New York Times and The Guardian in 2010, the data abruptly ended in April 2009 even though there should have been data from the entire rest of the year. What had happened?
The journalists had opened the data in Excel which, at that point, had a size limitation of 65.536 lines so that the file exceeded the spreadsheet's maximum capacity. All data after line 65.536 had simply been cut off.
Even though the maximum size has been increased to 1.048.576 lines since then, this story still neatly illustrates that Excel is not an adequate substitute for a professional database. This is especially true for research projects that expect to generate a large amount of data. In such cases, it is very important to consider suitable alternatives and associated costs early on.
- “When Good Data Turns Bad” from “Humble Pi: A Comedy of Maths Errors”, pages 244-245