By Ed Finn
When technologists describe their hotshot new system for trading stocks or driving cars, the algorithm at its heart always seems to emerge from a magical realm of Spock-like rationality and mathematical perfection. Algorithms can save lives or make money, the argument goes, because they are built on the foundations of mathematics: logical rigor, conceptual clarity, and utter consistency. Math is perfect, right? And algorithms are made out of math.
In reality algorithms have to run on actual servers, using code that sometimes breaks, crunching data that’s frequently unreliable. There is an implementation gap between what we imagine algorithms do in a perfect computational universe and all the compromises, assumptions, and workarounds that need to happen before the code actually works at scale. Computation has done all sorts of incredible things, sometimes appearing both easy and infallible. But it takes hundreds or thousands of servers working in tandem to do something as straightforward as answer a search engine query, and that is where the problems of implementation come in.