When technologists describe their hotshot new system for trading stocks or driving cars, the algorithm at its heart always seems to emerge from a magical realm of Spock-like rationality and mathematical perfection. Algorithms can save lives or make money, the argument goes, because they are built on the foundations of mathematics: logical rigor, conceptual clarity, and utter consistency. Math is perfect, right? And algorithms are made out of math.
We spend an awful lot of time now thinking about what algorithms know about us: the ads we see online, the deep archive of our search history, the automated photo-tagging of our families. We don’t spend as much time asking what algorithms want.