by Thomas Fernandes
One of my favorite thought experiments is the Library of Babel. Imagine an infinite library where the first book would contain only the letter A, the following book the letter B, and so on, with every character eventually moving to two-letter books, then three letters, on and on until all possible word combinations are represented in a book somewhere in the library. This library now contains all the available knowledge. All unavailable knowledge as well. Everything ever written is in there. Every paper that will ever be published, every conversation you have had, every YouTube comment under a given video, and your life story down to the actions you have not taken yet. In the thought experiment, all of it truly exists.
We already have trouble appreciating large numbers, infinity is so absurd that even some mathematicians would rather throw the concept out altogether. So a thought experiment helps us realize what infinity entails. It is also very conceptual. Revisiting the same library with a practical mind will point at a more practical truth. All knowledge is here, yes, but every meaningless combination of letters is there too, alongside multiple copies of every book differing by one typo. If you were to try to get something out of this library, your life’s achievement might be to decipher a “Hello World”, somewhere in the middle of a book of 873,618 characters.
This turns the thought experiment toward the power of curation. If generation of content is very large, you need to select at the same scale you generate, very relatable considering today’s entertainment landscape and algorithmic recommendation.
The only reliable way to ensure a similar rate of selection and production is to have them happen at the same time. If every book was filtered at production, the library would make sense and, to an outside visitor, the library would seem to be filled with human-made books (or at least purposefully written ones). Because, as mentioned above, if it were really random it would be filled with random, garbage books and the coherent ones would be the exception. The dual engines of generation and selection create the appearance of intelligent design. In the process, like in a social media algorithm, the algorithm is the content. Select for English books and your library will appear as if written by English people only. Select for success in spreading genes and you get natural selection.
Notice that in the Library of Babel the books are not randomly generated but are copies of the previous book with one modification. Inheritance is important as a log of explored space that you can deviate from, and it lets you avoid running in circles, printing the same book again and again. In natural selection, genes are selected at reproduction, having surviving children, which happens at the same time that we create small variations in the inherited genes.
Like the library, it historically left people convinced that there must be an intelligent design behind such non-random creation. In fact we now understand that the design is simply a three-part process of inheritance, variation and selection. Read more »