Max Tegmark in KurzweilAI:
Exactly three years ago, on January 13, 2011, humans were dethroned by a computer on the quiz show Jeopardy! A year later, a computer was licensed to drive cars in Nevada, after being judged safer than a human. (link to article) What’s next? Will computers eventually beat us at all tasks, developing superhuman intelligence? I have little doubt that this can happen: our brains are a bunch of particles obeying the laws of physics, and there’s no physical law precluding particles from being arranged in ways that can perform even more advanced computations. But will it happen anytime soon? Many experts are skeptical, while others such as Ray Kurzweil predict it will happen by 2045. What I think is quite clear is that if it happens, the effects will be explosive: as Irving Good realized in 1965, machines with superhuman intelligence could rapidly design even better machines. Vernor Vinge called the resulting intelligence explosion ”the singularity,” arguing that it was a point beyond which it was impossible for us to make reliable predictions. After this, life on Earth would never be the same. Whoever or whatever controls this technology would rapidly become the world’s wealthiest and most powerful, outsmarting all financial markets, out-inventing and out-patenting all human researchers, and out-manipulating all human leaders. Even if we humans nominally merge with such machines, we might have no guarantees whatsoever about the ultimate outcome, making it feel less like a merger and more like a hostile corporate takeover.
In summary, will there be a Singularity within our lifetime? And is this something we should work for or against? On one hand, it could potentially solve most of our problems, even mortality. It could also open up space, the final frontier: unshackled by the limitations of our human bodies, such advanced life could rise up and eventually make much of our observable universe come alive. On the other hand, it could destroy life as we know it and everything we care about — there are ample doomsday scenarios that look nothing like the Terminator movies, but are far more terrifying.
More here.
