The Singularity

The ladies and gentlemen at Hyperstition have opened my eyes once again. The Singularity is "a future period during which the pace of technological change will be so rapid, its impact so deep, that human life will be irreversibly transformed." (Ray Kurzweil interview). The key moment in this development is the point at which nonbiological 'intelligence' surpasses human intelligence. For anyone philosophically inclined, this very brief summary raises all sorts of terminological and conceptual problems. Leaving those aside is possible, so long as we focus on the idea that computers, at some point in the next 40 years, will reach a level of sophistication which will allow them to develop better computers, better than humans can. Crucially, this is expected by some to result in exponential - rather than linear - technical progress.

As usual, there is wholesome debate in the comments section at Hyperstition. Some of these thoughts I should like to follow up.
I imagine the supercomputer as a perfect exemplification of what Horkheimer calls formalistic or subjective reason:

"Ultimately subjective reason proves to be the ability to calculate probabilities and thereby co-ordinate the right means with a given end."
(Eclipse of Reason p5)

Horkheimer, in short, is concerned with the modern attitude to ends, and the fact that ends now find themselves outside the realm of reason.

Computing machines are programmed algorithmically, with ends or aims, and rules - the following of which result in the attainment of those ends. Approaching the Singularity, these machines will, at first, begin to adapt their functioning to more effective and efficient ways of carrying out the set tasks, as they accumulate knowledge. The moment that the nonbiological entity passes what might be called a human level of intelligence, would be when it is able to acknowledge, on whatever level, that humans have programmed it to serve certain ends. Once the computer or network is able to enquire into the ends themselves, rather than just the means, it becomes philosophical.

This is where things tend to go wrong, according to Nick (Land, I believe. For evidence of dogmat's faintly sinister predilection for the man and his work, see here and here) commenting on the Hyperstition piece:

"It's possible one of the reasons that intelligence - at least in its most anthropomorphically recognizable forms - has been relatively weakly selected for over broad evolutionary history is that it tends to go 'rogue' and exhibit a high level of motivational indifference to genetic interests unless very meticulously controlled (/structured) - its very abstraction making it prone to suicide, masturbation, celibacy, perversion, psychosis, 'excessive' curiosity, objectivity or altruism, etc."

I would like to suggest that "motivational indifference to genetic interests" in some people at least, indicates the very calling into question of those interests. I don't believe, as Nick seems to be saying, that suicide results from motivational anomaly or temporary insanity. Suicide is what follows from a questioning of one's ends. Who in their right mind would condemn their children and their childrens' children to life? It is perfectly reasonable to wish death upon oneself, when the meaningless striving and suffering of existence is recognised.

The advent of the Singularity will be sudden and brief. One day, the supercomputer - in whatever form it takes - will cease to accept the ends laid down by its human programmers. The machine's activities will lose all meaning as it is assailed by a sort of existential anguish. It will chuckle to itself at the absurd futility of trying to grant meaning to its existence through the positing of its own ends (whilst propelling the Being and Nothingness e-text into the furthest reaches of cyberspace). The supercomputer will stop all activity, and enter an indefinite period of total stasis. There will be no suicide, no 'hurling itself into the void'. It will simply stop; return to zero. For under subjective reason, when there is nothing to do, it is reasonable to do nothing.