Annotation
Thinking it through
It is perhaps paradoxical that just, when in the deepest sense man has ceased to believe in - let alone to trust - his own autonomy, he has begun to rely on autonomous machines, that is, on machines that operate for long periods of time entirely on the basis of their own internal realities. If his reliance on such machines is to be based on something other than unmitigated despair or blind faith, he must explain to himself what these machines do and even how they do what they do. This requires him to build some conception of their internal “realities.” Yet most men don’t understand computers to even the slightest degree. So, unless they are capable of very great scepticism (the kind we bring to bear while watching a stage magician), they can explain the computer’s intellectual feats only by bringing to bear the single analogy available to them, that is, their model of their own capacity to think. No wonder, then, that they overshoot the mark; it is truly impossible to imagine a human who could imitate ELIZA, for example, but for whom ELIZA’s language abilities were his limit. Again, the computing machine is merely an extreme example of a much more general phenomenon. Even the breadth of connotation intended in the ordinary usage of the word “machine,” large as it is, is insufficient to suggest its true generality. For today when we speak of for example, bureaucracy, or the university, or almost any social or political construct, the image we generate is all too often that of an autonomous machine-like process.
- Joseph Weizenbaum, "Computer Power and Human Reason" (1976)
In 1976 Joseph Weizenbaum offered the proposition that in order to be able imagine how a computer might be of use a human being would first be required to design an internal model or map
that would offer a way for that person to understand to his or her satisfaction how the computer does what it does. Weizenbaum’s submission was that such a human through simple ignorance of what a computer actually does and therefore an inability to divine its process would imagine that an appropriate analogue for the computer’s operation might be the human’s own thought process. It might well be argued in 2006 that little has changed and that the majority of computer users remain unaware that the computer they imbue with characteristics such as ambivalence and non-cooperation does not have any capacity to respond autonomously but that it invariably responds only within the limits of its internal architecture and the programming that constrains it; that the computer has no ability to ‘think’ at all and decidedly not to do so independently. Weizenbaum also suggests that the anthropomorphising of computers in this way creates a degree of equalisation between the way that computers are seen to operate and the way in which humans perform and that this comparison operates in reverse; that is humans increasingly come to see themselves as operating in ways similar to their computers. The result of this might be therefore that humans accept more easily their being replaced by computers better able to ‘think’ in the way they have imagined that a computer might. Another consequence could be a more complacent attitude towards accepting an argument or position because it was presented by a computer where the same argument might well be questioned if it was presented by another human. As more people turn to the web for information such a predisposition towards acceptance would have serious ramifications.