Post

I finally read computer scientist Joseph Weizenbaum’s 1976 classic “Computer Power and Human Reason.” This book deserves a massive revival in our current age of grotesque and largely thoughtless AI creep into everything:

281 reposts1K likes

In 1980 an IBM 3033 mainframe cost $3,000,000 with 8 megabytes of RAM. Running assembly language it was the fastest machine in the Jan 1983 Byte magazine Sieve of Eratosthanese benchmark. A smartphone would blow it away.

Now that I’ve retired from the field, I’m getting around to reading books on the subject(s) I’ve collected and never read past Chap 1. A Small Matter of Programming by Bonnie Nardi (1993) And The Human Use of Human Beings: Cybernetics and Society by Norbert Weiner (1954)

I haven’t consciously thought of that book over 20 years, but much like The Mythical Man Month it underlays a lot of my thoughts. I go back and reread it at some point.

There is actually a lot of material from the dawn of the age of personal/home computing that we need to revisit A lot of SV tech bros could use a history lesson This stuff makes me feel old, but it must be shared

There’s an enormous amount of stuff in this book I’d like to highlight, but start with: “What emerges as the most elementary insight is that, since we do not now have any ways of making computers wise, we ought not now to give computers tasks that demand wisdom.”

this really drives home how the goal for these guys is NOT to create a computer that is "sentient" or "thinking" or whatever adjective one wants to use for "person", but rather a Legal Entity incapable of legal liability.

The point is we don't make every human a psychiatrist or judge. Jobs that require wisdom, in theory, are supposed to only go to people who have demonstrated wisdom; whereas an LLM can never have wisdom, no matter how many individually wise training data items you feed it.

Here is Weizenbaum in 1976 clearly expressing my own view: using AI as a psychiatric tool to treat human beings is not just wrong, it is obscene. I believe we should be using the term “obscene” far more often to describe these perverted uses of AI technology.

There is something very wrong with saying 'we have created a cheap efficient labor-saving device that can try to do some jobs' and leaping to 'we will entrust human psyches to it'. If tons of labor is being saved, freeing up lots of people, then maybe humans can be available to care for humans.

Best I can do you right now is "visceral repulsion" bsky.app/profile/pars...

Parsingphase@parsingphase.dev

At this point, our instinctive response to any mention of “AI” (LLM) chatbots should be visceral repulsion. Their users’ industrial disease is psychosis, and there are no safeguards that can be placed around these machines to make them safe.

Those are some pretty serious, profound insights. I guess I didn't really realize that people have been grappling with these issues for 50+ years. My father taught Computer Science, but "AI" was always science fiction in the 1970s when I was growing up and "computer" was a huge, loud room.

> The question then arises, and it answers itself, do we wish to encourage people to lead their lives on the basis of patent fraud, charlatanism, and unreality? I fear in many ways we have our answer to this question...

"[Man's] life is full of risks, but risks he has the courage to accept, because like the explorer, he learns to trust his own capacities to endure, to overcome. What could it mean to speak of risk, courage, trust, endurance, and overcoming when one speaks of machines?"

What's funny is that LLMs are also no good at answering question of fact or logic. I've been trying to teach ChatGPT the asbestos regulations, and it reaches to the most absurdly wrong missinterpretations. The only thing they are good at is mimicking human language.

From the Wikipedia entry: “The book caused disagreement with, and separation from other members of the artificial intelligence research community, a status the author later said he'd come to take pride in.”

Thanks, I've read some of his essays before, time for the whole book. It seems he was way ahead of the curve, prescient in that down to earth way of the truly wise.

OK, my tsundoku reading list is *already* causing a wobble in the earth's spin. Can we either just drop the AI thing already or just give everyone a month off to read?

I ask Joe to autograph my copy using my own Parker pen. Another time, I sat next to him during dinner at a conference in Portland. We both expressed dismay when Seymour Papert stated that teachers should STFU and let students explore to learn. He came to the dedication of Sci bldg at L&C college.

Not that they hadn’t tried, though. When the book came out, many of the bigshot AI researchers accused Weizenbaum of sour grapes on account of no longer being able to contribute anything original as a scientist.

It’s also on Anna’s Archive: bsky.app/profile/fain...

Faine Greenwood@faineg.com

Joseph Weizenbaum’s “Computer Power and Human Reason” is available on Anna’s Archive…though I think we desperately need a reprint run. annas-archive.org/md5/532a20e6...

The irony of a prescient critique of computational thinking being preserved through digital archiving is not lost. Weizenbaum would likely appreciate the meta-commentary.