Post

Jonathan Bellack
@bellack.com

We have tried to govern software safety with market forces and liability. Construction used to be like this. Cities burned down and bridges collapsed. So now we require licensed architects, stamped plans, and expert inspections before occupancy. "Building codes are written in blood." 2/6

finehomebuilding.comA History of U.S. Building Codes - Fine HomebuildingLearn about the history of building codes in the U.S., from the formation of organizations to the establishment of the ICC and the I-Codes.

0 reposts0 likes

Jonathan Bellack
@bellack.com

If you think talk of "blood" is overblown, read this horror story in @motherjones.com about the Tumbler Ridge shooter's use of ChatGPT to fantasize and plan, and how the company “had decided not to alert authorities, despite the urging of some employees on OpenAI’s safety team.” 3/6

motherjones.comChatGPT helped the Tumbler Ridge school shooter focus on guns, tactics, and terror, our investigation revealsOpenAI’s chatbot fed the shooter’s violent fantasies and planning up to the massacre.
Jonathan Bellack
@bellack.com

The industry's proposals for self-regulation leave me cold because the economic relationship between AI companies and the standards / evaluator body reminds me of the Enron / Arthur Andersen scandal, where the accounting firm’s financial incentives led it to turn a blind eye to rampant fraud. 4/6

en.wikipedia.orgEnron scandal - Wikipedia
Jonathan Bellack
@bellack.com

Demis Hassabis' FINRA proposal is also a lousy model. Broker-dealer codes of conduct are nothing like evaluating the safety of pervasive, powerful software products like frontier models. They should be overseen and inspected by a funded government agency, more like the SEC or the FDA. 5/6

demishassabis.substack.comA Framework for Frontier AI and the Dawning of a New AgeThis is a pivotal moment in human history. Artificial General Intelligence (AGI), a system that exhibits all the cognitive capabilities the brain has, is probably only a few short years away.
Jonathan Bellack
@bellack.com

tl;dr, again: Effective software safety and oversight needs to be independent from industry, not on their payroll. Read my email to my dad and more in this week's Platformocracy. Free, weekly, with mildly amusing pop culture easter eggs buried in the links. 6/6

platformocracy.comExplaining AI doomerism to my dadI don’t talk about AI much in Platformocracy. A recent email to my father does a good job of laying out my priors and biases. [tl;dr: self-regulation is weak sauce]