Post

p sure folk here will hate this, but i think that open models above a certain strength (Mythos level??) should indeed be restricted access below that threshold i’m all in on open models, but above leads to some serious social risks idk maybe i’ll think different after significant diffusion

I'm very much an advocate for open ways, but I don't see things going well if everyone has open access to informational superweapons before society has adapted through sufficient hardening with that being said, a shock to the system would probably force everyone to take it more seriously?

You can't control everything. I don't want to rely on something that can be revoked by some arbitrary authority.

so control nothing? idk man, that seems like an unwise decision

There are plenty of controls on the physical world and supply chains for various equipment. Like I can't easily set up a bio weapons lab without getting a visit by authorities. Regardless of whether an LLM tells me how to make it

This is like the 8th strawman I've seen used in this thread.

Ok, but… so how do you do that? It sounds like that's impossible? Regulations?

no, i’m just talking about technical controls (that regulations could reference) if you put it behind an API, you can merely block one person’s account, or by location (IP addr), or put a content classifier on it

Sorry, I'm really not in the loop, I thought open models were those you run on your own computer, not from someone else's server with API access like Claude?

Ive been thinking on this too, because there is no easy resolution and that makes it hard to resist for me even if we could do perfect work with alignment, other countries can still release models without all that and so, as usual, i think history can offer some ideas

the way I think about moments like these is based on how folks with propaganda capabilities manage true, but damaging info being made public they say the info is a needle and they build huge haystacks around the needle, so you cant find it if you wanna hide a needle, build a haystack

in practice, this would mean the US makes lots of open models with restricted behaviors baked in if cheap open models, following US standards (whatever they were) flooded the market, I would expect open source minded folks to start helping similsr to how ATproto emerged with great tools for trolls

it’s okay, i give you permission to elaborate 🤣

Agreed there are risks (which are elevated based on model ability), but are these risks qualitatively different than the same risks with computing, encryption, the internet, and mobile computing? I’m not sure they are. Those did (and do) have significant proliferation risks, but…

i guess i’d argue how little skills you need in order to leverage AI vs the others e.g. guns are controlled but bomb ingredients aren’t, nor are knives

Say more about the risks? Cybersecurity? Something else? An unknown unknown? If it's cybersecurity, is it sufficient for institutions to have a month advance warning of new capabilities? Fwiw, I strongly suspect that what's happening rn is US gov protecting its own *offensive* edge.

"Fwiw, I strongly suspect that what's happening rn is US gov protecting its own *offensive* edge." mm hm.

i’m being vague on purpose it’s the ability to pull back a model that’s important. the actual crime will probably evolve a heck of a lot over the next many months, but you can’t even begin to address any of them without basic controls

It doesn’t help that there is probably a lot of overhang in terms of harness optimization etc. Models released tomorrow will end up being more capable a year from now

Hmm. The idea of controlling information to protect against an unspecified future threat is indeed going to run against the grain of many people's instincts. The old Oliver Wendell Holmes standard for restricting speech was "clear and present danger."