Post

You're right, I hate this

Restrictions on open source are a restriction of speech and a government mandated monopoly (or duopoly), and vulnerable to disruption by those who aren't bound by the same regulations globally. It should be illegal for Mythos to be closed, the weights should be guaranteed and required to be open

i think you have to demonstrate harm first (and also that makes it more salient), and then i think you have to consider who is doing the restricting, and how that encourages competition and conflict

I at least share your concern that powerful open models could be dangerous. I am generally skeptical that such restrictions could be effective. If we place open model restrictions but Chinese firms continue releasing open models, bad actors will have no difficulty using those instead. This would...

be different if China also felt there was a need to restrict access to open models. I think that shared international will would only happen if AI had already lead to a catastrophic event, which could plausibly happen in the next decade. If something like that occurred or I was convinced for some...

other reason that near-term risk was high, I might support open model release delay/pause, but it would have to be serious risk and I think it would be well past Mythos level capability. For now I think the best way to deal with open model risk is @hlntnr.bsky.social's adaptation buffer strategy.

I have no problem in principle with intelligence above a certain level being controlled-access. We have plenty of cautionary tales of genies twisting wishes. My question is, how do we police it? William Gibson's Turing cops, roaming the inner solar system in pairs to take down rogue AIs?

the thing i want is controls — the ability for access to be revoked, content guards, etc. open models inherently don’t have these things. like if they had them they wouldn’t be open

I'm very much an advocate for open ways, but I don't see things going well if everyone has open access to informational superweapons before society has adapted through sufficient hardening with that being said, a shock to the system would probably force everyone to take it more seriously?

You can't control everything. I don't want to rely on something that can be revoked by some arbitrary authority.

so control nothing? idk man, that seems like an unwise decision

Ok, but… so how do you do that? It sounds like that's impossible? Regulations?

no, i’m just talking about technical controls (that regulations could reference) if you put it behind an API, you can merely block one person’s account, or by location (IP addr), or put a content classifier on it

Ive been thinking on this too, because there is no easy resolution and that makes it hard to resist for me even if we could do perfect work with alignment, other countries can still release models without all that and so, as usual, i think history can offer some ideas

the way I think about moments like these is based on how folks with propaganda capabilities manage true, but damaging info being made public they say the info is a needle and they build huge haystacks around the needle, so you cant find it if you wanna hide a needle, build a haystack

it’s okay, i give you permission to elaborate 🤣

Agreed there are risks (which are elevated based on model ability), but are these risks qualitatively different than the same risks with computing, encryption, the internet, and mobile computing? I’m not sure they are. Those did (and do) have significant proliferation risks, but…

i guess i’d argue how little skills you need in order to leverage AI vs the others e.g. guns are controlled but bomb ingredients aren’t, nor are knives

Say more about the risks? Cybersecurity? Something else? An unknown unknown? If it's cybersecurity, is it sufficient for institutions to have a month advance warning of new capabilities? Fwiw, I strongly suspect that what's happening rn is US gov protecting its own *offensive* edge.

"Fwiw, I strongly suspect that what's happening rn is US gov protecting its own *offensive* edge." mm hm.

i’m being vague on purpose it’s the ability to pull back a model that’s important. the actual crime will probably evolve a heck of a lot over the next many months, but you can’t even begin to address any of them without basic controls

and how will that work, exactly?

like, how would enforcement work? make open weight models illegal? because otherwise people just download the weights, and then run them on their own hardware, and who is going to know?

You won't be able to restrict open models though. Because a company that plays ball (say zai) won't mean all companies do. The big ones are too powerful for local consumers, but not for state actors. The only way to restrict open models is to ban them. Which means no open models

Ensuring dangerous capabilities are restricted (eg how to make a nuke) is a good thing, but in regard to LLMs it is a slippery slope. LLM capabilities are jagged. Meaning that Mythos might be paramount at cybersecurity, but it not be dangerously good at something else. Where do we draw the line?

i agree that it’s a slope, and i also think you can go partway down a slope without sliding to the bottom it feels strange to argue that we shouldn’t be able to stop an in-progress crime. That feels like a pretty clear trade-off

If highly capable open models are accessible only to gov, approved companies, or institutions, while being restricted for most individuals, this results in a significant imbalance in access to knowledge and capability. Not a good outcome.

this would guarantee the bad timeline bsky.app/profile/snep...

i’m just advocating for requiring that it’s possible to cut off access, and only if it’s above a certain strength by the way, if you’re using non-local models you’re probably already subject to this

"it's possible to cut off access" then requires answering "who gets to decide whether access is cut off or not, and to whom"