Post

cpu is dogshit slow for inference, and self-replicating worms already exist (you may know them as rVPNs or botnets) alas they're just repackaging an existing problem as "you should be afraid of everyone's ai except ours"

151 reposts610 likes

yeah--i'm not all sanguine, but they sure do pick stories that happen to align with eliminating competition. 90s MS vibes. they're willing to sell to this admin's defense and spy agencies and don't have much to say about risks of oligopoly!

It's becoming quite clear the only way companies like Anthropic survive or remain at their current market share is by creating some kind of regulatory moat and capture.

Most of us also just shouldn't have smart home devices. My coffee and washing machines don't need to be online. My light switches are dumb and I like it that way. I don't install spy cameras so creepers at Ring can watch my home and I don't like visiting friends who have those.

considering the gaps in their monitoring, he could mean “worms will install new harnesses on compromised devices and we will happily perform interference for them until your api key runs out of budget”

Yeah, I can't see a rogue model (and it's weights) being a likely concern. But I could imagine a rogue prompt that could propagate itself, finding compute to run on. An LLM based worm.

It would be nice though if they started to take security on these devices a bit more seriously. There will be a real future problem of peoples local networks getting hacked through smart TVs and other devices with known exploits and no updates.

I haven't used public wifi in years. But I have no doubt on my local LAN at home there are devices, especially embedded, that could be exploited with a foot inside the network. These are the types of devices I would especially love to see improvements on.

I will eat my hat if a smart tv can host enough inference to devise *new* attacks rather than just exploit the same existing attacks and just happen to bring along an additional giant multi-gigabyte payload for the hell of it, which definitely lowers reproduction fitness.

I do expect (what I'm calling) "dark inference" to become a thing, especially as devices attempt to incorporate hardware capable of running small, simple local models (which can then be hijacked). maybe not this year. possibly next. certainly before the end of the decade.

We can distribute inference. This is all stuff that r/localllama has been doing for the last year LocalLlama: "I have a esp32 and an ipad. I'm going to run GLM. Hold my beer."

my macbook air barely manages to run a 27b parameter model, and that shit ain't capable of launching new attacks that an enterprising North Korean isn't already capable of. let along a dumb router or IoT shit device

The only smart thing I own is my Samsung Smart TV, and it's connected directly to cable. My router that is associated with the cable account has wifi, is that going to be an issue? Are my computer and iPhone going to turn into little Anthropic bots? I do not understand this stuff...

Question: Why do we need or want all this shit? It's a bad sci fi movie waiting to happen. "And Skynet because self aware..." Not that Terminator is a bad sci fi movie, it's a great sci fi movie but I don't want to move in.

i'm sure splitting over 15 devices and passing the vectors between them for every single token will speed it up dramatically once the devices aren't all on the same network in a lab :P

I do like the idea, as a sci fi concept, of a machine intelligence slicing its thought processes up over a very large and redundant network of machines whose users don't know it's there; by having a small foot print on any one machine, it can hide and survive, at the cost of thinking very slowly

(stop me before I contemplate homomorphic cryptosystems and whether a neural network over a galois field would work - 10^-9 tokens/sec here we come, baby!)

that would be like me saying I solved distributed systems by having a shared file on Novell NetWare without thinking about sharding and consistency and quorums and fragility, though

Exactly! “Oh no, we’re being P(doom)’ed at a rate of several tokens per day!” Doesn’t seem partially worrisome compared to, well, literally anything else bad that might happen.