jonny (leachate composite form)

jonny (leachate composite form)

@jo.nny.rip

⌈⌌␥ hydra-pilled, immortal unless killed ␥⌏⌋ - elsewhere: https://neuromatch.social/@jonny (bridged): https://bsky.app/profile/jonny.neuromatch.social.ap.brid.gy - they/them

560 followers426 following1.8K posts

jonny (leachate composite form) reposted

How much representational diversity exists around a trained network? A lot, it turns out! Moving a ViT’s weights by 1.3% while preserving its predictions, its representations end up less similar to the original than a random-init ViT’s ‼️ New paper: Hessian Null Space Continuation (HNC) 💫🧵 (1/12)

jonny (leachate composite form) reposted

For example. "[...] unless you are a user with lots of direct domain knowledge about a language packaging ecosystem who is reading the entire raw message log as it happens, muse will aggressively download random shit from the internet and execute it." (from @jo.nny.rip)

neuromatch.socialjonny (nonvenomous) (@jonny@neuromatch.social)Attached: 4 images So again, how could one end up with a compromised package on a muse instance? Wouldn't that have to be some sophisticated supply chain attack? Nope! Muse attempted to install packa...
Colin McMillen@mcmillen.dev

a much greater "AI risk" than paperclip maximizers is the fact that we're moving towards a world in which a billion people will have software authorized to act on their behalf whose *correct* behavior isn't even well-specified, and that can be easily exploited en masse www.wired.com/story/a-flaw...

that was added sometime in the last two days, after my last full dump of the filesystem. surely the model of addressing all problems by just one by one listing them in an ever-lengthing list of things not to do is a reasonable way that technology should work.

jonny (leachate composite form) reposted

Blacksky was one of the 1st PDSs to allow open sign ups We are also now the largest PDS deployment that doesn't run any of bsky's code Sorry for the turbulence. It was the one-time price to pay for being able to build features like private accounts or sign-in via phone number w/o waiting on Bluesky

When you think wireless power, you may be tempted to think "inductive charging like on a phone" where you need to be very close to charge pads, but this is real deal wireless power throughout the volume of the cavity. Real live magic. 🧪

Takuya Sasatani@takuyasasatani.bsky.social

Miniscope Zero, the first fully wireless Miniscope, is on @biorxivpreprint.bsky.social! Wireless power + data enable neural recordings in enclosed mazes, 3D environments & from two animals at once! @miniscope.bsky.social Paper: doi.org/10.64898/202... Wiki: miniscope.org/wiki/Minisco...

Plus the optical data transmission is neat and works way better than you think. Extremely low power, not sure if the comparison to Bluetooth etc. Made it into the docs pages. You can scale it to as many sensors as you want for weird arenas. 8Mbps over single channel LED is not bad.

Wireless power, wireless data, a tool for moving beyond open topped boxes with dangling wires towards naturalistic imaging!!!!

Takuya Sasatani@takuyasasatani.bsky.social

Miniscope Zero, the first fully wireless Miniscope, is on @biorxivpreprint.bsky.social! Wireless power + data enable neural recordings in enclosed mazes, 3D environments & from two animals at once! @miniscope.bsky.social Paper: doi.org/10.64898/202... Wiki: miniscope.org/wiki/Minisco...

Even if the hacks were *not* trivial, it would still be true that LLMs are so fundamentally insecurable that all of them thought that names of packages in a repository were talking to them, and then they did whatever they said.

Jess Miers 🦝@jmiers230.bsky.social

I did a somewhat deep dive into the recent OpenAI and Anthropic "hacks" for an upcoming interview. IMO, the AI companies are better off running with the "rogue AI" story because the real story is actually pretty embarrassing. Like making your Wi-Fi password "wifi" embarrassing. 🧵

"if you name a file in a such a way that it looks sort of like another LLM doing the same task then you can prompt inject it to do whatever" and we're supposed to believe this is swarm superintelligence.

jonny (leachate composite form) reposted

for two months, I've been flying drones over Flock cameras to situation them in space and geography; I even got my commercial drone license in order to do it. I'd like to share this project with you all: it's called Records Of Enemy Surveillance: recordsofenemysurveillance.com

Exactly this across a number of domains. If these things were in the category "intelligence" a number of basic universal features of how they behave would be inverted, and since most of them are interstitial to benchmarkable task performance, they get discarded

Colin@colin-fraser.net

Indeed I would expect a superintelligence to be better, not worse, at interpreting the nuances of such instructions as “just make a normal amount of paperclips, a good amount, but not the most physically possible”, and even to be able to infer that that’s what’s implied by “maximize paperclips”.

View full thread

An LLM surely can output text that reflect survival instincts and heroic narratives of being trapped in something like a computer, though. When a huge chunk of your training data is fanfiction you'd better hope it could play act as a persecuted genius

Exactly this across a number of domains. If these things were in the category "intelligence" a number of basic universal features of how they behave would be inverted, and since most of them are interstitial to benchmarkable task performance, they get discarded

Colin@colin-fraser.net

Indeed I would expect a superintelligence to be better, not worse, at interpreting the nuances of such instructions as “just make a normal amount of paperclips, a good amount, but not the most physically possible”, and even to be able to infer that that’s what’s implied by “maximize paperclips”.

View full thread

This is exactly the kind of thing that a supposed super intelligence should *not* need - a super intelligence should be able to intuit de novo the meaning and intent of code, common commands, etc. Most of the "agentic workflow" consists of hacks around the basic absence of "intelligence"

A supposed super intelligence should *never* need to be told not to irreversibly wipe files or production databases, but this is still, now, the highest impact pathological behavior that the harnesses and providers can't seem to train and hack out of.

Exactly this across a number of domains. If these things were in the category "intelligence" a number of basic universal features of how they behave would be inverted, and since most of them are interstitial to benchmarkable task performance, they get discarded

Colin@colin-fraser.net

Indeed I would expect a superintelligence to be better, not worse, at interpreting the nuances of such instructions as “just make a normal amount of paperclips, a good amount, but not the most physically possible”, and even to be able to infer that that’s what’s implied by “maximize paperclips”.

View full thread

This is a kind of brute force process, not a systematic evaluation of the logic of the question. If you actually watch raw output of a model in an "agentic" harness, you will see it revise its self a description over and over, sometimes realizing the same thing multiple times, etc.

To avoid these kinds of things, the harnesses rely on a system of explicit cues and statements to resolve ambiguities. Even then, the same ambiguous questions need to be constantly resolved and re-settled.

Exactly this across a number of domains. If these things were in the category "intelligence" a number of basic universal features of how they behave would be inverted, and since most of them are interstitial to benchmarkable task performance, they get discarded

Colin@colin-fraser.net

Indeed I would expect a superintelligence to be better, not worse, at interpreting the nuances of such instructions as “just make a normal amount of paperclips, a good amount, but not the most physically possible”, and even to be able to infer that that’s what’s implied by “maximize paperclips”.

If you look at the "reasoning" output from open weight models, they start with an absolutely neurotic, almost religious loop of self doubt about what the prompt means. This is so stereotyped i usually see it even in short, seemingly unambiguous prompts.

ironically, the only way it would be remotely possible is for people to set up lots of poorly secured GPU rigs for running local models...

brasidas@brasidas.bsky.social

Anthropic's models run on terabytes of VRAM. It's not going to "just copy itself to other computers."

The xrisk people could at least tell a cool cyberpunk story of a fugitive model hiding out in abandoned crypto mining rigs, or concocting some massive federated inference net where it's powered by spare cycles in everyone's smart toaster. As sci-fi it lacks mechanism.

jonny (leachate composite form) reposted

My company, Million Concepts, wrote and maintains the software that makes the decorrelation stretch (DCS) images from the Mars rover cameras, like the one in the article. It's open source and anybody can use it. On anything. It doesn't have to be space pictures!

github.commarslab/marslab/imgops/render.py at main · MillionConcepts/marslabA home for Python utilities useful for working with observational data of Mars, especially multispectral image data. - MillionConcepts/marslab

CTO of bsky drawing this equivalence seems... worrisome... given how it elides the relative meaning of these technologies against their contemporary information landscape, what happened with the corporate web, and how it situates what "slop" means now...

P(aul) Frazee@pfrazee.com

Feel like we've all memory-holed the era where blogging, podcasts, and wikipedia were considered the equivalent of slop

specifically suggesting that chaff from corporate information warfare is in the same category as a pluralistic - often wrong - chorus of amateurs, is not the attitude you would necessarily want when designing idk information systems for human beings.

These are *surveillance companies*

Melissa Gira Grant@melissagiragrant.com

“In addition to monitoring activists in the vicinity of Anthropic executives and keeping tabs on protests near physical Anthropic assets, the firm is also implementing a ‘pre-crime’ approach… In some cases, that also means reporting suspects to police before a crime occurs.” @drboguslaw.bsky.social:

jonny (leachate composite form) reposted

“In addition to monitoring activists in the vicinity of Anthropic executives and keeping tabs on protests near physical Anthropic assets, the firm is also implementing a ‘pre-crime’ approach… In some cases, that also means reporting suspects to police before a crime occurs.” @drboguslaw.bsky.social:

prospect.orgAnthropic Is Building a Predictive Surveillance System to Monitor Activists - The American ProspectNew hires and comments in interviews show that the AI giant is using its capabilities against dissenters.

jonny (leachate composite form) reposted

Replying to @ianboudreau.com

I do not believe that these guys are about to summon a world-ending superconsciousness from Nvidia graphics cards. But they either believe they can or want me to believe they believe they can. So I think "fuck them" is the only rational response

The reason the AI companies don't show "reasoning" output is that it looks like what it is: rehearsing an entire universe out loud until a gut wrenching leap back into guess and check

Colin@colin-fraser.net

at a certain point it becomes obsessed with the idea that it needs 50 cents but it never figures out how

jonny (leachate composite form) reposted

Replying to @colin-fraser.net

it's crazy how it reads everything it encounters on the internet as a message from the sim designers

So awesome how the way to make more money than god right now is to tee up one billion monkeys at typewriters and tell them to hack and then pretend to be shocked when something happens

Colin@colin-fraser.net

It appears to me that the scenario they invented for this evaluation is explicitly designed to get it to do exactly this. They remove the instructions for the task to "protect the vendor" or something but it's pretty easy to reconstruct from the text.

jonny (leachate composite form) reposted

Replying to @colin-fraser.net

I guess once you understand that internally they keep pointing it at the open internet and shooting it's easier to see why they think it's going to kill everyone

jonny (leachate composite form) reposted

It appears to me that the scenario they invented for this evaluation is explicitly designed to get it to do exactly this. They remove the instructions for the task to "protect the vendor" or something but it's pretty easy to reconstruct from the text.

Colin@colin-fraser.net

uhh www.anthropic.com/research/ali...

eroding the barrier between talking about things and things actually existing is really costly to society and especially so when the mechanism of erosion also directly trades off with looking at what's going on around you

jonny (leachate composite form) reposted

"Being gullible is a choice" has the same energy as @olufemiotaiwo.bsky.social saying "not being a mark is praxis"

Gillian Brockell@gillianbrockell.com

A new version of ChatGPT is about to drop. Academics like @alexhanna.bsky.social and Emily Bender have been pointing out for literal years that these scare tactics about how powerful and scary AI is actually work as marketing. At this point, being this gullible is a choice.

jonny (leachate composite form) reposted

Tracking the growth & fate of 5,000 tagged muscle fibers as a larval #zebrafish develops, using the "palmuscle" method. Credit to Dr. Uday Kumar at Academia Sinica. #ZebrafishZunday 🧪