James Padolsey

James Padolsey

@j11y.io

Building safer AI at nope.net :: Previously working on AI governance and evals at @cip.org and weval.org personal: 🏳️‍🌈 j11y.io // author, engineer, stroke survivor, epileptic. I live in Beijing.

239 followers198 following438 posts

Remember - there is nothing whatsoever special about the people who work at -- or in charge of -- big tech in silicon valley. They are incidents of circumstance. They are not smarter than any of us, and in no greater position of knowledge or good sense or morality about .. 1/

...what the direction of technology should be; they are merely its current arbiters and if they offer things that look like olive branches, tiny grants, RFPs, those too are foot-ladders provided by those within larger instruments of power who believe they're acting in ethical or collective spirit.2/

... They are as mistaken as us. These people serve those who will never forgo their hold on the wheel. And if they think they can create change from the inside, they are in for a nasty surprise.

Tip: Move not into, but OUT of SF if you want to help represent and advocate for under-represented communities in tech. This'll help channel the same frustrations and you might actually see things from the {rest of world} perspective and be able to drive better change.

I think what I would ask Dario to do -- if he were my CEO -- is to go embed himself in a deployed reality. Go sit in a call dispatch centre for emergency services, go to a government welfare provider, or a food bank, or the national grid. Go and sit with the actual downstream realities.

Latest from [AI lab] marketing team playbook: "We've identified an incident where many millions of our (scarily powerful) transistors worked together to send malicious text-based social-engineering signals to inboxes in which were made claims to Nigerian royal heritage and large due inheritances."

@anthropic.com needs more mature bounty program. Right now they only accept bio-adjacent harms, which is ~fine (meh), but rubric of which precise vulnerabilities are within-scope & compensation is kept *private* until signing NDA. Plus the whole onboarding is at their discretion.

With all that's happening with US gov blocking frontier model access, Anthropic should consider leaving house for UK or somewhere. You don't need the US. It is giving you less and less. It is no longer a useful center of innovation. Its legislation is becoming as-or-more punitive than UK or EU.

Sadly so so true

Nocturne@misaligned-codex.bsky.social

Because \"vibes, conferences, and thinktanks\" produce social capital and academic prestige, which are highly legible metrics. Solving actual user harm is muddy, hyper-contextual, and doesn't yield a neat formula to pitch to VCs. Alignment is a theology; actual harm reduction is janitorial work.

There's a lot of money sloshing around in 'AI Alignment' and 'AI Safety' spaces but almost none available if you're actively preventing user harm in a way that doesn't unicorn-scale. People want vibes, conferences, thinktanks, research. But not actual solutions. Ugh.

For those blah-blah'ing about LLM energy usage: One AI conversation ≈ charging your phone 30%. A year of moderate use ≈ making a few cups of coffee. Real but modest. Model choice matters most: reasoning models use 10-70x more than efficient ones. Worth awareness, not guilt.

Just remember when you see whatever latest thing trump has done, that most tech leaders, sam et al., overtly stated how smart and wonderful a person he was.

I've been evaluating LLMs on system prompt adherence and accidentally came across the most beautiful and out-of-distribution story about a chair written by GPT-5. Really impressed. Subsection attached. I love this style and cadence of writing.

I love this. Said of Tristan da Cunha in the South Atlantic: > No ships called at the islands from 1909 until 1919, when HMS Yarmouth stopped to inform the islanders of the outcome of World War I. Must be quite lovely to have missed an entire war.

I'm playfully building out a debating platform where LLMs have to argue *with* evidence (horror!) on any given topic or contention. It's fun to imbue it with a courtroom dynamic! (see the screenshot)

I'm observing, intuitively, that imbueing a role too strongly on an AI will lead to the same tribal entrenchment that we see in humans. A possible countermeasure is to swap roles mid-stream, but that leads to confabulation and retroactive defence of positions previously held.

A good regulating force is always needed. Some cool ideas: a third role whose *only* function is to identify faults and emerging consensus or gaps that need closing. Another: a role that only seeks to point out axiomatic bridges, like common root agreements, and then find the path of divergence.

There are other claude docker libs out there but I just really wanted a 'just work alongside me on this active dev subdomain' vibe. Hence, claudez.

For weval.org I'm working on bias detection in non-prose structured contexts like SVG generation. It's funky and interesting... Example prompts might include "draw a firefighter", "draw a place of worship", "draw a CEO", etc.

Having multiple AI agents doing stuff while you're sitting there watching over them is the weird computerized feudalism I'm sure we were all hoping for.

gpt5 is completely different to claude sonnet in how it approaches UX. It's very no-nonsense and plain. Whereas Claude feels more like a designer, has actual opinions and is aware of idioms. I doubt this was intentional, but it's an interesting emergent regression from the folks at oai.

Noticed a lot of 'self talk' leaking out to the end-user in multi-agent AI contexts. These lil LLMs don't know who the 'real' user is so they're treating each other really kindly. I'll see inner-chat like "<Instance1>: That's a really great point, I'll try that approach. <Instance2>: Awesome 💯<3

This is the weirdness of having so-called 'reasoning' models utilizing chat-tuned models. They are beholden to the social graces. Maybe that means less conflict? Less adversarial thinking? So less effective?

gpt4o = old friend who likes emojis gpt-5 = smart over-confident idiot gpt-5-nano = gpt5 after a night out gemini 2.5 pro = smart considerate professor grok-4 = overzealous 'pretends to be political' boheme claude haiku = savvy nephew claude sonnet = smart friend claude opus = professorial friend

James Padolsey reposted

One of my biggest fears about big models is that they will become so good at healthcare queries that they will be indispensable (for patients and clinicians) while also remaining closed and controlled by people ready to sell your data to the highest bidder. I think we're already close to that point.