Post

Whew. I can confirm that the people who aren't ready to talk about it *really* don't wanna talk about it. Many people will only accept violent rejection of LLM tech. And if you're doing anything other than that, you must be either the enemy or an ignorant rube.

yes. this is making it *really hard* to have the tough conversations we have to be collectively having about this stuff, and it frankly worries me

Honestly, I think it's pretty much always like this. This is why tackling these ethical issues is so difficult. You can't expect most people to engage thoughtfully. And if we're blocked on that, we can't take action to protect them anyway.

there's a certain set of people here who believe that if you are part of an imperfect system, you can never be trusted.

Yeah I get it. I've definitely been the person on the outside as well.

the crypto part of your thread really resonated with me because i am 99% "yes crypto is bad." the 1% is the trans folks i know who describe it as a good way to, for example, get black market hrt. but i also didn't come to the "it's bad" stance without trying to actually learn about + engage with it

yeah and my worry is that 1% of folks don't understand exactly how hard it is to preserve opsec while using it for the purpose they think it's good for, which is going to land them in jail for fincrimes. cash is a better option a vast majority of the time for even that use case.

People who actually get value out of it are going to reach for the tech. That's just how it is. My issue with crypto was that it wasn't just bad tech. But the way it is designed was primarily to enable graft and fraud. The positive cases can be solved with other tech that isn't inherently scammy.

People don’t realize how many places it exists already that were not marketed as “AI”. Image search and general search (think “did you mean…?”) are places LLMs are used heavily.

The only generative case I’ve found useful is code completion. With local context it can save some typing. One problem with most generative uses of AI is people believe they can avoid *thinking*. They want to believe they can gain a skill with no effort, which is dangerous and wrong.

yeah, the de-skilling is pretty scary. I mean, if the abstraction is non-leaky, it's great, but I've been biased by many many years of abstractions promising to save work and then causing arcane failures after the fact.

A friend was a TA for an intro programming class that was heavy on the AI, and students didn’t understand why their code worked at the end, even though they had complete programs. Scary to think about a generation of programmers working in this way.

Okay so the real talk here is that people used to say this about assembly when it was new. “They don’t even understand the machine code they generate anymore. Look how bad and wasteful their programs are.” Yet, we continued.

I remember when C/C++ was the devil that would kill "real" programming skill.

I think it’s a mistake to compare this to C++/assembly. In those cases we got better abstractions, but people still had to understand what they were doing. Now we are saying “you can just keep typing prompts in until it does what you want”. IMO that is not a new abstraction.

i think the devil is in the "understand what they were doing" bit

something I’ve been nibbling on the edges of: there’s (at least some) work as imagined of programming that leads to a mental model of “write instructions, computer executes them” that, in modern practice, isn’t quite true. It’s already a bit like plinko as code descends through abstraction layers

not _really_ plinko, but from a code authoring standpoint, unless I am one of the most sophisticated and well versed in the field, imagining how my lines of code become processor instructions… is a lot, and not really my primary mode of thinking

yes, exactly! and even then: do you think about the way that your processor's cache coherency works, at all times? no! or how the electricity makes the hardware function to do those jobs? not at all. it's always been a fuzzy understanding

Compilers translate from one form of code into another. Saying “give me a program to count the most frequent words in a string” is doing something else.

Yeah, there're some tools in common, but parts of speech haven't proven useful for analyzing code, and PLs' twin audiences, people and engines, break mutual intelligibility as a way to chart language community boundaries.

(you nerd-sniped me here–) code is a language intended for a human to precisely communicate intent with another human, one whose utterances can (incidentally) be automatically, deterministically interpreted by a certain class of virtual machine

From that angle treating human language as the HLL to LLM-generated code's LLL rhymes a lot with C/asm or asm/machine code. It feels odd to generate human-oriented text just for machine consumption (but, as you know, it's one of CS's most cherished traditions, along with considering things harmful)

We're able to not think about cache lines and electrical engineering because each of those interfaces aims to implement a deterministic virtual machine - they hide information. For right now the lack of determinism in LLM behavior & output means the details of the output PL behavior peeks through.

Both are languages, it's just that the grammar for your average PL is probably designed to not be ambiguous. English, and probably most human languages, have constructs than can be parsed in multiple ways. For english, ambiguity can be a feature - puns/jokes/etc., for your PL it seems uniformly bad.

programs have ambiguity too! sometimes as grammar, but puns and jokes absolutely can exist in code as well

Absolutely programs have ambiguity, but do they arise from the grammar ? Aspirationally at least, your PL grammar is (almost) free of ambiguity. (Then we add \#pragma, and whatever other escape hatch or vendor specific extension)

Code is not a language in the same way English is a language. One is for making transistors flip in the right order, another is for people to express their hopes, dreams, ideas, longings. I think it’s a mistake to say an LLM is “translating” from English to python.

gotcha. i feel very differently, personally. but i also know that i tend to have a broader view of compilers than a lot of people (I think 'transpiler' is nonsense, for example)

coming from a language/psych background, something that I think underpins this is, while English and Python can both be languages, the way we use English is different. we are not just “compiling” or even decoding English, we negotiate, a lot, to develop and refine models of meaning

This is a bit of a historical artifact though. Natural languages are parsed (haha) very differently from programming languages, even though a lot of the ideas come from the same discipline. They also behave differently and are apparently even processed by different parts of the brain

whether or not the grammar is context free is important context free gets us computer languages. law might be the opposite, where judges notoriously interpret things how they want and, lately anyway, the same inouts yield different outputs

basically no real computer language is context free but yeah i've also considered the connection to law. would almost fun to be a lawyer instead of a programmer

I was going to write that the similarity is mostly superficial but I don’t think that’s really true. But I do think that they are fundamentally different in ways that I think makes the comparison break down when you try and get into the details. Programming languages are just* a vehicle for formal

semantics, whereas as natural languages are almost purely informal and their meaning shifts person to person context to context, and half the time the semantics isn’t the most important part! I’m not entirely sure that the most important part of language isn’t the squishiness and variability

i think the main difference is that human language is cooperative; you’re generally going to interpret whatever someone else is saying in a way that fits your view of reality

Assuming a fixed model, random seed, etc a prompt could be seem as deterministic, like a program. However I think there's two issues: - impossible to analyze whether a prompt is valid "code" in any sense without running it - no meaningful way to combine two working prompts into a bigger "program"