Post

SE Gyges
@segyges.bsky.social

being anti AI is a luxury belief only available to first world people, especially Americans imagine how important machine translation is for people whose first languages aren't that common

aly
@aly.codes

all contemporary computer translation services use language models more or less identical to the usual suspects, they've used machine learning algorithms for something like 15 years and transformer architectures specifically for over half a decade.

0 reposts85 likes

graham-o.bsky.social
@graham-o.bsky.social

So they are just lookup functions with a bloated name. Intelligence would allow a word derived from two root meanings to be worked out. AI only does that if it’s told what the answer is from its data. It can’t ’work out’ anything.

aly
@aly.codes

this is not actually true, linguists have been using machine learning for comparative linguistics and reconstruction for many years. it's actually quite trivial and you can, in fact, produce novel results.

oliver
@eikopf.com

i regularly portmanteau words together to express some composite meaning when i’m working because i trust the robots to figure it out more than i do actual people

oliver
@eikopf.com

go ask an llm what an invented term like “SEOslop” should mean, i guarantee they’ll get it

Ori Vandewalle
@orivandewalle.bsky.social

ChatGPT created a portmanteau unasked in an interaction I had with it last year.

Ori Vandewalle@orivandewalle.bsky.social

LLMs are just fancy autocomplete on stolen data. They could never come up with anything new... like a neologism for a kind of anagram.

oliver
@eikopf.com

it’s all reinforcement learning no one ever told them what words mean. the whole point of llms is that language is too complicated to define formally and you have to rl your way to understanding

jae (hallowed)
@fubarchitect.com

from gemini: can you show an example portmanteau that def isn't in your training data, and define it, in under 200char? > Keycrastination (Keyboard + Procrastination): Typing random text on a blank screen just to look productive while completely avoiding actual work.

Peter
@petmouse.bluesky.mousses.xyz

that's actually not how translations have worked for quite a while now. transformers use a tokenizer to break down words in any language, so yeah by your definition they're intelligent can you work out root meanings of words only when you've already heard of it? that's a ridiculous bar to set

graham-o.bsky.social
@graham-o.bsky.social

That’s what intelligent humans do. They do research and work it out. AI doesnt do anything like that. It looks up what others have decided is true and quotes that. If a word combines two different languages together, a human would recognise the two languages, and work it out. AI would spin its wheel

Peter
@petmouse.bluesky.mousses.xyz

I think you're misunderstanding what LLMs(aka AI) actually do. the model weights bias the probability of the next token. for example, if you ask "translate to Spanish: what", the model predicts que from the algorithm in its weights also Large *Language* Models excel at translating between languages

Cody
@cody.chaos.net.nz

This is simply untrue. You can use novel portmanteaus, you can misspell words to a hilarious degree, you can make up slang, and the AI will figure it out. Better in many cases than a human would. Whatever else is going on, it's not a simple lookup.

Cody
@cody.chaos.net.nz

For another example, you can ask a model to add two 12-digit numbers and force it to answer "internally" (not using a tool like a calculator). Current models can do this correctly, but a lookup table is too large to be contained in the model (or even the training data).