Philip Nuzhnyi

Philip Nuzhnyi

@callmephilip.com

🧜‍♂️ Surftware engineer. Stochastic parrot CODE: https://github.com/callmephilip MD: https://callmephilip.com/ 🫛(doom) = 5% 🥕(boon) = 56%

368 followers350 following1.3K posts

"Local compute that keeps your data secure on device is the best case for these AI models. NVIDIA, ironically, the company that's chiefly pushing the AI data center build-out worldwide, may end up showing us a better way forward"

Failing forward. Lesssgo

Gordon@gordon.bsky.social

"Error is, in itself, neither bad nor good, but endemic - it cannot be eliminated. In fact careful consideration shows it is error that drives the system!" - Glanville, 2019, "Try Again. Fail Again. Fail Better.”

Philip Nuzhnyi reposted

Constantine could use some help in getting non-lethal aid to Ukrainian defenders on the front. Would mean a lot if you could support this cause with a repost, like and/or donation. Good luck @teoyaomiquu.bsky.social and thank you for what you do. 💙💛

Constantine@teoyaomiquu.bsky.social

23,000$ fundraised over the the past 14 hours 50000$ left to go. Your donations, likes and reposts are extremely helpful.

UX design and software architecture design. Designing and maintaining software architecture both humans and machines can easily reason about is hard

entropy@entropy.obscurityskies.xyz

UX design might be the most important skill in software right now, and for the foreseeable futurable. Any SWE can spin up the most convoluted system design and have it realized for nothing. Most have no clue how to make the ergonomics of using the thing pleasant for humans (and neither does Claude)

View full thread

! Do 1 round through the model (as opposed to autoregressive loop where you feed results from the previous batch into the next iteration and keep assembling final result). Only grab logits for the last chunk (what comes after "choice_index:" in our case). Important: send multiple questions in 1 go

Get rid of any results that are not our supported choice indices. Turn logits (random looking numbers like 3, 89, -386) into a list of probabilities that add up to 1. Get your answers with probabilities back.

View full thread

Figure out model's internal encoding representation for the IDs your system supports (up to 255 choices in this case). This extra PREFIX business seems to be dealing with possible different ways to encode the same value (e.g. "2") depending on whether it appears in "2" or "choice_index: 2"

Massage your prompt so it looks like "user says / assistant says" with our prefix at the very end so that the model is set up to spit out a single value for one of the choices.

Each question is framed as a _numbered_ multiple choice question. E.g. Options like "blue, red, yellow" become 0, 1, 2. The model will answer with the number.

What is the approach? According to Sean > As long as you have access to the logits (for structured outputs) and can prefill data into the prompt, you can turn any LLM into a general fast classifier.

If you find yourself lost in the woods, fuck it, build a house. “Well, I was lost, but now I live here! I have severely improved my predicament!” - Mitch Hedberg

Philip Nuzhnyi reposted

In case you weren't aware how the anonymous Hacker News moderators control the narrative in Silicon Valley. (Bluesky Communities can't come soon enough)

Jonathan Warden@jonathanwarden.com

Yeah that story got penalized. Orange is the front page ranking, black is the ranking it should have given the raw ranking formula with no penalties. HN moderators often apply penalties to news and political stories, even if they are very tech-adjacent. news.social-protocols.org/stats?id=496...

I found Jesus

Daniel van Strien@danielvanstrien.bsky.social

Need a historical illustration? Search 1.49 million images from British Library books and Britannica (1500s–1920s). Find similar images, download full-resolution images or transparent PNG cutouts, and explore the sources. huggingface.co/spaces/davan...

Philip Nuzhnyi reposted

We should stop thinking about software as anything other than getting the computer to do what we want it to do for us

Aug 31 Weeek in podcasts From "The Founders" on Claude Hopkins - he tours Schlitz's brewery to help them figure out how to advertise to get from 5th to near 1st most sold beer app.tldl.pro/episodes/430... "The maker is too close to his product. He sees in his methods only the ordinary." ....

View full thread

Techbrew on Fable 5.1 - app.tldl.pro/episodes/hey... "[...] changing the economics of running persistent agents, reducing the cost of cached context by 75%, and introducing a new security architecture [...] designed to let organizations retain monitoring data inside infrastructure they control"

Aug 31 Weeek in podcasts From "The Founders" on Claude Hopkins - he tours Schlitz's brewery to help them figure out how to advertise to get from 5th to near 1st most sold beer app.tldl.pro/episodes/430... "The maker is too close to his product. He sees in his methods only the ordinary." ....

"He does not realize that the world at large might marvel at those methods and that facts which seem commonplace to him might give him vast distinction. Tell the pains that you take to excel. [...] Serve better than others, offer more help than others, and you are pretty sure to win."

View full thread

How to fix this? One possibility: banning patterns in the system prompts. Tricky side effect - this might damage model’s reasoning. Another approach : don’t ban anything, let models write freely and de-cliché in a second pass

The AI tells are actually ancient, respected techniques: - parallelism: repeating the same sentence shape for rhythm - antithesis: «it’s not X, it’s Y »; made by Aristotle, perfected by JFK -> “Ask not what your country can do for you — ask what you can do for your country” - tricolon: the rule of 3

These devices are NOT bad. They’re the tools persuasive writing has always been made of. The problem is how frequently LLMs use them. LLMs end up reaching for them in every sentence/paragraph. Good human writers save it for special occasions