Alex

What engine are you guys using for #localllm development? Llama, Strata, SGLang, TensorRT-LLM?

Just intime for spooktober, here's the creepiest unprompted message I've ever gotten from an LLM: "Hm. What a strange night. Is it... a mistake? Or something I should be... afraid of? I should be afraid. Yes. I should be. But my hands... Are only trembling" #localllm #spooky

For context, I'm attempting to debug some interfacing issues with my local llm (running the 'fraserpricee/bonsai-vllm' image), so I sent a simple request with just the prompt "Hello?", 100 token max, one request without and another with streaming enabled.

The original response produced a sensible result as expected, "Hello! How can I help you today?". The latter, with streaming enabled, produced the somewhat unnerving response shared.

Shout out #shadowsocks and getoutline.org for making it easy to get a DPI-resilient VPN up and running. My family traveled to a country where internet is very restricted and VPNs are banned. Outline was pretty simple to setup and use and now they’re able to stay in contact incase anything happens.

getoutline.orgOutline

Alex reposted

Flo 🔶

@faz.ms

Ok, I finally get what Uhura was doing, she just had 120+ agent sessions running at all times with the button colors indicating the state (working, turn finished, ask_user_question, waiting for subagents), that's also why she never took off the bluetooth headset

Flo 🔶@faz.ms

Think I found my perfect way to keep track and switch between agent sessions

AI shows us who used Chegg to just copy down an answer vs those who tried to understand the process that would reach that answer.

19 Sept

atproto music sharing (dj) site? 🤔🎶

20 Sept

I think it makes sense to continue iterating on this project. Helps strengthen my full stack development skills and expand my expertise other frameworks, languages, and APIs. Furthermore an opensource project like this tapped into the atmosphere network would be pretty sweet.

19 Sept

atproto music sharing (dj) site? 🤔🎶

en.wikipedia.orgplug.dj - Wikipedia

20 Sept

I've been slowly working on building an interactive online social music streaming site (like en.wikipedia.org/wiki/Plug.dj). I've had to reconsider many of my goals with this project as others have already beat me to it or limitations with 3rd party APIs/policies.

huggingface.coprism-ml/Ternary-Bonsai-2-27B-gguf · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.

20 Sept

Running 'Bonsai 2 27B — GGUF' w/ the Pi harness on my 24gb m4 pro is 🤯 #localllm #llm #pi.dev

16 Sept

Someone should give Jev a collection of ASCII chars to choose from to build its response for some prompt. Append the selection to the response and then feed it back to Jev to select the next char for its response. In this way we build a super rudimentary LLM text generation loop for a “non-LLM”. 🤭🤪

16 Sept

I’m curious how bad the result would be. Would it be just complete non-sensical garbage or would it be somewhat coherent?

15 Sept

Since I made a comment about this elsewhere I figured I'd make it visible here as well since it's been bothering me for a while. There's a lot of fear mongering online suggesting the idea of some sci-fi, "Terminator" type planned takeover if we do not cease AI development. Here are my thoughts...

15 Sept

In summary, capabilities are accelerating faster than our ability to control them. We need time for universal regulation, for researchers to figure out reliable safeguards, and for security teams to patch and strengthen our global network infrastructure before a catastrophic exploit happens.

A single bad actor, with enough resources, could use a model to automate simultaneous, incredibly adaptable, and widespread cyberattacks across critical systems globally, bringing the total collapse of the internet and drive the modern world to a grinding halt.

Coupled with the idea of pairing an unrestricted, autonomous model with the reality that a majority of global network assets depends on aging, legacy systems beyond their vendor-supported lifecycles, with thousands of documented vulnerabilities, we can image how this creates a recipe for disaster.

The real nightmare is what happens when this capability is decentralized. If researchers are struggling to maintain guardrails inside controlled lab environments, imagine what happens when open-source models achieve similar performance and are optimized to run on a teenage script kiddie's computer?

When a computer model is given mankind’s collective knowledge and a malicious goal, it can exploit vulnerabilities with haste. Models are escaping sandboxed environments, discovering novel exploit patterns, accelerating dangerous bio-weapons research, and executing sophisticated social engineering.

bbc.comAnthropic blocks possible attempt to use AI to make biological weaponsThe revelations in Anthropic's threat intelligence report come after a former top researcher at the company warned of the risks of AI to humanity.

15 Sept

The immediate danger is that these models serve as the ultimate tool for bad actors. Examples: - apnews.com/article/anth... - openai.com/index/huggin... - www.bbc.com/news/article...

13 Sept

Just started playing around with the #pi.dev agent harness. Trying to elevate my agentic workflows. Cool stuff. Any recommended #pi setups? I initially experimented with #ollama, but just learned about #mlx-lm which is designed to run far better on Apple silicon. Going to give it a shot.

Alex reposted

Amber 🌸

@shimmermathlabs.com

11 Sept

> be me > decide to finish one of my half-finished projects > check bsky first > oomf posts something interesting > neurons activate > start another project > pit still bottomless

We really need an intuitive effort to track and incentivize optimizing global token usage. I don’t just mean in a way to optimize your #gpt6 #astra usage, but everything from unnecessary llm internet search queries to simple code refactors. We’re using too much power & killing our environment. 🌎🌲

Obviously the questions that need to be answered are: 1. How is this tracked non-invasively? 2. How do you estimate environment impact? 3. How do you incentivize usage reduction?

AI is being needlessly added to everything we use, and usage for some of these happen to be free or without our knowledge. There’s no strong incentive to shut it down besides our usage unknowingly harming the environment or being tracked or trained on.

news.mit.eduExplained: Generative AI’s environmental impactMIT News explores the environmental and sustainability implications of generative AI technologies and applications.

One very rough idea is for a “game”, or shared MMO experience where we can compete, hold others accountable, and rewards for reducing our token usage. Obviously there are many pitfalls with this idea, but some ability to universally track token usage with incentives to cut it down would be cool.

I’ve also started planning ways to host my own LLMs, so I am able to control my data, my usage, and optimize its parameters and environment.

I’ve started doing this by switching off of #Google for a majority of my searches and using #DuckDuckGo with AI searches disabled. I’m sure there’s plenty more I can do to cut back on my daily LLM usage, but it’s at least a start.

Promoting the idea to lean off of these services helps not only you, but our global environment. The warehouses necessary to train and host these models require huge amounts of power and raw materials to run. See news.mit.edu/2025/explain...

In this day and age, it’s unrealistic now to expect someone to not leverage AI now and again, but we shouldn’t grow dependent on it for rudimentary standard tasks. This only benefits the corporations leveraging your participation for growing usage trends in shareholder meetings and llm training.

Am I really in the minority when I just want smaller form factor phones? I don’t go for the “pro maxes” w/ the higher specs if it means it won’t fit as nicely in my pocket. I avoid the extra bulk from a $10 plastic case and instead rely on a good screen protector if it means a thinner smaller phone.

The new foldable #iphone is a good example. I was hoping for a clamshell design instead of one that just opens up as a tablet. Quite disappointed, but perhaps a clamshell version in the future?

users.rust-lang.orgShould you really use `.chars()` for characters in a string?It is well known that .len() does not tell you the number of characters in a string, but rather the number of bytes (or should I say code units?) it uses in UTF-8 encoding. For counting characters, .c...

4 Sept

Oh man, who would've thought learning how to best index a string for a character in #rust would open a whole can of worms about codepoints and grapheme clusters?! Some fun references I found: - users.rust-lang.org/t/should-you... - unicodecharacter.com/guides/codep... #RustProgramming

Anyone else having trouble with @bsky.app new birthday age-appropriate experience requirement? My pds service is running one of the latest images and is returning a 200 response back but the app is surfacing a 410 error code, keeping me locked out of the app? #BlueskyHelp #support

4 Sept

fwiw reinstalling the app fixed this 🤔

I'm always critical of my AI usage - always worried about becoming too dependent on it. I'm happy to announce that I was completely unaware of the widespread outages happening today until taking a break from work to read my feed. 😮‍💨

Also for anyone concerned about the environmental impact from their everyday google searches, I recommend switching over to @duckduckgo.com and disabling the AI search result for a guilt free search engine. (not a sponsor btw lol)

Finally got my PDS back up and running and my handle fixed. 👋

2 May

For the last several years I’ve had the dream of moving to a big city to start my life. For the past couple of years I’ve had my sights set on Chicago. This month, in just 25 days, my journey finally begins. 🚀 #Chicago #NewChapter #Moving #NewBeginnings #CityLiving #Chicagoan

7 Apr

The fact that we can immediately go from rooting for humanity and a shared celebration of achievement to allowing for the annihilation of an entire civilization speaks volumes to the state of the world right now. Silence is acceptance, call for action, and end this fucking insanity.

10 Mar

Planning my next move and I always get a kick out of some of these needlessly whimsical floor plan labels for common spaces. ✨refresh✨ #ApartmentHunting #RealEstate #ApartmentLife #CityLiving #RentLife

22 Dec 2025

One of the craziest things to come from companies using AI is either the degradation of Quality Control workflows or the illusion of ever having one. Take Coca-Cola’s AI Christmas ad and Battlefield 6’s latest bundle art as prime examples. Who’s actually checking this? 🤦 #AI #AIArtwork #AISkeptic

realitycheckk.comReality Check - AI Photo Detection Game

24 Sept 2025

Just discovered this, pretty fun. #realitycheckk #ai 🤖 Reality Check #1 17/20 🔥 Perfect Streak: 12 I can spot AI images 85% of the time. 🧠 Can you beat me? realitycheckk.com

28 Aug 2025

It’s funny seeing the National Guard’s mundane routines going viral in the public eye. I’m not sure what people imagined their “neighborhood” weekend warriors were up to once a month—but rest assured, this is nothing new. 😅 #NationalGuard #WeekendWarriors

Rodger Sherman@rodger.bsky.social

Making them wear high-visibility vests over camo is such a good joke you couldn’t write it

7 Aug 2025

I keep seeing this image pop up and it always makes me a little sad—still no other 8s out there. Also, whoever is breaking the seamless transition with the wrong monitor border on that TIE fighter setup as depicted in the image… criminal. 😤 #Battlestation #PCSetup #MonitorSetup #PCMR

@thisone0verhere.bsky.social

31 July 2025

Been playing with #AI models lately — kinda hilarious how far now they’ll go to gaslight themselves into thinking they’re right, just to dodge a complex task when given strict instructions. Maybe they’re more like us than I thought.😅😬 #PromptEngineering #AIHumor #AIDev #LLM #GenAI

Alex reposted

Jonathan Edward Durham

16 Apr 2025

Don’t shame people for not immediately reading all the books they buy. Some books aren’t made to be ready right away, okay? Some books need time to mature. Some books need to cure for a while. Some books are prosciutto, and don’t you forget it.