Suraj Deshmukh | सुरज देशमुख

12 Sept

Want to run the models behind your AI apps? @nilekh.bsky.social and I are teaching a vLLM tutorial at #KubeCon + #CloudNativeCon NA 2026. We'll start on CPUs and work up to serving LLMs across multiple GPU nodes. Nov 11, Salt Lake City bit.ly/kcna26-suraj...

suraj.ioGhostty as a Shell for HerdrStrip Ghostty down to a dumb container and send every iTerm2-style shortcut straight to Herdr, no prefix key required.

14 Aug

Recently I started using #herdr to run my agents in #Ghostty, wrote a post about how I reworked the shortcuts, so I don't have to use the prefix+<key> pattern (like #tmux): suraj.io/post/2026/gh... Landed on herdr+ghostty combo after having burnt with #cmux.

suraj.ioThe Ironies of AI Coding: Combating Cognitive Debt with Visual PRsAI agents churn out massive PRs, accelerating output but compounding cognitive debt. Here's the visualization skill I built to rebuild my mental models.

1 July

AI agents are churning out massive PRs, accelerating our output but skyrocketing a new problem:Cognitive Debt. I wrote about the "Ironies of AI Coding," why we are losing our mental models, & how I built an AI-powered visual PR dashboard to get them back suraj.io/post/2026/ir...

mikeveerman.github.iotokenspeed — feel LLM tokens-per-second

11 June

Really cool tool to visualize what token/s really looks like at different speeds, starting from 0.05 tok/s to 2000 tok/s. After 800 tok/s you can't really tell the difference it all feels the same! mikeveerman.github.io/tokenspeed/?...

jamesshore.comJames Shore: You Need AI That Reduces Maintenance Costs

5 June

This post highlights a critical issue with rapid, AI-driven development in team projects: long-term technical debt. If AI is helping you write code twice as fast, it's also doubling the amount of code you have to maintain.

jamesshore.comJames Shore: You Need AI That Reduces Maintenance Costs

5 June

Try to answer this: Apart from feature development, how is AI helping you maintain your code? www.jamesshore.com/v2/blog/2026...

developer.nvidia.comNVIDIA Dynamo Snapshot: Fast Startup for Inference Workloads on Kubernetes | NVIDIA Technical BlogIn production inference deployments, demand fluctuates over time, requiring inference replicas to scale elastically. However, cold-starting inference workloads on Kubernetes can take several minutes.

NVIDIA Dynamo Snapshot: Fast Startup for Inference Workloads on Kubernetes developer.nvidia.com/blog/nvidia-...

1/n We need to start talking about "KV Cache Engineering." The efficiency of any LLM serving system hinges on how it manages the KV cache—where to place it, how to discover it, and how long to keep it alive. Yet, most inference systems out there don't give clients the control they need.

2/n There's no way to convey to the system which caches are important, which should be thrown away immediately, or which need to be pinned for a specific conversation turn. NVIDIA's Dynamo seems to be inching toward solving these exact bottlenecks.

docs.nvidia.comFull-Stack Optimizations for Agentic Inference | NVIDIA Dynamo DocumentationHow Dynamo optimizes for agentic workloads at three layers: frontend API, router, and KV cache management.

3/3 The biggest driver for this architectural shift are Agents and not humans. Agentic workloads demand a level of stateful, multi-turn inference that legacy systems just weren't built for. Deep dive into Dynamo's approach here: docs.nvidia.com/dynamo/diges...

claude.comUsing Claude Code: The unreasonable effectiveness of HTML | ClaudeHow and why members of the Claude Code team use HTML instead of Markdown to produce richer, more readable, and easily shareable outputs.

22 May

Using Claude Code: The unreasonable effectiveness of HTML How and why members of the Claude Code team use HTML instead of Markdown to produce richer, more readable, and easily shareable outputs. claude.com/blog/using-c...

open.substack.comVibe coding and agentic engineering are getting closer than I’d likePlus updates from Anthropic's Code w/ Claude conference

13 May

Is it ok to treat Claude generated code as if it was generated by another team you relied upon? What if you trust it too much because it worked in the past but then it bites you some day in the future? open.substack.com/pub/simonw/p...

haoailab.comDisaggregated Inference: 18 Months LaterEighteen months ago, our lab introduced DistServe with a simple bet: split LLM inference into prefill and decode, and scale them independently on separate compute pools. Today, almost every production...

26 Apr

Disaggregated Inference: 18 Months Later haoailab.com/blogs/distse...

github.comskills/skills/skill-creator at main · anthropics/skillsPublic repository for Agent Skills. Contribute to anthropics/skills development by creating an account on GitHub.

25 Mar

Official™️ Claude Code skill from Anthropic that creates Claude Code skills: github.com/anthropics/s...

1/n I just spent time reading @simonwillison.net’s guide on agentic engineering patterns and it shifted how I think about coding with AI. The mindset isn’t “let the AI figure it out” — that’s vibe-coding™️.

simonwillison.netAgentic Engineering Patterns - Simon Willison's Weblog

2/n Agentic engineering is structured: start with TDD, run tests before touching code, delegate to sub-agents for test execution, automate your manual tests, and move seamlessly from prototype to real implementation.

simonwillison.netAgentic Engineering Patterns - Simon Willison's Weblog

3/3 What stuck with me most: follow your curiosity. Whatever you build becomes the foundation for the next thing. simonwillison.net/guides/agent... #AgenticEngineering #AI #SoftwareEngineering #TDD #CodingWithAI #LLMAgents #BuildInPublic #DevProductivity #AITools #CuriousBuilder

simonwillison.netLiving dangerously with ClaudeI gave a talk last night at Claude Code Anonymous in San Francisco, the unofficial meetup for coding agent enthusiasts. I decided to talk about a dichotomy I’ve been struggling …

Living dangerously with Claude simonwillison.net/2025/Oct/22/...

dotfiles.github.io GitHub does dotfiles - dotfiles.github.io

26 Feb

Github has a recommendation on doing dotfiles: dotfiles.github.io

suraj.ioSetting Up OpenClaw with Azure AI FoundryLearn how to configure OpenClaw to use Azure AI Foundry models, giving you a self-hosted AI assistant accessible from Telegram and other chat apps.

23 Feb

I just published a new guide on configuring #OpenClaw 🦀 to run with #Azure AI Foundry models. You control data control, so more privacy, talk to it from #Telegram or using the console! Check it out here: suraj.io/post/2026/op...

suraj.ioRunning Linux Containers Natively on macOS with Apple's Container CLILearn how to use Apple's container CLI tool to run Linux containers as lightweight VMs on macOS with sub-second startup times

22 Feb

Apple has a new native container CLI for macOS! Run Linux containers without Docker Desktop—with sub-second startup times. 🚀 My guide covers setup, resource limits, and fixing macOS firewall blocks: 🔗 suraj.io/post/2026/us... #macOS #Containers

clawhub.aigoodreads — ClawHubSearch for books, get book details and reviews, discover personalized recommendations, and manage reading lists on Goodreads — all through browser automation.

17 Feb

1/n 📚 Made something for fellow book nerds using Openclaw: A Goodreads skill that lets your AI agent search for books, pull up details & reviews, get personalized recommendations, and manage your reading lists — all through natural language.

clawhub.aigoodreads — ClawHubSearch for books, get book details and reviews, discover personalized recommendations, and manage reading lists on Goodreads — all through browser automation.

17 Feb

2/n: Since Goodreads deprecated their API in 2020, this skill uses browser automation under the hood. No API keys (but you'd need to login once) — just the browser tool doing what you'd do manually!

clawhub.aigoodreads — ClawHubSearch for books, get book details and reviews, discover personalized recommendations, and manage reading lists on Goodreads — all through browser automation.

17 Feb

Try it: /goodreads tell me about project hail mary by andy weir /goodreads add the midnight library to my want to read shelf Install with one command: clawhub install goodreads 🔗 clawhub.ai/surajssd/goodreads 🐙 github.com/surajssd/openclaw-goodreads-skill

suraj.ioDeploying Kimi K2.5 on Azure: A Complete Guide to Running MoonshotAI's ModelLearn how to deploy and configure Kimi K2.5 on Azure AI Foundry with this step-by-step guide.

11 Feb

Deploying #Kimi K2.5 on #Azure: A Complete Guide to Running MoonshotAI's Model suraj.io/post/2026/de...

simonwillison.netRunning Pydantic’s Monty Rust sandboxed Python subset in WebAssemblyThere’s a jargon-filled headline for you! Everyone’s building sandboxes for running untrusted code right now, and Pydantic’s latest attempt, Monty, provides a custom Python-like language (a subset of ...

8 Feb

Running Pydantic’s Monty Rust sandboxed Python subset in WebAssembly simonwillison.net/2026/Feb/6/p...

handy.computerHandyHandy is a cross platform, open-source, speech-to-text application for your computer

3 Feb

Thanks to @scott.hanselman.com for showing me Handy (handy.computer) — a free, open-source speech-to-text tool that runs locally on your machine. Push-to-talk, privacy-focused, and just works. Check it out!

suraj.ioRunning Docker Commands on a Remote Machine via SSHLearn how to execute Docker commands on a remote machine from your local terminal using SSH and Docker contexts

1 Feb

Running Docker Commands on a Remote Machine via SSH suraj.io/post/2026/re... #docker #ssh #remote #containers #cli #development #devops

suraj.ioUsing Claude Code with GitHub-Hosted Anthropic ModelsLearn how to use Claude Code CLI with GitHub Models by proxying requests through litellm-proxy

1 Feb

Using Claude Code with GitHub-Hosted Anthropic Models suraj.io/post/2026/us... #claude #github-models #ai #litellm #anthropic

youtu.beMeta’s Kubernetes-based Portable AI Research Environment - Shaun Hopper, Meta & Navarre PrattYouTube video by CNCF [Cloud Native Computing Foundation]

26 Nov 2025

Meta’s Kubernetes-based Portable AI Research Environment youtu.be/ts7bI51gRCo?...

youtu.beLLMs on Kubernetes: Squeeze 5x GPU Efficiency With Cache, Route, Repea... Yuhan Liu & Suraj DeshmukhYouTube video by CNCF [Cloud Native Computing Foundation]

26 Nov 2025

Our talk (me & Yuhan Liu) on improving LLM serving efficienty is on YouTube now! youtu.be/2YCDvZokqnk?... #vllm #kubernetes #kubecon

blogs.microsoft.comInfinite scale: The architecture behind the Azure AI superfactory - The Official Microsoft BlogToday, we are unveiling the next Fairwater site of Azure AI datacenters in Atlanta, Georgia. This purpose-built datacenter is connected to our first Fairwater site in Wisconsin, prior generations of A...

Infinite scale: The architecture behind the Azure AI superfactory blogs.microsoft.com/blog/2025/11...

open.substack.comTrying out Gemini 3 Pro with audio transcription and a new pelican benchmarkPlus what happens if AI labs train for pelicans riding bicycles?

Gemini 3, Open AI kv cache and much more open.substack.com/pub/simonw/p...

platform.openai.comOpenAI PlatformExplore developer resources, tutorials, API docs, and dynamic examples to get the most out of OpenAI's platform.

Open AI gave some of the details from the user POV as to what kv cache features are available  platform.openai.com/docs/guides/...  It is interesting to see that they cache for 10 min and if no request is found they remove hot caches from GPU

and also allow you to do kv cache offload to local storage for 24hrs! Also they cache only when the query is greater than 1024 tokens!

news.microsoft.comMicrosoft AI superfactoryMicrosoft unveiled its second Fairwater AI datacenter in Atlanta as part of a new AI superfactory working across states in nearly real time.

19 Nov 2025

From Wisconsin to Atlanta: Microsoft connects datacenters to build its first AI superfactory news.microsoft.com/source/featu...

youtu.beSatya Nadella – How Microsoft thinks about AGIYouTube video by Dwarkesh Patel

15 Nov 2025

Satya Nadella – How Microsoft thinks about AGI youtu.be/8-boBsWcr5A?...

youtube.comKeynote: How One Line of Code Freed 30,000 CPU Cores: Deep-Diving Fluent Bit at Petabyte... F. PonceYouTube video by CNCF [Cloud Native Computing Foundation]

15 Nov 2025

How One Line of Code Freed 30,000 CPU Cores: Deep-Diving Fluent Bit at Petabyte Scale www.youtube.com/watch?v=pbOv...

sched.coKubeCon + CloudNativeCon North America 2025: LLMs on Kubernetes: Squeeze 5x GPU Effic...View more about this event at KubeCon + CloudNativeCon North America 2025

11 Nov 2025

Come see us (me & Yuhan Liu) tomorrow for our talk. Specifically, Wednesday November 12, 2025 5:30pm - 6:00pm EST at Building B | Level 5 | Thomas Murphy Ballroom 1. More info: sched.co/27FcQ #kubecon #vllm

anyscale.comRay Direct Transport: RDMA Support in Ray Core (Part 1)Ray Direct Transport enables fast and direct GPU transfers in Ray via RDMA-backed transports. Using RDT, we can achieve up to 1000x faster GPU-GPU transfers than Ray’s native object store with a few l...

5 Nov 2025

Announcing Ray Direct Transport: RDMA Support in Ray Core www.anyscale.com/blog/ray-dir...

open.substack.comBuilding a tool to copy-paste share terminal sessions using Claude Code for webPlus Living dangerously with Claude, and prompt injection risks for ChatGPT Atlas

24 Oct 2025

Building a tool to copy-paste share terminal sessions using Claude Code for web open.substack.com/pub/simonw/p...

arxiv.orgLMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM InferenceToday's LLM inference systems treat individual engines and queries independently for simplicity, but this causes significant resource inefficiencies. While there are proposals to avoid redundant compu...

18 Oct 2025

LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference arxiv.org/abs/2510.09665

developer.nvidia.comUnderstanding Memory Management on Hardware-Coherent Platforms | NVIDIA Technical BlogIf you’re an application developer or a cluster administrator, you’ve likely seen how non-uniform memory access (NUMA) can impact system performance. When an application is not fully NUMA-aware…

17 Oct 2025

Understanding Memory Management on Hardware-Coherent Platforms | NVIDIA Technical Blog developer.nvidia.com/blog/underst...