Ishan Khatri

9 July 2025

My team at Stack just open sourced our runtime shape checking library for PyTorch & Numpy. It's hugely inspired by @patrickkidger.bsky.social's Jaxtyping but fixes a few key issues we ran into (namely, compatibility with torch.jit.trace & torch.compile). Try it with `pip install dltype`

Overdue career update time! I’ve moved to Toronto and will be starting graduate school (MS/PhD) @utoronto.ca & @vectorinstitute.ai, advised by Florian Shkurti! I will remain at Stack AV until the end of the year to wrap up some projects, after which I’m open for internships!👇

View full thread

Last but certainly not least, Kyle and Neehar, the entire reason that research has been fun is you guys and I wouldn’t be here without you.

I’ll have a few more exciting things to share as my time at Stack comes to a close, and I’m looking forward to the next big adventure 🫡

Finally I want to thank a ton of people. All of my colleagues from Argo, Motional & Stack and my management chain who supported me the entire time. These are all incredible places to work, filled with the smartest people and if you’re looking to work in autonomous driving you should 100% apply.

It’s thanks to my internship at Argo that I met and collaborated with the best in this field, and I’m eternally grateful to Deva for taking me under his wing and the entire lab at CMU (Tarasha, Jenny, Aljosa, Jono) for adopting me.

Over the past 6 years working on autonomous driving I have learned so much about deploying robotics systems into the real world. I'll be taking this experience and pivoting slightly to focus on dexterous manipulation and related topics.

23 Sept

Quick belated paper promotion. We had 2 papers at ECCV that I was in Malmo to present! UniFlow is a foundation model for 3D LiDAR scene flow, demonstrating zero-shot generalization across datasets and LiDAR sensors for the first time. We also showed that semantics don’t generalize the same way.

23 Sept

And OV3D-Bench is a re-evaluation of 3D monocular object detectors. Honestly a reckoning that’s a long time coming. I love this field but this “target aware” evaluation crap needed to go. You can’t prompt with only classes that are present in the image!!! This hides the false positives…

23 Sept

… and artificially inflates the metrics. We re-evaluate using sane metrics and find that open vocabulary models suck at classification. It turns out if you just compute SigLIP embeddings on the crops and re-classify within the taxonomy using cosine similarity, you get much better results.

Ishan Khatri reposted

Nathan Lambert

@natolambert.bsky.social

9 Sept

It's a common agreement among my friends not at OpenAI/Anthropic that people at the labs operate with a religious energy (Ant especially). It manifests very out of touch interactions, which carries into public comms here (e.g the recent quitting). Company cultures create this.

2 Sept

Agreed. I feel like I’m in loony toons land every time I see anyone talk about this. Even people who I respect are falling into this trap and it’s leaking outside the AI bubble and into the mainstream + the government. Giving legitimacy to this nonsense is damaging our field!!!

Ben Recht@beenwrekt.bsky.social

The overreaction to this Hugging Face thing is driving me nuts. Come flay me for being wrong but... 1/x

2 Sept

And the gall of OAI and other modeling companies to use their incompetence as a marketing exercise. Like I know u guys have to pump the stock but really??? As a reminder *taps the sign*

2 Sept

bsky.app/profile/isha...

Ishan Khatri@ishan.khatri.io

A surprising amount of AI fear mongering can be boiled down to this simple trope. See also: PROGRAMMER: “I gave an LLM access to launch the nukes with no oversight” LLM: “I launched the nukes” PROGRAMMER: “Oh no how could you have done this?!? AI is so unsafe”

5 Aug

Becky chambers is the GOAT. U should read this book if you haven’t already :)

Naomi Saphra@nsaphra.bsky.social

15. “Record of a Spaceborne Few“ is Becky Chambers’ greatest book to date. It doesn’t require reading the other Wayfarers. Beautiful, I view it as a response to the last page in The Dispossessed, which asks how spacefaring anarcho-syndicalists with scarce resources should handle immigration.

@eugenevinitsky.bsky.social

29 July

Yeah this is a banger format

Eugene Vinitsky 🍒@eugenevinitsky.bsky.social

My favorite thing about this is you can interpret it to be subtweeting literally anything

Ishan Khatri reposted

Eugene Vinitsky 🍒

15 June

We're hiring a postdoc to push a moonshot that mixes world-modeling, unsupervised RL, and autonomy. More details here, and please help us get the word out! docs.google.com/document/d/1...

docs.google.comEMERGE Lab World Models / Unsupervised RL PostdocPOSTDOCTORAL RESEARCHER WORLD MODELING / UNSUPERVISED RL EMERGE Lab The EMERGE lab at NYU is looking to hire a postdoc for an exciting moonshot project aimed at exploring the limits of learning in ...

14 June

It’s a beautiful day to make some GPUs go brr

10 Apr

The exact number really doesn't matter here, but it's super nice to see that people are actually reading the things I have worked on :)

7 Mar

Am I allowed to tell an AC that I'm too stupid to review a paper?

youtu.beI Trained My Own AI... It beat ChatGPTYouTube video by PewDiePie

28 Feb

Guys I never expected to go from watching this guy play minecraft as a kid (this is why I learned how to code btw) to watching him train LLMs. What the actual fuck: youtu.be/aV4j5pXLP-I?...

22 Feb

The last time I can remember a public official who was this effortlessly cool is Obama.

Mayor Zohran Kwame Mamdani@mayor.nyc.gov

The call has been made! NYC Public Schools will have a full snow day tomorrow.

theverge.comDiscord will require a face scan or ID for full access next monthAge verification for all.

18 Feb

Made a dedicated account for photography. Best viewed on @flashes.blue :)

Ishan Khatri | Photography@ishankhatri.com

Discovery.

Ishan Khatri reposted

Jess Miers 🦝

@jmiers230.bsky.social

10 Feb

The Discord age verification discourse focuses on adults losing access to communities they have a right to access. That's important. I'm also worried, though, about the kids. They seem to be an afterthought in these conversations. They have much to lose too. 🧵 www.theverge.com/tech/875309/...

Thorne 🌸 - Play Cozysters! 🎢

7 Feb

Inside you there is an agent swarm

Grace@gracekind.net

Inside you there are tens of trillions of agents

Ishan Khatri reposted

@ens0.me

2 Feb

Replying to @ens0.me

I would be deeply disappointed if Gavin is the nominee I would still vote for him, and so should you

kimi.comKimi K2.5: Visual Agentic IntelligenceTry Kimi K2.5, the strongest open-source model for visual coding. Explore agent swarm preview for massive tasks. Simplify complex Office work with precision.

2 Feb

Chris Paxton@cpaxton.bsky.social

Moltbook

👀

Uhh guys… did they just build gastown into the model?????? www.kimi.com/blog/kimi-k2...

20 Jan

Guys I just read Gas Town… what the fuck?

steve-yegge.medium.comBAGS and the Creator EconomyI’ve told this bizarre story to a few people now, including my wife Linh, who was certain it was a scam, and everyone’s eyes cross trying…

20 Jan

There’s a fucking meme coin?!?!?!?!? steve-yegge.medium.com/bags-and-the... Note: under no circumstances should be linking this article be seen as a promotion of this or any other crypto. IMHO meme coins are a scourge on society and gambling should be illegal in all its forms.

20 Jan

Yes….But also I think that this just proves programmers will still be needed… you need to acquire the skills somehow? Everyone else is just playing slots… maybe the barrier to entry for the skills is lower? Though tbh id argue for coding it was already low (see self-taught boot campers)…

alice valiant 🎃@alice.strange.domains

if this is how you use claude code then I’m sorry to say: skill issue, not the LLM’s fault

17 Jan

Fellow Linux enjoyers, is Gnome still the best DE with good touch support? Or is there something else I can use? (Have an older surface pro tablet I’d like to use for notes and pdfs etc)

16 Jan

me sowing: "I'll just use some attention here, a bit of softmax there ezpz" me reaping: inf/0 = NaN hahahahahahahaha

citationneeded.news“Wait, not like that”: Free and open access in the age of generative AIThe real threat isn’t AI using open knowledge — it’s AI companies killing the projects that make knowledge free

15 Jan

Wrote a whole ass thread om this a while ago: bsky.app/profile/isha...

Molly White@molly.wiki

This is how free and open access works. There is no way to have a free and open access project and then go “wait, no, not like that”. www.citationneeded.news/free-and-ope...

vedder.ioA Love Letter to CVPRA Love Letter to CVPR

15 Jan

Kyle is unfortunately not on BlueSky but he wrote this awesome blog post which is a love letter to CVPR. A conference that has in many ways become our “home conference”. It’s a reflection on the past and a call to action to preserve the incredible culture of CVPR. vedder.io/misc/lover_l...

Ishan Khatri reposted

Colin

@colin-fraser.net

8 Jan

Some vibe-coding thoughts: I think the zero-coding-experience vibe coders miss how much of what impresses them when Claude produces what they perceive to be a fully functional app comes from the ready-built frameworks that enable the construction of such apps just by writing some boilerplate.

7 Jan

The biggest misconception I see online about self-driving is that end-to-end == no LiDAR (camera only). The sensor suite doesn’t determine the algorithmic approach, you can have an end to end approach with multiple sensors 😑

Alright in the interest of creating the kind of discussion/posting I want to see on this site, I'm gonna share something that I learned in the past few weeks. Not new, not mind blowing, but I was reading the VAE and VQ-VAE papers this winter break for some projects so here we go 🧵

Anyway this isn't mindblowing or anything but it's just a neat nuance of these two types of models that I only really understood after implementing them from scratch with some toy data to get a feel for how they worked. Highly recommend this exercise if you really want to understand something deeply

arxiv.orgAuto-Encoding Variational BayesHow can we perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets? We in...

Arxiv links so that it gets picked up by the paper discussion feed. VAE: arxiv.org/abs/1312.6114 and VQ-VAE: arxiv.org/abs/1711.00937

The interesting part is that now you can't do generation anymore! Because the latents in the codebook don't follow any particular distribution you can't just sample from them and get a coherent image anymore!

The issue is that you don't know which embedding for a certain patch should be used together with the others. This is why you need another model along with a VQ-VAE (like a transformer or CNN) to do next token prediction or mask infilling in the embedding space in order to actually generate images.

You can think of a VAE like an image compression algorithm. Encode to a latent & then reconstruct via the decoder. Except in a regular VAE there's regularization to keep the distribution of the latent within a Gaussian. This is what lets you "generate" images...

because you can then sample noise from that Gaussian and use the decoder to turn it into an image. In a VQ-VAE instead of regularizing by matching to a Gaussian, you instead of a fixed set of embedding vectors (called the "codebook") and you snap the latents to their nearest neighbor in the codebook

Working on this 🦾

hailey@hailey.at

what i want from ai: please just do my laundry for me and make me tasty snacks what im getting from ai: here’s some generated anime waifus that we think you might enjoy

stackav.comICCV 2025 - Stack AVJoin Stack AV at ICCV 2025 - Booth #230. Explore how we're advancing autonomous trucking through world-class robotics and computer vision.

8 Oct 2025

Stack is going to be at ICCV with a booth! And I'm hiring interns for next summer! If you want to work on 4D reconstruction, novel view synthesis and "world modeling" for AVs come to our booth (#230) at ICCV and chat with our team! stackav.com/announcement...

8 Oct 2025

We're doing cutting edge work on dynamic reconstructions and deploying this technology as the basis for our data pipeline at Stack. If you want to do amazing research that has real world impact, on a team with insanely talented people then you should join us!