Post

Cacheflowe
@cacheflowe.bsky.social

I've been working towards implementing a number of onnx computer vision models directly in TouchDesigner. It's wild how much python I'm writing these days. This is DepthAnythingV2 running at 60fps with a 256x256 input. ~10ms inference time on a laptop 4090 🤯

3 reposts22 likes

Piter Pasma
@piterpasma.nl

oh whoa that is crazy!! ... I've been running my rayhatcher/rayscratcher algos using AI models to generate(/hallucinate) their normal maps and/or depth maps

Kevin Nørby Andersen
@kevinandersen.dk

How did you learn this? I often find myself struggling to take a given model and make it work in different contexts. It sometimes feels like there's a fundamental interface between high-level programming languages and models that I'm missing.

Cacheflowe
@cacheflowe.bsky.social

Onnx models seem to be the most portable format across languages/frameworks. There’s an onnx SDK for lots of languages. I will admit that understanding the input/output format took a lot of time to understand, and I used some coding assistance to get there! I certainly spent days learning and trying

Nick Fox-Gieg
@n1ckfg.bsky.social

You take a Pytorch model and convert it to onnx--the tricky bit is that there's no generic converter; you need to modify the original Python inference code to write the file. But it's worth the effort imo because an onnx model can be used in many more places