Post

Run any model locally with React Native AI and AI SDK ⚡️

What is going on here? - MLC is an optimized runtime for running different models on iOS, Android, and desktop GPUs - We download the model weights onto the device - Finally, we load it into the engine and set up everything needed for inference

Unlike the Apple provider, the MLC provider doesn’t rely on system-wide models. It downloads weights for each model you use, which can take time.

It's an interesting alternative: you can mix both providers, e.g. run Apple on-device first while your remote model downloads in the background. x.com/grabbou/sta...