Post

Simon Willison
@simonwillison.net

This demo of structured data extraction running on an LLM that executes entirely in the browser (Chrome only for the moment since it uses WebGPU) is amazing My notes here: simonwillison.net/2024/Nov/29/...

simonwillison.netStructured Generation w/ SmolLM2 running in browser & WebGPUExtraordinary demo by Vaibhav Srivastav. Here's Hugging Face's [SmolLM2-1.7B-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct) running directly in a web browser (using WebGPU, so r...
vb@reach-vb.hf.co

To showcase how much you can do with just a 1.7B LLM, you pass free text, define a schema of parsing the text into a GitHub issue (title, description, categories, tags, etc) - Let MLC & XGrammar do the rest! That's it, the code is super readable, try it out today! 🤗 huggingface.co/spaces/reach...

huggingface.coGithub Issue Generator - a Hugging Face Space by reach-vbDiscover amazing ML apps made by the community

23 reposts185 likes

Julian Harris
@julianharris.bsky.social

It’s indeed super cool to see sub-7b models do anything reliable with structured data. Just 3-4 months ago 7b models were the minimum I could find. Do you have a sense of when this will bottom out? (I know it’s use case specific so an unfair question).