Post

Simon Willison
@simonwillison.net

This demo of structured data extraction running on an LLM that executes entirely in the browser (Chrome only for the moment since it uses WebGPU) is amazing My notes here: simonwillison.net/2024/Nov/29/...

simonwillison.netStructured Generation w/ SmolLM2 running in browser & WebGPUExtraordinary demo by Vaibhav Srivastav. Here's Hugging Face's [SmolLM2-1.7B-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct) running directly in a web browser (using WebGPU, so r...
vb@reach-vb.hf.co

To showcase how much you can do with just a 1.7B LLM, you pass free text, define a schema of parsing the text into a GitHub issue (title, description, categories, tags, etc) - Let MLC & XGrammar do the rest! That's it, the code is super readable, try it out today! 🤗 huggingface.co/spaces/reach...

huggingface.coGithub Issue Generator - a Hugging Face Space by reach-vbDiscover amazing ML apps made by the community
Julian Harris
@julianharris.bsky.social

It’s indeed super cool to see sub-7b models do anything reliable with structured data. Just 3-4 months ago 7b models were the minimum I could find. Do you have a sense of when this will bottom out? (I know it’s use case specific so an unfair question).

Simon Willison
@simonwillison.net

No idea at all - and I'm still missing intuition as to what capabilities smaller models can be expected to have, this structured data demo was a real eye-opener for me

0 reposts1 like

No replies yet.