merve

merve

@merve.bsky.social

proud mediterrenean 🧿 open-sourceress at hugging face πŸ€— multimodality, zero-shot vision, vision language models, transformers

8.6K followers675 following242 posts

merve
@merve.bsky.social

Why do people sleep on DSE multimodal retrieval models? πŸ‘€ They're just like ColPali, but highly scalable, fast and you can even make them more efficient with binarization or matryoshka with little degradation πŸͺ†βš‘️ I collected some here huggingface.co/collections/...

merve
@merve.bsky.social

I'm so hooked on @hf.co Inference Providers (specifically Qwen2.5-VL-72B) for multimodal agentic workflows with smolagents πŸ₯Ή get started ‡️ > filter models provided by different providers > test them through widget or Python/JS/cURL

merve
@merve.bsky.social

Hello friends πŸ‘‹πŸΌ If visit Turkey this summer, know that millions of Turkish people are doing a boycott, once a week not buying anything and rest of the week only buying necessities if you have plans, here's a post that summarizes where you should buy stuff from www.instagram.com/share/BADrkS...

instagram.comLogin β€’ InstagramWelcome back to Instagram. Sign in to check out what your friends, family & interests have been capturing & sharing around the world.

merve reposted

merve
@merve.bsky.social

SmolVLM paper is out and it's packed with great findings on training a good smol vision LM! Andi summarized them below, give it a read if you want to see more insights 🀠

Andi@andimara.bsky.social

Today, we share the tech report forΒ SmolVLM: Redefining small and efficient multimodal models. πŸ”₯ Explaining how to create a tiny 256M VLM that uses less than 1GB of RAM and outperforms our 80B models from 18 months ago! huggingface.co/papers/2504....

huggingface.coPaper page - SmolVLM: Redefining small and efficient multimodal modelsJoin the discussion on this paper page
merve
@merve.bsky.social

DO NOT SLEEP ON THIS MODEL Kimi-VL-A3B-Thinking is the first ever capable open-source reasoning VLM with MIT license ❀️ > it has only 2.8B activated params πŸ‘ > it's agentic πŸ”₯ works on GUIs > surpasses gpt-4o I've put it to test (see below ‡️) huggingface.co/spaces/moons...

View full thread
merve
@merve.bsky.social

DO NOT SLEEP ON THIS MODEL Kimi-VL-A3B-Thinking is the first ever capable open-source reasoning VLM with MIT license ❀️ > it has only 2.8B activated params πŸ‘ > it's agentic πŸ”₯ works on GUIs > surpasses gpt-4o I've put it to test (see below ‡️) huggingface.co/spaces/moons...

merve
@merve.bsky.social

This model consists of a dynamic res handling MoonViT encoder, a projection layer and a 16B MoE decoder (with 2.8B active params) the paper introduces an interesting pre-training pipeline to handle long context and the model saw 4.4T tokens arxiv.org/pdf/2504.07491

merve
@merve.bsky.social

InternVL3 is out πŸ’₯ > 7 ckpts with various sizes (1B to 78B) > Built on InternViT encoder and Qwen2.5VL decoder, improves on Qwen2.5VL > Can do reasoning, document tasks, extending to tool use and agentic capabilities πŸ€– > easily use with Hugging Face transformers πŸ€— huggingface.co/collections/...

merve reposted

jsulz
@jsulz.com

Xet infra now backs 1000s of repos on @hf.co , which means we get to put on our researcher hats and peer into the bytes πŸ‘€ πŸ€“ Xet clients chunk files (~64KB) and skip uploads of duplicate content, but what if those chunks are already in _another_ repo? We skip those too.

huggingface.coFrom Chunks to Blocks: Accelerating Uploads and Downloads on the HubWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
merve
@merve.bsky.social

SmolVLM paper is out and it's packed with great findings on training a good smol vision LM! Andi summarized them below, give it a read if you want to see more insights 🀠

Andi@andimara.bsky.social

Today, we share the tech report forΒ SmolVLM: Redefining small and efficient multimodal models. πŸ”₯ Explaining how to create a tiny 256M VLM that uses less than 1GB of RAM and outperforms our 80B models from 18 months ago! huggingface.co/papers/2504....

huggingface.coPaper page - SmolVLM: Redefining small and efficient multimodal modelsJoin the discussion on this paper page
merve
@merve.bsky.social

X'in politikaları sebebiyle işimle alakalı post'ları burada da paylaşıyor olacağım, takip edebilirsiniz 😊

merve
@merve.bsky.social

All the multimodal document retrieval models (ColPali, DSE et al) are now under visual document retrieval at @hf.co πŸ“πŸ€— take your favorite VDR model out for multimodal RAG 🀝

merve reposted

Andi
@andimara.bsky.social

Introducing the smollest VLMs yet! 🀏 SmolVLM (256M & 500M) runs on <1GB GPU memory. Fine-tune it on your laptop and run it on your toaster. πŸš€ Even the 256M model outperforms our Idefics 80B (Aug '23). How small can we go? πŸ‘€

merve
@merve.bsky.social

ByteDance just dropped SA2VA: a new family of vision LMs combining Qwen2VL/InternVL and SAM2 with MIT license πŸ’— The models are capable of tasks involving vision-language understanding and visual referrals (referring segmentation) both for images and videos ⏯️

merve
@merve.bsky.social

The model is very interesting, it has different encoders for different modalities each (visual prompt, text prompt, image and video) then it concatenates these to feed into LLM πŸ’¬ the output segmentation tokens are passed to SAM2, to sort of match text (captions or semantic classes) to masks ‡️

View full thread
merve
@merve.bsky.social

you can still do traditional tool calling where you can do tool calling with JSON writing a tool and using it is very easy, just decorate the function with `@tool` what's cooler is that you can push and pull tools from Hugging Face Hub! see below

merve
@merve.bsky.social

smolagents is a barebones library to unlock both native and traditional tool calling for language models LLMs can already write code and do reasoning, so why bother yourself with writing the tool? CodeAgent class is here for it! see it in action below

merve
@merve.bsky.social

It is very easy to use CodeAgent! Just initialize it with the tool of your choice and the model of your choice See below how you can get started, you can use the models with HF Inference API as well as locally!

merve
@merve.bsky.social

you can now stay up-to-date with big AI research labs' updates on @hf.co easily over org activity page πŸ₯Ή I have been looking forward to this feature as I felt most back to back releases are overwhelming and I tend to miss out 🀠

merve
@merve.bsky.social

Aya by Cohere For AI can now see! πŸ‘€ C4AI community has built Maya 8B, a new open-source multilingual VLM built on SigLIP and Aya 8B 🌱 works on 8 languages! πŸ—£οΈ The authors extend Llava dataset using Aya's translation capabilities with 558k examples! works very well ⬇️ huggingface.co/spaces/kkr51...

merve
@merve.bsky.social

VLMs go MoE ✨ DeepSeek AI dropped three new commercially permissive vision LMs based on SigLIP encoder and their DeepSeek-MoE decoder 🐳 the models come in 1.0B, 2.8B and 4.5B active params πŸ₯Ή models seem to catch up with state-of-the-art with less active parameters! huggingface.co/collections/...

merve reposted

Linoy Tsaban πŸŽ—οΈ
@linoy.hf.co

'tis the season of open-source video models πŸŽ„πŸ“Ήβš‘οΈ Tencent just dropped the weights for ✨HunyuanVideo✨ - 13B parameters - competitive with closed source - code & weights released - demo coming soon πŸ”₯

merve
@merve.bsky.social

this was SmolVLM fine-tuning, and you can also do this! πŸ€— I made a notebook that includes all the goodies: QLoRA, gradient accumulation, gradient checkpointing with explanations on how they work πŸ’ below snapshot is with bsz=4 with simulated bsz=16 on L4 🀠 github.com/huggingface/...

merve@merve.bsky.social

something small but might approaches

merve
@merve.bsky.social

people on this platform will take your words out of context, twist, not mention your correction, because they just want to hate on what you work on, and insult you comfortably. I'll keep posting here about my work but will not be interacting with anyone who wants to bash on my company.

merve
@merve.bsky.social

I'm a center left/social democrat, who doesn't like neoliberalism as much as the next person, it was why I wanted to give this platform a chance. but the hate here is even worse and unhinged than X. I am baffled

merve
@merve.bsky.social

muting this conversation πŸ‘‹πŸΌ with some people there's not much winning really, nothing actionable I will keep shipping & posting about my projects, thanks everyone for the input πŸ€—πŸ§‘

merve@merve.bsky.social

It's pretty sad to see the negative sentiment towards Hugging Face on this platform due to a dataset put by one of the employees. I want to write a small piece. 🧡 Hugging Face empowers everyone to use AI to create value and is against monopolization of AI it's a hosting platform above all.