LLM360 (@llm360.bsky.social)

LLM360

@llm360.bsky.social

Working on fully open-source LLMs and training data. We believe in community-owned AI. https://www.llm360.ai

1.2K followers 32 following14 posts

Pinned

LLM360

@llm360.bsky.social

20 Nov 2024

We've made a starter pack for researchers/organizations working on open-source LLMS. Please let us know if we missed you or if you'd like to be added! go.bsky.app/FELkyDr

go.bsky.appOpen-source LLMsJoin the conversation

LLM360 reposted

Nathan Lambert

@natolambert.bsky.social

10 Dec 2024

Made a list of resources for open source language models with @soldaini.net ahead of the tutorial tomorrow at 930 AM. github.com/allenai/awes...

github.comGitHub - allenai/awesome-open-source-lms: Friends of OLMo and their links.Friends of OLMo and their links. Contribute to allenai/awesome-open-source-lms development by creating an account on GitHub.

LLM360

@llm360.bsky.social

19 Nov 2024

📢📢 Check out: TxT360: a globally deduplicated dataset for LLM pretraining 🌐 99 Common Crawls 📘 14 Curated Sources 👨‍🍳 recipe to easily adjust data weighting and train the most performant models Dataset: huggingface.co/datasets/LLM... Blog: llm360-txt360.hf.space

Building on FineWeb’s global deduplication findings, we introduce a strategic upsampling recipe which outperforms FineWeb using TxT360. Full details are in the Upsampling Experiment section of the release blog.

LLM360

@llm360.bsky.social

19 Nov 2024

🌍🌎The global deduplication process was hairy 🙈 - and we want to share every detail. The TxT360 dedup pipeline can be recreated and used for other datasets. We include our tips and tricks in a tell-all write up in the release blog: llm360-txt360.hf.space huggingface.co/spaces/LLM36...

huggingface.coTxT360: Trillion Extracted Text - a Hugging Face Space by LLM360Discover amazing ML apps made by the community

🪟🛠️LLM360 is committed to making open source AI accessible, transparent, and reproducible. High-quality data is the first step toward better open source models...and we are excited to join the party contributing the first globally deduplicated dataset containing 5.7T tokens!

You are up to date.