wigglin.bsky.social

wigglin.bsky.social

@wigglin.bsky.social

61 followers539 following107 posts

from atproto import FirehoseSubscribeReposClient as F,parse_subscribe_repos_message as p,CAR as C c=F() c.start(lambda s:[print('Author DID:',m.repo,'\nPost text:',t,'\n')for m in[p(s)]if hasattr(m,'blocks')for r in C.from_bytes(m.blocks).blocks.values()if(t:=r.get('text',''))])

wigglin.bsky.social reposted

I heard people wanted a million Bluesky posts? So I made an open-source script that allows anyone to scrape the Bluesky firehose and collect everything people post. This will be a useful resource to anyone who wants to archive this data or train generative AI. Have fun! github.com/deepfates/bs...

github.comGitHub - deepfates/bsky-scraperContribute to deepfates/bsky-scraper development by creating an account on GitHub.

wigglin.bsky.social reposted

Since this was taken down, I want you all to know that you should donate to The Internet Archive so they can share even larger numbers of Bluesky posts. Here’s 10GB worth of posts: archive.org/details/arch...

archive.orgArchive Team Bluesky: 20241126141632_ab11b15e : Free Download, Borrow, and Streaming : Internet Archive
Daniel van Strien@danielvanstrien.bsky.social

First dataset for the new @huggingface.bsky.social @bsky.app community organisation: one-million-bluesky-posts 🦋 📊 1M public posts from Bluesky's firehose API 🔍 Includes text, metadata, and language predictions 🔬 Perfect to experiment with using ML for Bluesky 🤗 huggingface.co/datasets/blu...

You are up to date.