CocoIndex

CocoIndex

@cocoindex.bsky.social

#OpenSource Transform data for AI 🥥🌴 ⭐ https://github.com/cocoindex-io/cocoindex

46 followers30 following120 posts

View full thread

We use CocoIndex and @pydantic.dev to build the pipeline. CocoIndex is a data indexing framework for building incremental data transformation pipelines — it tracks what's been processed, so re-running the pipeline only processes new or changed episodes.

Podcasts are one of the richest sources of expert knowledge on the internet. A single Lex Fridman or Dwarkesh Patel episode can contain dozens of substantive claims about people, technologies, and organizations — but it's all locked inside hours of audio.

we'll build a cocoindex pipeline that turns YouTube podcast episodes into knowledge graph. The pipeline downloads audio, transcribes with speaker diarization, uses an LLM to extract structured statements and entities, resolves duplicates across episodes, and stores everything in @surrealdb.com

CocoIndex reposted

Our first San Francisco developer meetup set the bar high. 70+ attendees and a room of builders focused on giving production AI agents the right context.

Super excited to launch 𝐜𝐨𝐜𝐨𝐢𝐧𝐝𝐞𝐱-𝐜𝐨𝐝𝐞 𝐂𝐋𝐈 🚀 ! A super lightweight, 𝐀𝐒𝐓-𝐛𝐚𝐬𝐞𝐝 𝐜𝐨𝐝𝐞 #OpenSource 𝐂𝐋𝐈 pair with any coding agent that actually understands your codebase and just works and improves code completion speed and quality. CLI is having a comeback 🙂 and having its own moment 🔥 .

CocoIndex reposted

Replying to @jaimemh.bsky.social

Subscribe to our events calendar on Luma for all our meetups across NYC, London, SF, Amsterdam, and Asia: luma.com/surrealdb Next stop: San Francisco. We’ll see you there for our next SurrealDB meetup with @pydantic.dev and @cocoindex.bsky.social during NVIDIA GTC.

luma.comSurrealDB Events · Events CalendarView and subscribe to events from SurrealDB Events on Luma. Learn, connect, and build with SurrealDB: the multi-model database for AI agents. Discover upcoming SurrealDB events, webinars or community-...

CocoIndex reposted

CocoIndex reposted

Kicking off 2026 with Rust Delhi Meetup #12, and what a way to start. Thanks to Guinnes Singh for accepting my talk proposal & being a great host throughout. I presented on building incremental data pipelines with @cocoindex.bsky.social, a framework I've been contributing towards. (1/n)

Why build in #opensource? Best infrastructure is transparent, composable, community-driven. CocoIndex (Rust + Python): data transformation for AI with incremental sync, schema validation, context extraction. No vendor lock-in. No black boxes. Fast, type-safe pipelines. #rustlang #python

🚀 CocoIndex + DSPy: structured extraction from patient intake forms LLMs as typed extractors, not black boxes. Production-ready with incremental sync. No OCR/regex mess. Clean separation between reasoning (DSPy) & data infra (CocoIndex). 💻 github.com/cocoindex-io/cocoindex #AI #DataEngineering

github.comcocoindex/examples/patient_intake_extraction_dspy at main · cocoindex-io/cocoindexData transformation framework for AI. Ultra performant, with incremental processing. 🌟 Star if you like it! - cocoindex-io/cocoindex

🚀 CocoIndex just hit Rust Trending Again! 🦀 Why Rust? ⚡ Speed - C++ speed + memory safety 🔒 Reliability - No GC pauses or segfaults. Predictable performance always Speed + Fresh data = Critical infra for AI agents github.com/cocoindex-io... #Rust #AI #Opensource

github.comGitHub - cocoindex-io/cocoindex: Data transformation framework for AI. Ultra performant, with incremental processing. 🌟 Star if you like it!Data transformation framework for AI. Ultra performant, with incremental processing. 🌟 Star if you like it! - cocoindex-io/cocoindex

🦀 CocoIndex 0.3.10 drops today! Rust improvements: • Async runtime fixes (no deadlocks) • Smarter incremental recomputation • Optimized embedding batches • Robust file/streaming handling Why Rust for data? Memory safety + performance + async = reliable AI infra at scale

cocoindex.ioCocoIndex Changelog 0.2.21 - 0.3.10 | CocoIndexFeaturing batching support for CocoIndex functions, execution robustness, schema & type system improvements, custom source support, and more.

Context engineering just got easier 🚀 CocoIndex transforms how you build AI systems: ✨ Incremental processing - only recompute what changed 🔄 Real-time sync - keep context always fresh 🎯 Data lineage built-in ⚡ Rust-powered Perfect for RAG, knowledge graphs & AI agents github.com/cocoindex-io/c…

github.comGitHub - cocoindex-io/cocoindex: Data transformation framework for AI. Ultra performant, with incremental processing. 🌟 Star if you like it!Data transformation framework for AI. Ultra performant, with incremental processing. 🌟 Star if you like it! - cocoindex-io/cocoindex

Custom Sources in CocoIndex 🚀 Pull data from ANY system - APIs, databases, cloud storage. CocoIndex now ingests incrementally, tracks changes efficiently, and integrates seamlessly. No longer limited by prebuilt connectors. Use CocoIndex for anything. #opensource cocoindex.io/blogs/custom...

cocoindex.ioBring your own data: Index any data with Custom Sources | CocoIndexCocoIndex now officially supports custom sources — giving you the power to read data from any system you want. You can use CocoIndex for anything, and enjoy the robust incremental computing to build f...

Automatic Batching in CocoIndex 🚀 CocoIndex now supports automatic batching for all functions. ~5× higher throughput when embedding our codebase. Best part? Adaptive and knob-free - no timers, no tuning. Just set batching=True. #opensource cocoindex.io/blogs/batching

cocoindex.ioAdaptive Batching - 5x throughput on your data pipelines | CocoIndexDiscover how CocoIndex delivers automatic batch processing for GPU workloads and machine learning pipelines. Framework-level batching optimizes performance for text embeddings and other AI operations ...

CocoIndex v0.3.10 just shipped with automatic batching support! 🚀 ~5× faster throughput when embedding our codebase. Best part? Zero tuning needed - it's adaptive and knob-free. Just set batching=True. Read the benchmarks 👇

CocoIndex reposted

CocoIndex made to #1 Github trending global in Rust - data transformation engine. Grateful to the Rust open source community and all the amazing Rustaceans for the support 💛 #rustlang #github #opensource

A mental framework to really understand how Rust’s ownership, borrowing, and memory safety work. With this - moves, borrows, Send, Sync, and runtime checks become intuitive and predictable tools in your programming toolbox. cocoindex.io/blogs/rust-o... #rustlang #opensource #programming

cocoindex.ioThinking in Rust: Ownership, Access, and Memory Safety | CocoIndexA mental framework for Rust's memory safety concepts. Think systematically about ownership, references, Send, Sync, and Rc, Arc, RefCell, Mutex, etc.

CocoIndex reposted

CocoIndex is officially supporting custom targets - cocoindex.io/docs/custom_.... We believe this work will add more flexibility for using coco / bring your own lego for targets as well beyond the flow ops. Thanks our community for the great suggestions!

cocoindex.ioCustom Targets | CocoIndexLearn how to create custom targets in CocoIndex to export data to any destination including databases, cloud storage, file systems, and APIs. Build target specs and connectors with setup and data meth...

🚀 Build Your Own Google Photo Search with Face Indexing at Scale. We just dropped a new tutorial on building a scalable face recognition pipeline using CocoIndex and Qdrant. cocoindex.io/blogs/face-d... #opensource #faceRecognition #LLM #AI #startup #Qdrant #CocoIndex #ColdPlay

cocoindex.ioIndexing Faces for Scalable Visual Search - Build your own Google Photo Search | CocoIndexBuild a scalable face detection and recognition pipeline using CocoIndex. This tutorial covers extracting and embedding faces from images, structuring data for visual search, and exporting to a vect...

#windsurf #cursor #codingagents #llm #coding #tech

Linghua Jin@badmonster0.bsky.social

Context + Reliable LLM e.g., Claude code will be the future of coding agents. Checkout Cocoindex that brings fresh context for reliable coding agents! github.com/cocoindex-io... (Appreciate a star!) Full blog here: cocoindex.io/blogs/index-... #opensource #codeagent

CocoIndex reposted

Replying to @badmonster0.bsky.social

What you get: ✅ LLM-extracted title, authors, abstract ✅ Embeddings for smart search ✅ Build 3 tables in one shot: metadata, author→paper, embeddings ✅ Real-time updates via Postgres + PGVector (one-line switch to Qdrant) Appreciate a star on the repo if it is helpful github.com/cocoindex-io...

github.comGitHub - cocoindex-io/cocoindex: Data transformation framework for AI. Ultra performant, with incremental processing.Data transformation framework for AI. Ultra performant, with incremental processing. - cocoindex-io/cocoindex