Egor Tarasenko

Egor Tarasenko

@dataish.bsky.social

Data & AI Engineer building scalable data platforms and AI systems with real-world impact. https://www.linkedin.com/in/egorseno/

7 followers2 following134 posts

EU AI Act transparency rules kicked in this week, disclose when users are talking to AI, mark synthetic content, or face fines up to €15M. Where do you land? 1. Good start 2. Right idea, too coarse 3. Compliance theater 4. Not my problem (no EU) #EUAIAct #AIGovernance #AI

DuckDB now runs inside the actual dbt Fusion engine, not just the old adapter -- full local lineage, zero warehouse cost. Real catch: dev/prod become different SQL dialects, not just different manifests. Test before you trust. Full breakdown: t.me/broken_pipel... #dbt #DataEngineering

t.meBroken PipelinesYou don't need a warehouse account to learn dbt anymore. dbt Labs shipped DuckDB support for the Fusion CLI in beta. Not the old dbt-duckdb adapter -- that's existed for years. This is the actual Fus...

Custom dbt schemas + --defer aren't two separate tricks for local dev and CI -- same manifest, same contract. Watch out though: dbt Core and Cloud don't always generate identical manifests. Full breakdown: t.me/broken_pipel... #dbt #DataEngineering

t.meBroken PipelinesPeople tend to treat "fast local dev" and "safe CI" as two separate dbt problems. They're the same problem: how do you avoid rebuilding the whole DAG just to test a change. The fix is the same in bot...

AI doesn't need less data engineering, it needs more. Every RAG setup, every fine-tune, is downstream of clean lineage and sane schemas. Models are non-deterministic. The pipelines feeding them can't be. Agree?

Text answers from your data are nice. Charts are nicer. We taught our Cube + RAG stack to render BI charts straight in the chat β€” ask in plain English, get a governed, non-hallucinated graph in ~11 seconds. Video + writeup: telegram.me/broken_pipel... #ConversationalBI #DataEngineering

telegram.meBroken PipelinesRound three of the experiment. The 2-3 minute crowd keeps watching, so here's another blogpost in a nutshell. Last time we put RAG on top of Cube so you could ask questions in plain English and get an...

Airflow 3.3 fixed the retry problem every data engineer has been burned by... You know this one. A task submits a Spark job. The job runs fine, but the worker dies -- OOM, node rotation, whatever. Airflow retries the task, and it submits a NEW one.... t.me/broken_pipel...

t.meBroken PipelinesAirflow 3.3 finally fixed the retry problem every data engineer has been burned by πŸ”₯ You know this one. A task submits a Spark job. The job runs fine, but the worker dies -- OOM, node rotation, whatev...
View full thread

A few months ago I started Broken Pipelines -- a Telegram channel about data engineering. Yesterday it passed 100 subscribers πŸŽ‰ Quick story of why it exists.

View full thread

A few months ago I started Broken Pipelines -- a Telegram channel about data engineering. Yesterday it passed 100 subscribers πŸŽ‰ Quick story of why it exists.

I'm Egor, a Data & AI Engineer. I spend my days building data platforms -- Airflow pipelines, Trino, dbt, lakehouses, non-generic plugins, custom data solutions, and open-source projects. All the plumbing that turns raw chaos into data someone can actually trust.

Somewhere along the way I noticed a gap: everyone talks about AI models, almost nobody talks about what feeds them. AI is only as good as its data. Someone has to build those systems -- and that someone is us, the data folks.

"dbt Core vs Cloud" is a false binary. dbt is five decisions -- development, execution, CI, orchestration, metadata -- and you don't have to answer them the same way. Where I landed: development + execution in Cloud (the IDE won). CI in Core -- Cloud's CI is opinionated, my workflow disagrees.

Orchestration in Airflow: triggering Cloud jobs is one operator; wiring non-dbt dependencies around Cloud's scheduler is webhooks. And keep a dbt Core backup image regardless. same code, same repo, a DR plan that costs one Dockerfile. When dbt Cloud is down, "fully managed" means fully waiting.

DuckLake 1.0 as the Iceberg killer? Apples to cars. Iceberg's JSON-file mess is the price of engine neutrality; DuckLake reintroduces the exact DB dependency table formats were built to remove. It's Postgres + Parquet, formalized. A tidy garage doesn't overtake a shipping port.

Fable 5 is back after 18 days. The "jailbreak" was reading code and finding bugs -- GPT-5.5 and even Haiku could already do it. The real news: Anthropic now owes the gov't pre-release access + jailbreak disclosure, permanently. Users pay via usage cuts. Enforcement first, rules after. #AI #Anthropic

OpenSpec sat in my feed for 4 months. I kept scrolling -- I've watched too many "spec-driven" frameworks become folders nobody opens. Then I had 3 people and an agent in one dbt repo and nobody could say what our models were even supposed to do. So I finally tried it.

openspec.devOpenSpec β€” A lightweight spec‑driven frameworkOpenSpec is a lightweight, spec‑driven framework for coding agents and CLIs β€” universal, open source, and no API keys or MCP required.

Simpler than the hype. Two folders in git: specs/ is the truth, changes/ is work in flight like a feature branch. What won me over: spec review is separate from code review. A spec diff shows what behavior changed -- readable in seconds, not a 600-line PR at 2am.

openspec.devOpenSpec β€” A lightweight spec‑driven frameworkOpenSpec is a lightweight, spec‑driven framework for coding agents and CLIs β€” universal, open source, and no API keys or MCP required.

The catch: it only knows about changes that go through it. The second a teammate edits a model and merges to main, the spec doesn't move -- your source of truth quietly starts lying. So: how do you handle manual vs agent changes in one repo? openspec.dev

openspec.devOpenSpec β€” A lightweight spec‑driven frameworkOpenSpec is a lightweight, spec‑driven framework for coding agents and CLIs β€” universal, open source, and no API keys or MCP required.
View full thread

Fix #3: reranking via Cohere on Bedrock. KNN gets the ballpark right but the ordering wrong. A reranker reads query + candidate together in one pass -- cross-attention, not independent vectors. Run it on a shortlist of 20-50, not the whole dataset.

The UI shows both layers -- KNN results with cosine scores, reranked results with Cohere scores. Comparing the two orderings is where you actually learn what vector similarity gets wrong. #dbt #DataEngineering #RAG #FalkorDB #KnowledgeGraph

View full thread

Fix #1: embeddings stored directly on each FalkorDB node. No separate vector store. No parallel infrastructure to maintain. Upload your manifest and catalog -- every node is searchable by meaning before you open the chat.

Fix #2: full-text search via RediSearch. Semantic search has a blind spot -- when you know exactly what you want, approximation hurts. Boolean operators, prefix matching, fuzzy typo tolerance. The agent picks the right tool for the right question.

The original pipeline stored dbt nodes in FalkorDB -- models, sources, seeds, edges for dependencies and tests. Great for traversal. Useless if you asked "find models related to student engagement" and none of the names matched.

View full thread

And the chain: Anthropic's largest investor (Amazon, $13B in) bypassed the company to brief Treasury directly, and within 24 hours the product was offline under threat of criminal penalties. Same pattern as Huawei, TikTok, the 1990s encryption wars.

View full thread

The process story is worse than the technical one. 90 minutes to comply. No formal finding. Export control law written for physical goods, applied to a public AI model for the first time.

On June 9 Anthropic released their most capable public model ever. On June 12 the US government ordered it offline. AWS, Google Cloud, Microsoft Foundry, all Claude APIs, dark simultaneously. No warning. First time a government has pulled a deployed frontier AI model from the market.

define once they said. you now have 7 semantic layers, final_final_v2, and a prayer. a semantic layer without multiple query interfaces is just well-documented SQL. is owning the full stack the only way to actually solve the semantic layer problem?

I just received some photos from my talk with Alex Selkirk at SQLBits 2026, where we had two sessions about solutions we've built at Ponder Labs -- so I figured it's a great opportunity to recap what we covered, share the blog posts again, and show off my beautiful photo 😊

View full thread

I just received some photos from my talk with Alex Selkirk at SQLBits 2026, where we had two sessions about solutions we've built at Ponder Labs -- so I figured it's a great opportunity to recap what we covered, share the blog posts again, and show off my beautiful photo 😊