David Jayatillake

How big should your data team be? Data teams are often oversized. A company of 200 people rarely needs 15+ data staff, usually 5% of org size is enough dataactionmentor.com/knowledge-ba...

The amount you love someone is proportional to how often you Ghiblify their pictures.

This week I look at agents. I think this is a new way to build where we don’t intentionally build code-based software. open.substack.com/pub/davidsj/...

BERT and ERNIE! 😂 tracking.tldrnewsletter.com/CL0/https:%2...

I don't usually share photos of my family on social media for good reason, but I'm happy to share these ones!

This post encapsulates how I feel about the current state of LLMs and doomers etc. Really great read: fly.io/blog/youre-a...

So when I've attended Snowflake summit before, I've usually written a blog post talking about the new features released, etc. Is someone going to do that this year, given I didn't go? 😊 #datasky #databs

It is possible to build machine learning systems which punch up instead of punching down.

Got a cool story about something in the data engineering space? You should 💯 submit it as a talk to Current 2025 in New Orleans 😁 Do it! Now! CfP is open until 15th June. sessionize.com/current-2025... (Pro-tip: you only need an abstract at this point; writing the talk can be later 😅) #dataBS

At the London Data Practitioners Meetup with @pedramnavid.com @jayatillake.bsky.social @rittmananalytics.bsky.social and the London Dagster community

Doctor’s orders 🫡

I still think this is the biggest prize in AI. If Siri could actually do most things you do on a phone manually... 9to5mac.com/2025/04/22/s...

Has anyone tried Llama 4 Maverick yet? How big a machine does it need to run locally? @simonwillison.net

Looks like Nintendo became the best at console FPS.

Clickbench says @duckdb.org is a great analytics database www.mooncake.dev/blog/clickbe...

A couple of weeks ago I wrote a post about @pola.rs Cloud, and as coincidence would have it, DeepSeek then releases smallpond - a way to use @duckdb.org scaled out. It's pretty cool that an org has managed to build this, but is it the right way? davidsj.substack.com/p/many-ducks...

🦆🦆🦆🦆 everywhere these days count.co/blog/announc...

New episode: “Semantic Layers: The Missing Link Between AI and Data” with @jayatillake.bsky.social . We discuss how semantic layers bridge raw data and AI, achieving 100% accuracy for natural language queries, and what’s next for LLM-powered data pipelines. 🎧 https://techontherocks.show/14 🎧

@tojyouso.bsky.social is this you? 😂

pola.rs/posts/polars... This so reminds me of @motherduck.com and @duckdb.org, I guess the open source Polars is as good as it can be on one machine, and Polars Cloud is as good as it can be on many.

Cancer incidence rises when people live longer and have access to preventive care to diagnose it in time to treat it, and effective treatment increases cancer prevalence because we count the people who are still alive – this is a good thing, and I'm so fucking tired.

Is there anything specific you want to know about OLAP cubes, especially newer open-source systems (e.g., ClickHouse, Druid, Starrocks, Pinot, Kylin, Doris, Cube, and more)? Which one is your favorite and why?

DuckDB out here making me avoid other SQL dialects

DuckDB 1.2 brings serious performance to real-world queries! Selective joins, IN lists, Top N, partitioned parquet reads, and more get much faster! Performance is also incrementally improved ~across the board. Plus, the CSV reader improves (latin-1 for Windows folks!), SQL is friendlier, and more!

@anthropic.com's Dario Amodei wrote a really thoughtful piece on the @deepseek.bsky.social situation and how it relates to overall macro end games with AI: darioamodei.com/on-deepseek-...

It's weird how people see these two as mutually exclusive.

They need to substantiate this. Otherwise, it sounds like sour 🍇. www.macrumors.com/2025/01/29/o...

OpenAI right now

I didn't realise there was another way... I either discover code or copy-paste and edit an existing solution... plus now AI. jimmyhmiller.github.io/discovery-co...

If you're considering SQLMesh, @jayatillake.bsky.social has a great blog series from the end of last year that does in depth comparisons with DBT