Nikhil Benesch
Nikhil Benesch
@benesch.bsky.social
Systems engineer @turbopuffer.bsky.social. Former CTO @materialize.com.
1.3K followers606 following218 posts
tpuf Python client went async today! π‘ β€οΈ π Async client perf slightly edges out sync client perf under heavy query load. github.com/turbopuffer/...
Nikhil Benesch reposted
Couldn't be more excited to be joining the team at @turbopuffer.bsky.social. π‘π¨
welcoming two excellent additions to team tpuf this week: @arash11gt as a customer engineer, and @nikhilbenesch to the DB team from pasts at Materialize & Cockroach. we're locking in a p999 eng team to build the best search engine for scale.
Well well well: www.crunchydata.com/blog/pg_incr... Incremental pipelines come to Postgres via Crunchy Data! This is like "dbt incremental", not true incremental view maintenance like @materialize.com or Snowflake's dynamic tables, but it's a neat step towards IVM.
Maybe EDB or Crunchy could pull it off, a la Crunchy Data Warehouse. But itβs not clear the business case is sound. IVM means more efficient read workloads, which means customers spend *less* money on Postgres.
Nikhil Benesch reposted
The 6th (and last of 2024!) NYC Systems talks are next Thursday! We've got @jaronoff.com of Omlet and @benesch.bsky.social of Materialize speaking. :) nycsystems.xyz/december-202...
Nikhil Benesch reposted
π Postgres Compatibility Index (PCI): Think your shiny new Postgres derivative is "Postgres-compatible"? Test it, score it, and know the truth. π Thanks @gunnarmorling.dev @tudor.golubenco.org @benesch.bsky.social @deverts.bsky.social for the inspiration. Blog ==> tinyurl.com/mmfcbczz
Something like this? Postgres compatibility index "alpha version" github.com/secp256k1-sh...
Nikhil Benesch reposted
First new post in a couple of weeks! There's been a lot of activity around regattastorage.com this week, so I decided to write about the space. tl;dr It's pretty exciting!
Nikhil Benesch reposted
Today's third blog post on Aurora DSQL: this time looking at the write and commit path, and how conflicts are detected and handled: brooker.co.za/blog/2024/12...
Nikhil Benesch reposted
Nikhil Benesch reposted
My second blog post on Aurora DSQL, with a deep dive on how reads work: brooker.co.za/blog/2024/12...
S3 (Iceberg) Tables is everything I dreamt of, and more. I blogged some long-form thoughts: meltware.com/2024/12/04/s... I think we're about to see an explosion of data tools (@materialize.com, @clickhouse.com, @duckdb.org, et al.) learn to write Iceberg tables via S3 table buckets. #databs
Nikhil Benesch reposted
GCP is now the only major cloud provider without a serverless Postgres offering. As of today AWS has DSQL: aws.amazon.com/about-aws/wh... Azure partnered with Neon in August: neon.tech/blog/neon-is...
Actually looking into this more, it looks like DSQL is a ground up reimplementation of a PostgreSQL-compatible database. At least, that's what I infer from from @marcbrooker.bsky.social's blog post: brooker.co.za/blog/2024/12...
S3 gets *native* Iceberg support. π€― From the AWS blog: aws.amazon.com/blogs/aws/ne... We knew something like this was coming (bsky.app/profile/bene...), but it's an even deeper integration than I expected.
Something interesting is brewing in Iceberg-on-S3 land. π lists.apache.org/thread/v7x65... cc @eatonphil.bsky.social
Nikhil Benesch reposted
IT'S HERE! Amazon Aurora Distributed SQL (DSQL). It's DynamoDB and Postgres merged into a truly serverless relational database. This is THE announcement of Re:Invent 2024. Everything else is irrelevant.
@glaubercosta.bsky.social nails an explanation of 1) why vector databases are mostly a feature not a product but 2) why nonetheless @turbopuffer.bsky.social is on to something. From a Software Huddle interview a few months back: www.youtube.com/watch?v=a4X7...
@fasterthanli.me what's the performance of loona like? π I'm in the market for a faster S3 client in Rust and loona is the first plausible project I surfaced. bsky.app/profile/bene...
Does it use io_uring? All I want for Christmas is an S3 client that uses io_uring.
Awesome work from @antiguru.bsky.social that can cut @materialize.com memory use by 50% on a common class of workload. Most importantly for our customers: Materialize is now 50% cheaper on these workloads!
Sometimes it's easier to mitigate a problem than solve it: a feature I recently implemented reduces the memory and CPU @materialize.com requires to maintain future updates. Some details why solving the problem is hard and the mitigation easy-ish. materialize.com/changelog/20...