Post

Making us all look bad by having a "thanks to" for other people who helped him, AS WE SHOULD HAVE DONE

------- "1000x: The Power of an Interface for Performance", Joran Dirk Greef, #sd25 "A million dollars, 16 machines, a race" between good implementations and a good interface. Interfaces expose, and impose, and play a profound influence on how well systems scale.

"We're gonna do a challenge. We're gonna take one of the most valuable workloads in the world today and show that popular database have a performance limit, and this comes from the interface, not implementation", meaning adding more hardware won't improve the limit. Then we'll fix the limit.

We're being told to get our cameras ready.

Standard interface is SQL ∪ over network ∪ trasnsaction processing. "No OLGP or OLAP", specifically ultra-high volume transactions, OLTP in the most restricted sense. Inventor of OLTP Jim Grey proposed in 1985 three benchmarks: scan, sort, and DebitCredit. kicked off optimization race. #sd25

DebitCredit is "rewrite account, write history, rewrite teller, rewrite *branch*." Branch is a "hot account", adding in contention. It's possible, with hot accounts like a company account, for upwards of 90% of transactions to have contention. Can't handle big scale without handling contention.

Example: 14mm people tried to get tickets for Oasis, and they sold out in minutes. Pareto principle: 80% of volume involves 20% of power users. Transactions are not complex queries. Rather, simpler queries but lots of them. In some fields, up 10000x volume since 2015.

We're going to benchmark four databases. "Before we use interfaces to unlock performance, we will use them to LOCK performance". First db, Postgres, OLGP (online general purpose). Have to build out transfers as a general relational table. #sd25

To transfer money from alice to bob, psql first gets their balanes, then adds to alice's debits and bob's balance, then writes an (alice, bob, amnt) record to the transfers table. Will take somewhere between 1 and 10 roundtrips. "How many transactions/sec?" I guessed 2

For low contention, 10k transactions/sec. For high contention, gradually drops to 4 transactions/sec (I was closest!) Postgres super sensitive to network costs: 10ms roundtrips more than 10x worse than 1ms roundtrips. Needs to do row locks across 2 roundtrips, which creates bottlenecks.

Amdahl's law is a killer. That, plus network times, give hard limits to how fast a truly general db can process transactions under high contention. Many engineers give up, believing we either need "correct but not fast" or "fast but not correct". "No. Give me serializable or give me death!" #sd25

"Those who would trade safety for performance deserve neither safety nor performance." What if we use a CLOUD database? They give UNLIMITED* INFINITE* SCALE* These often have the same interface as postgres, but as a proprietary cluster. "Looks like / talks like Postgres", runs on many machines

They'll benchmark the "cluster" on 16 machines that cost 84k/month total, or 1m/year. Instead of 12k transactions/sec under 0% contention, got 7.5k transactions/sec, a little better than half (for a million dollars a year). 10% contention: 570 t/s. 8 machines: 600 t/s. 1 machine: 800 t/s!

Write contention is what separates OLTP from OLGP. At 10% contention, the proprietary cluster never goes above 12% cpu utilization. What about stored procedures? Keep processing in a single in-process transaction without row locks! Benchmark #3: DuckDB. Kinda ill-suited, as DuckDB is OLAP #sd25

So benchmark is adapted to look like an analytics workload, just for fun. DuckDB hits 2k transactions/sec for 0% contention. 10% contention: 2k t/s. 90% contention: 2k t/s. 90% with 10ms roundtrip: 2k t/s. No performance degradation because no row locks.

tbf it's not fair, because it was provided as one giant batch. Also since DuckDB is OLAP it doesn't have any durability or high-availability. Just shows that if we can avoid locks, we can process many more transactions. Benchmark #4: Tigerbeetle (duh). True OLTP, nothing BUT transactions.

Tigerbeetle learns from history. While SQL is the language of databases for 5 decades, DebitCredit has been language of transactions for 5 centuries. Tailor interface for that. We can fit 8000+ transactions in 1mb, getting a lot more work done per network round-trip. #sd25

Again, this ONLY does OLTP. No general-purpose processing, no custom data, no user passwords or workflow state machines, just accounts and debits/credits, nothing else. Specialization gives special interfaces gives power. Benchmark time! Every db [except cluster?] runs on i8g.16xlarge...

client runs on separate machine. Proprietary cluster users anonymized. Missed a bit of theother benchmark details. "The time has come to race." Gonna replay the real performance traces for us. Showing that now for psql, plus analytic database

THE SYNTHWAVE HAS STARTED

I mean they had the potential to change color, they just didn't until the closing keynote. To the beat of the synthwave. That is playing. What a conference, man Aaaaaanyway that was a benchmark visualization. Postgres avged 1k tps, tigerbeetle avged 300k tps for same latency levels

Tigerbeetle also had the ability to do full replication and durability checks, since it was fast enough to afford to spend cycles on that. That was at 10% contention. Now benchmarking 50% contention with the proprietary postgres million-dollar cluster. Gets 2 minute headstart vs tigerbeetle

These guys employ a lot of artists #sd25