Apurva Mehta

The inner join between sets of people who build databases, write, and draw? Low cardinality. I'm in that set, so I'm starting a blog! Here's my first post: www.bitsxpages.com/p/frameworks...

Today marks SlateDB’s one year anniversary! It’s been a lot of fun. Thanks to @rohanpd.bsky.social @flaneur2024.bsky.social @almog.ai @vigneshc.bsky.social @paulbutler.org Jason Gustafson, David Moravek, and many others for joining the project. 😀

I’m a sucker for a good cost calculator. This one is pretty slick. Lots of components contribute to cost in Kafka Streams.

SlateDB 0.5.0 is out! Features: - Checkpoints - Clones - Read only client - Split/merge database foundation - TTL filtering on reads - Last version with breaking byte format changes By the numbers: - 62 commits - 2 new contributors - 10 total contributors github.com/slatedb/slat...

Related: super interesting discussion on stateful stream processing vs OLTP with @apurvamehta.com from @responsive.dev open.spotify.com/episode/1hsH... (start at 18:46)

I'm always excited to chat with @apurvamehta.com about what @responsive.dev is building. Streaming and real time as terms are being constantly reinvented as the market needs change rapidly, and Apurva is one of the best to talk about that. Check the conversation here: techontherocks.show/15

I’m excited to listen to this one. Some of my favorite folks all in one podcast 😀

This week, our 101 series has @ableegoldman.bsky.social diving deep into the world of performance optimization for Kafka Streams. This isn't just theory: she shows how we realized 10x throughput improvements by implementing these techniques in prod! www.responsive.dev/blog/perform...

New episode: Reinventing Stream Processing with @apurvamehta.com from @responsive.dev We cover: Real-time vs low latency in streaming Why stateful streaming is hard at scale RocksDB, Postgres limitations SQL vs flexible APIs Decoupling state & compute with SlateDB. 🎧 👉

Next entry in the Kafka Streams 101 series: application lifecycle. Very beginner friendly, but even the pros might find a useful tip or two -- for example: have you heard of the new standby task listener? Check it out: www.responsive.dev/blog/app-lif...

We're keeping the ball rolling on the Kafka Streams 101 series with yet another blog post! This time the focus is on configuration, and breaking down the most important configs to focus on for various needs (such as correctness, resiliency, etc) Check it out here: www.responsive.dev/blog/importa...

We are happy to announce that we have completed another SOC2 Type 2 audit along with completing another successful penetration test against our cloud services. You can find the latest reports on our trust center: trust.responsive.dev

Had two people ask me for guidance about open source foundations this week. Lots of trepidation around Apache, not much familiarity with CNCF, and curiousity about why I chose Commonhaus for SlateDB. Here are my thoughts:

#StreamProcessing with @kafkastreams.bsky.social

Should you use Kafka? If so, when? And what are the tradeoffs presented by the dizzying variety of Kafka-adjacent technologies? I hope that my latest blog post provides unique and useful answers to these questions. Let me know what you all think! 👉 www.responsive.dev/blog/why-whe...

Thanks everyone for all the feedback on blog post ideas! Performance and upgrades were most popular so stay tuned for these deep dives - for beginners and advanced users alike Check out our new Kafka Streams 101 site and subscribe to get notified for these and other upcoming lessons! bit.ly/40LWJZL

I'm enjoying putting the finishing touches on my next blog post on the Why, When, and How of Kafka applications. It's a distillation of my point of view from a decade in the space and I hope it's as useful to read as it was fun to write. Coming Tuesday!

I've used ChatGPT and/or Gemini to brainstorm ideas for presenting concepts visually multiple times recently. They have unlocked ideas I wouldn't have thought of otherwise. And sometimes, they generate the graphics. For the rest, Figma is perfect. It's such a major boost either way!

How did Responsive simplify its architecture and achieve monstrously high availability with unlimited scale potential? Co-founder @almog.ai will share how replacing RocksDB with #ScyllaDB in #Kafka Streams led to big wins for their team at Monster Scale Summit. www.scylladb.com/monster-scal...

The first of our Kafka Streams 101 Series. Practical knowledge on thinking about windowing and time in @kafkastreams.bsky.social We are going to ship new tutorials regularly over the next few months, and you can find them all here: www.responsive.dev/resources/ka...

There are some learnings that only come from having seen a wildly successful technology go from a nascent project to ubiquitous. These insights can change the trajectory for new tech. Jason's summary about error handling is a great read for practical engineers: github.com/slatedb/slat...

Sophie arguably has the best @kafkastreams.bsky.social blog posts on the internet. Now you get a say on what she should write about next!

👋

Kafka Streams had an amazing 2024! Check out @ableegoldman.bsky.social's year in review, where she recaps the 25 major improvements the community made to Kafka Streams! www.responsive.dev/blog/kafka-s...

SlateDB now has range scans! An important feature for joins, time series, secondary indexes, and so on. Props to the folks @responsive.dev for getting this done. github.com/slatedb/slat...

It was so much fun to observe the @responsive.dev team brainstorm distributed state snapshots for Kafka Streams using SlateDB/RS3. Nothing beats the thrill of having a team of the best engineers solving hard problems with nothing but a whiteboard in front of them!

Great to see so many people online. If you want to hear about how Metronome scaled their Kafka streams apps to 10's of TBs of state and 10's of thousands of events/s, do tune in for the last few minutes!

The momentum on SlateDB development can't be stopped! We're getting close to closing out another foundational RFC by @flaneur2024.bsky.social: transactions 🏦 Mostly taking inspiration from Badger, but there's some innovative stuff there. Check it out: github.com/slatedb/slat...

For any Kafka Streams fans out there who want to know what it's like scaling an EOS application up to multiple TB of state at ~10k+ records/s with minimal downtime -- make sure to catch the Q&A we're hosting tomorrow with Metronome. Join us Dec 12th at 9:30am PST! www.linkedin.com/events/72647...

The best thing about doing a startup is that you have the absolute freedom to invest in creative ideas with absolute accountability. The bad thing is that 99% of your ideas will fail. The most important asset in my experience is being able to use failure as a fuel for more creativity.

First new post in a couple of weeks! There's been a lot of activity around regattastorage.com this week, so I decided to write about the space. tl;dr It's pretty exciting!

It's remarkable how popular Kafka Streams is. Here's data for people searching for solutions to their Kafka Streams problems and landing on @responsive.dev. That's an up-and-to-the-right chart I love! On that note, what's the best book or blog you've read for optimizing website conversions?

"SQL is a great language for expressing ETL and analytics logic, but it has never been a language developers pick for expressing application logic. By contrast, Responsive was founded on the belief that the true value of stream processing is in the new applications you can build with event streams."

KIP-1104 is completed. It closes such an obvious gap in the FK-join API that I am wondering why it was not done this way in the first place... (and why did it not come up much earlier?) Well, we will have it with upcoming 4.0 release. Better now than never. cwiki.apache.org/confluence/d...

Is stream processing interesting because of the new apps it enables, or because it promises better data processing? I'm in the first camp and am proud of the major contributions Responsive has made to the application space. My reflections on what's next: www.responsive.dev/blog/respons...

I've always wanted to have a landing page for apurvamehta.com. But a blog was too much of a commitment and a short bio was too simple. @bsky.app allows you to display your feed on a static webpage, which is a perfect setup and is perfectly aligned with the 'your domain is your id' philosophy too!