AJ Stuyvenberg

I use AWS a ton but Lambda still astounds me. Throw some code in a function, send 1m requests as fast as you can. It ate up all available file descriptors on my little t3 box and still ran 18k RPS with a p99 of 0.3479s. Not many services can go from 0 to 18k RPS instantaneously with this p99.

Lambda now charges for init time, so it's useful to count sandboxes which are proactively initialized but never receive a request. Here's what happens after a 10k request burst. Hundreds of sandbox shutdowns, along with 22 sandboxes which were spun up but never received a request.

NEW: Lambda can now send up to 200mb payloads using response streaming! I assume this is mostly directed at LLM inference workloads, where chatbots can stream large amounts of data over the wire as it becomes available.

I've long been an advocate for the Lambda Web Adapter project which lets anyone pretty easily ship an app to Lambda without learning about the event model/API. Honestly AWS should simply support this natively.

That said, I'm excited to share that @Datadog's Serverless monitoring product now supports LWA! Thanks to Harold and AWS Labs for collaborating with us on the PRs, and huge thanks to Alex Gallotta for driving this work.

"We run benchmarks continually across all of our competitors, not just queries - even connections, ensuring we don't add any latency at all." @isamlambert Performance is such a competitive advantage which easily slips away if you're not constantly paying attention to it.

Here's another 33% cold start reduction, which comes from deferring expensive decryption calls made to AWS Secrets Manager until the secret is actually needed. Lazy loading is great!

Here's how to visualize a 100% memory allocation improvement! A recent stress test revealed that malloc calls bottlenecked when sending > 100k spans through the API and aggregator pipelines in Lambda.

Here's how to visualize a 100% memory allocation improvement! A recent stress test revealed that malloc calls bottlenecked when sending > 100k spans through the API and aggregator pipelines in Lambda.

NEW: A recent blog post went viral in the AWS ecosystem, about how there's a silent crash in AWS Lambda's NodeJS runtime. Today I'll step you through the actual Lambda runtime code which causes this confusing issue, and walk you through how to safely perform async work in Lambda:

NEW: A recent blog post went viral in the AWS ecosystem, about how there's a silent crash in AWS Lambda's NodeJS runtime. Today I'll step you through the actual Lambda runtime code which causes this confusing issue, and walk you through how to safely perform async work in Lambda:

Lambda's fleet management shutdown algorithm is learning faster! I'm calling this function every 8 or so minutes. At first the gap from invocation to shutdown is about 5-6 minutes, which was the fastest I've observed during previous experiments.

Now after only ~4 invocation/shutdown cycles, Lambda shuts down the sandbox 1 minute after my request:

Real-time audio processing is an extremely interesting corner of performance engineering. You've gotta process data faster than it streams in, otherwise the work is wasted. The engineering team behind CoScreen just open sourced a new library, dtln-rs, suitable for removing noise from all kinds

Here's another huge p99 cliff! One unreasonably effective way to lower the average latency of a service is to minimize the causes of p99 events. Here, we've managed to absolutely crush the Max Post Runtime Duration from ~80ms to 500µs!

So the strategy looks something like this:

Here's another huge p99 cliff! One unreasonably effective way to lower the average latency of a service is to minimize the causes of p99 events. Here, we've managed to absolutely crush the Max Post Runtime Duration from ~80ms to 500µs!

How? In short, I cheated. If the invocations are frequent enough (more frequent than the TCP Keep Alive/server timeout), you can drive one flush request asynchronously across multiple invocations and it costs you $0 in billed time. That's not the hard part though.

Here's another huge p99 cliff! One unreasonably effective way to lower the average latency of a service is to minimize the causes of p99 events. Here, we've managed to absolutely crush the Max Post Runtime Duration from ~80ms to 500µs!

PRICE CUT: Lambda slashes the price of cloudwatch logs for high volume. From $.50/gb down to $0.05/gb after 50TB. You've gotta be spending a decent chunk of $$$ on cloudwatch logs for this to help, but still – a price cut is a price cut!

NEW: AWS Lambda will now begin billing for the INIT phase for all runtimes (not just custom/container runtimes). Your cold starts cost money now! My hottest take is that AWS should have done this years ago.

Free init time incentivizes abuse such as: 1. crypto mining or scrape websites in 10s of init time with 10gb functions, making it slower for all of us 2. Lambda team wasting time tracking all this down 3. Programming in Java ;)