Post

😱 You see this distribution of response times from running 500 requests/sec against some microservice for 120 sec. What do you do to in order to investigate what's going on?

You measure for longer to see if the pattern repeats itself to see if there's any hint on when this happens. If possible, try to color it based on the container where it is running in?

What kind of program is it? Does it make requests? It kind of feels like there is an exponential backoff that doesn't trigger the first try.

It runs queries against a database. I suspected the connection pool, but the same behavior is exposed when just returning some dummy data from the service itself. Starting to wonder whether it's an artifact of the load generator (oha) and/or running this on macOS. Will try on a Linux box.

Try Hyperfoil 😉 Seriously, we've tested it against other load drivers and it's reporting is more accurate. New blog post coming in the not too distant future 🙂