Post

A very cool FLIP (Flink Improvement Proposal) has recently been accepted: Running Flink jobs in MiniCluster using the Kubernetes Operator. cwiki.apache.org/confluence/s...

cwiki.apache.orgFLIP-611: Running Flink jobs in MiniCluster using the Kubernetes Operator - Apache Flink - Apache Software FoundationDUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.

Yaroslav Tkachenko

@sap1ens.com

Historically, every Flink pipeline required: - At least two pods (JobManager and TaskManager). - Each pod needs at least 0.5 vCPU and 1GB RAM (it's possible but really tricky to go lower).

Yaroslav Tkachenko

@sap1ens.com

So that's 1 vCPU and 2GB RAM for any pipeline to start, even if it processes one message a minute. Imagine running that at scale (I don't need to imagine...)? At the same time, other tools (e.g., Rust- or Go-based) can have a much lower footprint.

7/10/26, 4:04 pm0 reposts0 likes

Yaroslav Tkachenko

@sap1ens.com

The FLIP addresses the problem by running both components (JobManager and TaskManager) in the same JVM process on a single pod. It's a big step in making Flink simpler and more efficient to operate.

Yaroslav Tkachenko

@sap1ens.com

I especially like that you can easily migrate from this setup to a "classic" deployment by simply using a savepoint. Start small, then scale out.