Post

What's been sitting in my Obsidian notes for over a year now, I finally took the time to put it up on my blog. I have a few more blog posts coming up as part of this series. #kubernetes #ipv6 #cilium netops2devops.net/series/kuber...

part1 covers underlying #ipv6 infrastructure requirements before installing #kubernetes netops2devops.net/series/k8s-c...

part2 covers building the simplest #IPv6 only #k3s cluster using @cilium.io netops2devops.net/series/k8s-c...

topics for subsequent posts: - IPv6 address planning to run #cilium in native routing with cluster pool vs. multi-pool ipam. - Cilium BGP control plane with LB IPAM and GatewayAPI for ingress - Predictable egress IP without using an egress gateway (aka no SNAT/masquerading) Stay tuned ;)

Also cover BGP ECMP/Anycast of pods + seperate GUA addressing for unique unicast per node to eliminate the need for GENEVE or other host-to-host tunnelling. Would love to see a single blog post or blog post that covers this end-to-end.

I’m planning to cover just that in part4. Still working on part 3 this week😅 but yes I do have a working architecture/config with #cilium that uses native routing i.e no tunneling or masquerading. Each namespace gets a clean /64 using multi-pool ipam. POD addresses announced via BGP.

That's just just unicast, right? Or Anycast for global ingress + seperate unicast for pod-to-pod comms or pod-to-WAN comms.

pod to pod and pod to wan is Unicast. Ingress has to be a virtual IP. It’s basically a load balancer for your Pods. Albeit, since Cilium uses Envoy proxy as L7 proxy underneath it preserves the original source IP. So when user traffic reaches pod it sees real source IP

What if my application is SIP server using native SIP (no UDP) or native SCTP? Ideally I'd like to avoid reverse proxy of any kind.

Envoy uses a feature within the Linux Kernel called TPROXY (transparent proxy). So technically your application sees the original client IP and not the reverse proxy IP, if that helps. Now granted I have never worked with SIP so I could be mistaken.

Quick research suggests TPROXY is ossified and limited only to TCP/UDP. This leaves out all other standardised layer 3/4 protocols (including native IPSec in IPv6 without UDP encap hack).

I am not K8s pro, but in Docker this is easily achieved: Use different subnets for Anycast/unicast, container is exposed to WAN on both subnets natively, no reverse proxy. As long as container app talks SIP/Other protocols, it should work. BGP handles routing with the leave switches.

In K8s I believe a custom CNI that permits seperate subnets per-Pod should in theory do the trick.