Post

I finally made a page on my Dish activation function, replacing my deleted Tweet: danieldk.eu/Dish-Activat... It's a non-monotonic function similar to GELU/SiLU, but does not require elementary functions, making it faster on various hardware. I'll leave the empirical evaluation to someone else 😁.

13/12/25, 11:09 am0 reposts1 like

Daniël de Kok

@danieldk.eu

13 Dec 2025

It started out as a joke with @kadarakos.bsky.social in 2022 when we worked at @explosion.ai that we should make an activation function using the fast inverse sqrt of Kahan/Walsh and famously used in Quake 3.

Daniël de Kok

@danieldk.eu

13 Dec 2025

We are currently doing a reading group on RISC-V and its vector extension. I actually got to implement it using the fast inverse square root because the T-Head board that we use does not have the vfrsqrt7.v instruction. So, full-circle I guess. github.com/danieldk/low...

github.comgithub.com