Daniël de Kok on Mythryl: “TGI v3.3.1 is released! This version switches to Torch 2.7 and CUDA…”

TGI v3.3.1 is released! This version switches to Torch 2.7 and CUDA 12.8. This should improve support for GPUs with compute capabilities 10.0 (B200) and 12.0 (RTX50x0 and NVIDIA RTX PRO Blackwell GPUs). github.com/huggingface/...

github.comRelease v3.3.1 · huggingface/text-generation-inferenceThis release updates TGI to Torch 2.7 and CUDA 12.8. What's Changed change HPU warmup logic: seq length should be with exponential growth by @kaixuanliu in #3217 adjust the round_up_seq logic to a...

22/5/25, 1:40 pm0 reposts0 likes

Post