Torsten Hoefler 🇨🇭

​It was an honor to have Her Highness Sheikha Shamma bint Sultan Al Nahyan attend my talk at the ADIA Lab Symposium. ​AI will bring about one of the most transformative eras in human history with extraordinary acceleration of scientific discovery. Day 2 starts now—join us online or in person!

Horst Simon opens the ADIA Lab symposium 2026 in Abu Dhabi today! "I hope each of you will leave with at least one new idea and one new connection." AI for Science will be the center of attention with Nobel Laureate speakers in the next three days. Join us online or in person.

Congrats to students Golnoosh and Patrick on winning the Emerging Research Award at FTC 2026in Berlin! 🏆 The surprise: adding uncertainty estimates through quantile regression actually improved accuracy in scalable federated learning up to 1000 clients 🚀.

Ultra Ethernet 1.1 finally released last week! Was great fun to catch up with key folks of the crew at OCP in San Jose. Supports Scale-Up Networking, Programmable Congestion Control, In-Network Computing, and In-network Congestion Singaling. Check out the spec at buff.ly/odjAfEQ

How much network bandwidth do you need? The answer is: more! Maia 200 tops the record with 1.4 TB/s (full duplex) Ethernet bandwidth per chip. More than 1.5× over NVLink 5 (Blackwell). It turns out that "bidirectional" was doing quite a bit of work in NLink's 1.8 TB/s number #HPC #AI #Accelerator

When AI, HPC, and Quantum meet at SC Asia 2027 in SIngapore. 📢 Due to popular demand, the paper submission deadline has been extended. We invite the community's very best papers. 🔗 buff.ly/oRS1TE1 #SCA27 #Supercomputing #AI #HPC #QuantumComputing #Singapore

"Everything verifiable will be automated." Mathematics is far more than proving theorems. For young researchers, the opportunity is clear: build the automation, write the specifications, and develop the verification tools. My HLF spark talk: buff.ly/mCW9JI2

Industry reacted quickly. From lossy packet delivery to reliable multipath spraying: Multipath Reliable Connection (MRC) turns many "Issues at Hyperscale" into engineering solutions. buff.ly/NH2FvrQ buff.ly/pueSosc buff.ly/QpvkyJ0 On the path to Ultra Ethernet. #AI #HPC #Networking

Want to help shape the future of AI infrastructure? We're hiring at Microsoft Azure. Our work spans AI systems and networking powering the next generation of intelligent systems. Looking for exceptional engineers who enjoy tackling hard problems with real-world impact. Apply here: buff.ly/Myvgeay

We keep talking about flop/s. Maybe we should talk about flop/s per square millimeter. Maia 200 achieves ~12 Tflop/s FP4 (6 FP8) in just 1 mm², roughly the size of a grain of salt. When data movement dominates energy consumption, compute density becomes a first-class design objective. #AI #HPC

Digital for Good: What if climate predictions could be generated 1,000× more efficiently and with 1,000× less CO₂? In this interview, Fanny Lehmann and the ESFM team explain how AI foundation models are transforming weather forecasting. buff.ly/Il8H4Td buff.ly/yRF5Txv #ClimateAI #AIforScience

Send your best papers to SC Asia before the end-of-month deadline! 🚀 SC Asia is among the flagship international forums for breakthroughs in supercomputing, AI infrastructure, system architecture, networking, quantum, cloud and scientific computing. buff.ly/3C7EUpz #SCAsia #Supercomputing #HPC

Most filtered vector search benchmarks rely on outdated datasets. SPCL student Patrick presents arxiv-for-fanns: a new benchmark with 2.7M modern embeddings concludes: There is no universally best filtered ANN algorithm. 📖 buff.ly/hHvx0bl 📽️ buff.ly/r1ERkxb #VectorSearch #RAG #ANN #Databases

Nothing beats chatting and traveling with president Ramos-Horta, talking to the next generation in Timor-Leste, and discussing deep topics about #AI for education and development with Turing Awardees and Nobel Laureates on tropical islands. buff.ly/n0V8ikW

We presented how Maia 200's Software-Define Dataflow delivers highest inference performance for production LLMs in Microsoft's fleet at #HotChips26 today. We also released the full paper with many details on arXiv: buff.ly/KJ8yMOp And a blog post: buff.ly/TsrPppe

Check out SPCL's Tommaso explaining REPS for efficient packet spraying in Ethernet! Developed in collaboration with Microsoft and now part of Ultra Ethernet, REPS delivers adaptive, load balancing and failure mitigation. Already powering large-scale AI clusters! 📺 buff.ly/sBLvG87

This Thursday at Hot Interconnects #HOTI26: "Demystifying NVSHMEM". Learn how the old #HPC PGAS and symmetric heap concepts nicely accelerate GPU communications 🚀 Reminiscient of our earlier dCUDA paper that turns 10 years now! Great to see concepts productized 🤗 buff.ly/K4aRYqz buff.ly/Z9mHPso

Samantika Sury from HPE talks about Macro-Heterogeneity for #AI for #Science at #ModSim26. “A system with diverse hardware should logically look like a unit to applications”. Totally agreed why do we always expose the guts in #HPC 😊? Glad to collaborate through SPCL!

"Messages matter packets don't" - learn how our work on Multipath Reliable Connection (MRC) at #HOTI26 shifts to an application-centric view. Join us online next Wednesday 1pm PDT with Rip Sohan (AMD) and Eric Spada (BRCM)! A bigstep towards Ultra Ethernet! buff.ly/K4aRYqz buff.ly/NIZXU9A

At #ModSim26, Jeff Vetter highlighted DOE's ambitious goal: 150-250 fault-tolerant logical qubits by 2028 at ≤10⁻⁸ logical error rate. The science is remarkable, but the real story is that #Quantum computing is becoming an engineering discipline - excited to share some new architecture ideas soon.

Hal Finkel highlights at #ModSim26 how the U.S. Department of Energy will converge #HPC, simulation, and #AI pushing 20% of its budget into compute infrastructure. Looking forward to my panel on how Agentic AI will reshape science, engineering, and discovery. #AI #AgenticAI #Accelerate #Science

It seems like “linear attention” now means almost anything—as long as there’s an O(L) somewhere in the paper 🙃. Linear compute? Memory capacity? Memory traffic? Those are three different bills, and our accelerators pay all of them 💰️. My plea: show me all three: buff.ly/Yd35IUv

Honored that our paper won the 2025 IEEE TPDS Best Paper Award! buff.ly/4AiBN0S The main idea is to use differential analysis on dependence graphs to causally determine root causes of performance issues. Credits to Yuyang for the hard work, Jidong for leadership, and to all of our co-authors!

Happy Birthday, Jack J. Dongarra! Dongarra received the #ACMTuringAward in 2021 for his pioneering contributions to numerical algorithms and libraries that have enabled high-performance computational software to keep pace with exponential hardware improvements. youtu.be/Oe9LRKoE6L0?...

A successful week at #ICML26 is nearing its end: Learn about (1) weather model scaling laws from Yuejiang, (2) near-optimal quantization from Jiale, (3) sparse data scaling laws from Boqian, (4) a cool algorithm for sparse training from Qiao, and (5) process reward agents from Jiwoong, Tomasz et al

The actual #AI and #HPC accelerators are the building blocks: various precision matrix and vector units and high-bandwidth memory will define the future of accelerated computation much more than #GPUs vs. #CPUs. Also opening the door for highly-efficient software-defined dataflow.

It was great attending the 75th Lindau Nobel Laureate Meeting last week. I'm very much looking forward to deliver the Heidelberg Plenary Lecture in #AI next year at the 76th! Congratulations to both Peter and Nikolaus for receiving those special honors.

ADIA Lab in Abu Dhabi is looking for a Senior Computational Scientist in Trustworthy #AI to lead a world-class research program. If you have a PhD, 5+ years of experience, and a strong publication record in top-tier venues like #NeurIPS and #ICML, then go and check it out! Apply:

Big congrats to our Racklette team for the 1st prize in the #ISC26 student cluster competition. Switzerland is back, very proud of this repeated success!! Thanks to Hussein and CSCS for the strong support. Look at that big golden trophy for the future leaders in #AI and #HPC!

Saverio presents the #ISC26 Hans Meuer award paper. Selecting and tuning collective operations is not simple. Need to consider various systems effects, for example, load on global vs. local links and different phases. Also using ATLAHS #HPC for selection isc-hpc.com/isc-high-per...

This year's Jack Dongarra award goes to Devesh Tiwari. He starts his talk with an hommage to Jack and then lures the audience into quantum computing. Well done 😀.

Kudos to ETH Zurich's team RACKlette competing at the #ISC26 conference. Next generation #HPC and #AI experts are competing at one of the most prestigious international venues. Learn about the team at buff.ly/ZFBJRSh Wish them all luck!! Fight hard to win!

Surprise: Chinese LineShine supercomputer takes the #1 spot on the #Top500 and #HPCG lists! First time since a while that spot is taken by a homogeneous CPU machine. So is heterogeneity here to stay? CPUs with HBM and tensor cores (e.g., ARM SME). Congrats Yutong and folks.

Martin Schulz from TUM says "heterogeneity is here to stay" and highlights several approaches like quantum, neuromorphic, and ofc. GPUs. Quantum and all of those are #HPC and that's what we're here for at #ISC26! Great opening. We may see a surprise here in an hour 😉.

As #ISC26 is coming up, I wrote down thoughts on the central metric in our #HPC and #AI community: flop/s. A flop is a count; flop/s is a rate. FLOPS is yelling, flops is confusing, and “TFLOPS” is how unit symbols go to therapy. HPC deserves better. buff.ly/LpWDYBU Enjoy & see you in Hamburg!

At #AI4X, Michele Ceriotti questions inductive biases even in material sciences, calling "Let them learn". An interesting hypothesis also in other sciences. I will give an overview of AI for Science in Climate Sciences, one of the hardest to validate. Join us tomorrow 9am! #AI and #HPC

Arrived at #AI4X in Singapore for Chorh Chuan Tan's opening remarks setting the stage for AI for Science as an accelerator. "Nations not using AI in Science will likely fall behind". SG will "converge science" with computation and AI through autonomous labs. What a time to be in #HPC and #AI

News from Jensen Huang @ Build: "Microsoft deployed the largest number of Grace Blackwells in the world today." Kudos to the team for the elegant network with MRC, a step towards UE - enabling very large, cost-effective, and resilient deployments. buff.ly/7WtYXiL buff.ly/rZGTEqm [29:45ff]

The SciFM conversation is shifting from models to systems. The interesting questions this year were increasingly about how to connect agents, instruments, data, simulations, and compute into effective discovery ecosystems. cs.uchicago.edu/news/scifm-2... @ramanathanlab.bsky.social

Big announcements on MS Build 💪: Maia 200 now online in Iowa and Arizona, going international soon - at 30% better TCO than best GPUs! Big success for our SDLA architecture. 🚀 Running GPT-5.5/copilot and MAI's thinking model achieves 1.4x perf/W vs. GB200! 👏 buff.ly/WuGgZm4 (22:40 and 1:55:17)

After a nice early morning run, #SciFM26 in Chicago continues. Rafael shows how many physics models actually learn the same representations in their latent space if you compare proximity of embeddings. Next: my panel on the disappointment of physical biases and priors in climate foundation models.

Just arrived at SciFM in Chigago. First thing I hear from the panel is great: "Agents are the new programming. They form their own language and system stack. And we computer scientists should contribute the algorithms." 💯, we also need to develop the best architectures! #HPC

Mateo Valero discusses history of vector architectures: Back to the Future in #HPC and #AI. He also mentions the three quantum computers connected to an NVIDIA Blackwell system in BSC's chapel ⛪️. Ending his talk looking for a future site 😀.

Taisuke Boku follows on another interesting note: Japan is "changing lanes" from CPU to GPU and from "Made in Japan" to a collaborative "Made with Japan", partnering Fujitsu with NVIDIA. "We don't chase fp64 performance but focus on #AI flops". Onward!

Yutong Lu reveals the LineShine exascale architecture for #HPC - #AI convergence: 2 exaflop sustained fp64 at 40 MW with 1.34 million ARM CPUs using SME and 4TB/s HBM connected with 800G. From "offline to online acceleration with SME".

Jack Dongarra's opening keynote at HACI 2026! #HPC in transition - the investments in compute infrastructure are unprecedented but focus mostly on #AI workloads. Scientific numerical simulations need to adjust! Many interesting challenges ahead.

Read how we designed and deployed Multipath Reliable Connection (MRC), a multi-company step towards Ultra Ethernet, at 800G in Microsoft's datacenters. Demonstrated with a 75k GPU pretraining job running stably through multiple faults. OpenAI blog: buff.ly/X0KL4Xy Paper: buff.ly/I3OcXWn

The future is here on my way back from Salishan. My air office is a bit narrow but with both faster Internet than at home and a better view. Very well done Starlink and United! Now, Swiss and Lufthansa - time to catch up :-).