Post

Actually, they are deterministic, right ? They don't feel that way because we use random seeds ( we are not deterministic in our inputs)

Even with a fixed seed you can still get non-deterministic results out of most LLMs, because it turns out floating point math isn't commutative so when you run it on GPUs that execute multiple operations in parallel the order in which they finish can have an impact on the final result

Really ? I remember using Phi in local with the default seed, and having always the same outputs ( as I was always testing with the same inputs). And it was a strange feeling when you are used to have different answer from LLMs.

Test-driven development is going to be huge. It's a good paradigm also without the LLMs but now it's going to be orders of magnitude more important. I like how I'm already proofing all my code against people/machines who modify it without much understanding. The future is all about verifying!

Yes, but! Testability is deeply synergetic with good design. Unless continual refactoring is accomplished to a good standard as well, a pile of generated tests may become a burden as much as an asset. There's a real risk of shortchanging that, as folks already do even without genAI.

True, no LLM will replace understanding and good design. Good architecture and thorough testing will guide your attention if something breaks. And I do see the risk if both tests and the "main" code are be generated with nobody processing any of it well.

Wait, sheepdogs are terrible at woodworking. I have enough trouble keeping my border collie entertained let alone getting her to hold the wood still while I saw.

This is one of the things that gives me pause; it’s hard, I think, to see how all those ‘Build awesome stuff in Python!’ books aimed at children translate to a Cursor/CoPilot/Sonnet world. Teaching by typing in listings no longer seems ‘right,’ but we don’t yet have an alternative pedagogy.

Well. One of the things that typing in listings teaches you is that individual words and characters matter, and how to notice and fix such issues. That is a thing that LLMs don't change and no other pedagogy even approaches.

That’s true, but also misses my point. Which is that ‘typing code’ no longer has to be the principle mode of interaction with that code. I struggle with how best to demonstrate process for my undergrads, let alone my kids.