Post

@javier-romera.bsky.social

I especially appreciate the standalone builds on HPC clusters. Where I usually have to do `module load python/3.11.6` and remember the correct Python version I created whichever venv with. With uv's "only-managed" option, I no longer have to worry about that.

Javier Romera

14 Dec 2024

Great news. Do not forget pyoxidozer. It is a great tool, that just needs a bit of more compatibility, and could be part of uv, maybe as "uv bundle" command. It would be great that Indygreg huge effort does not vanish.

Chris Kornaros

@chriskornaros.bsky.social

Great read!

First, some context on the problem. Normally, when you build CPython on Linux or macOS, several system paths are hardcoded into the binary.

This is fine if you're building and installing Python on a single machine... but it's a problem if you want to pre-build Python and then distribute it to other machines, since those paths might be wrong!

Normally, CPython is also dynamically linked against a number of system libraries, which can cause trouble if the target machine doesn't have those libraries installed.

In this way, CPython is not "standalone": it's tightly coupled to the system on which it was built (at least on Linux and macOS).

Hynek Schlawack

@hynek.me

Which reminds me of my main problem with standalone builds on macOS: I can’t compile wheels because they assume some super weird Xcode path. I think it’s fair to say they can’t go full mainstream with that limitation so I wonder if that’s solvable at all? :-/

What’s this? I don’t have this problem and I build wheels all the time. Is there an issue somewhere?

Brandon Y. Yang

@brandonyifanyang.com

uv on HPC is such a cheat code. It is so easy to share python environments with collaborators, no more locally installed conda envs, or funky envs that don’t work. @crmarsh.com Do you have any other suggestions on how uv can be optimized for HPC usage?

nanobowers.bsky.social

@nanobowers.bsky.social

@notatallshaw.bsky.social

Can you share any links / whitepapers on how to best manage varied python installs and venvs on HPC?

Damian

Conda solves some of these problems, but not the relocatable issue.

Sebastian Penhouet

@spenhouet.com