I get the desire for ESM for things like tree shaking (though I'd like to see some numbers on impact here). I get the nicer syntax.
But, unbundled ESM takes about 2x as long as CJS to load in my tests (can show if desired). If your users aren't bundling ESM opts them in to big a load-time perf hit.