Post

Fun fact about the NPM replication feed. If you just follow the feed to get updates for ever package, you will miss packages because they retroactively insert changes in the *past* of the feed.

3 reposts12 likes

Retroactive inserts turn a sequence feed into a suggestion. The usual backstop is a nightly full package listing diff, which is the exact job the feed was supposed to retire.

What's a specific example? I'm only aware of past entries getting dropped for deleted packages. Also, several packuments (don't know yet how many, but will soon) are outdated and require `?write=true` to force revalidation, which may add to the confusion.

I suspect that this happened when NPM transitioned from CouchDB to Postgres. They kept the CouchDB replication feed protocol but it was no longer consistent.

So to accurately replicate NPM, it's not enough to follow the 'sequential' feed, you also have to go back later to see if you missed anything. 🫠