Post

Does anyone have opinions on non-cryptographic hashes that are relatively simple, can be done streaming (don't know the length apriori), have good avalanche characteristics, and produce a 64-bit output? Currently using fnv1a64 for postcard-rpc schema hashing, wondering if there's something better.

0 reposts8 likes

I don’t have an answer to your question, but is there a “design doc” for postcard-rpc that explains why design decisions were made? I’m wondering why a small, serially allocated integer is insufficient, when both sides know the full set of possible message types?

I've started working on it at postcard.rs, but the short answer is that postcard is a non self describing binary format, and so if you change the schema on one side but not the other, you could misinterpret or fail to decode messages. Using a hash of the schema means the ID changes if the type does.

postcard.rspostcard.rs

So, the main answer is: because you might change the code for the client and the server at different times, we want to make sure both ends agree, even over time and different firmware/software versions. The versioning is also per-endpoint, so you can also use this to support multiple revs over time

Why simple: I do it in const context in Rust Why no length: I do it recursively, would prefer not to need to collect and then hash, instead of hash as I recursively walk Why 64 bit: I dunno that's the size I chose. I could mix down with xor if I need to I guess

I did look at murmur3 and cityhash, the former was more complex than I'd hoped (but I only took a quick look, also I think it was arch-dependent?), and cityhash looked good but requires you to know the length ahead of time.

I did look at murmur3, but it seemed non-deterministic on different architectures, and only has 32 and 128 bit versions. I didn't look too closely at previous versions.