The best single quality benchmark is the SMHasher test suite.
The best single quality benchmark is the SMHasher test suite. For most purposes it is very thorough at finding defects in non-cryptographic hash function designs. A strong design has good statistical properties regardless of the types of key sets and SMHasher tests many kinds of key sets, including several that are designed to be pathological. There are two caveats with using SMHasher to be aware of. First, it does not thoroughly exercise the internals of some hash function constructions that you sometimes see in larger hashes. You can modify the source to add this coverage but it is not there by default. Second, the speed test implementation is broken. It is good enough to give you a rough idea of what is going on for bulk hashing relative to other algorithms, but the absolute performance measurement deviate from proper testing by a significant amount (up to 20-25% anecdotally). Also really fast small key algorithms can be significantly off. Basically, performance is measured using a method that Intel's performance measurement documentation strongly advises to not do. This can give inconsistent results even across CPUs that are based on the same microarchitecture.
Posted by jandrewrogers on May 27, 2015, 9:56 PM
Tags
Related issues
- This is my biggest problem with agents, they constantly defensively code for cases that have no chance of happening and in most cases a p...
- I wish there was something like LEAN for biology.
- The issue is really the extreme difficulty of programming them, especially for anything sequential (rather than combinatorial).
- I probably could have beared with it if it was only limited to programming, but it showing up in creative fields has made me feel all sor...
- I hate that I'm in deep enough to know what you mean.
- The biggest problem with “AI” is that big corps are shoving it down our throats.
Raise real-world problems, find people facing the same issue, and discuss solutions.6,500 issues0 comments
reliability (Other)
This is my biggest problem with agents, they constantly defensively code for cases that have no chance of happening and in most cases a p...
I wish there was something like LEAN for biology.
The issue is really the extreme difficulty of programming them, especially for anything sequential (rather than combinatorial).
I probably could have beared with it if it was only limited to programming, but it showing up in creative fields has made me feel all sor...
I hate that I'm in deep enough to know what you mean.
The biggest problem with “AI” is that big corps are shoving it down our throats.
in the middel of a coding session with 5.6 sol.
[Hero Run] 535B-A23B on 18T tokens
The best single quality benchmark is the SMHasher test suite.
The best single quality benchmark is the SMHasher test suite. For most purposes it is very thorough at finding defects in non-cryptographic hash function designs. A strong design has good statistical properties regardless of the types of key sets and SMHasher tests many kinds of key sets, including several that are designed to be pathological. There are two caveats with using SMHasher to be aware of. First, it does not thoroughly exercise the internals of some hash function constructions that you sometimes see in larger hashes. You can modify the source to add this coverage but it is not there by default. Second, the speed test implementation is broken. It is good enough to give you a rough idea of what is going on for bulk hashing relative to other algorithms, but the absolute performance measurement deviate from proper testing by a significant amount (up to 20-25% anecdotally). Also really fast small key algorithms can be significantly off. Basically, performance is measured using a method that Intel's performance measurement documentation strongly advises to not do. This can give inconsistent results even across CPUs that are based on the same microarchitecture.
Comments (0)
No comments yet. Start the discussion.