Fix JsonPrimitive.hashCode to honor the equals/hashCode contract (#992)#3059
Fix JsonPrimitive.hashCode to honor the equals/hashCode contract (#992)#3059earfman wants to merge 3 commits into
Conversation
eamonnmcmanus
left a comment
There was a problem hiding this comment.
Thanks for looking into this! I agree that it is a problem that the current JsonPrimitive.hashCode() violates the contract for hashCode().
Would it be possible to structure the code of the rewritten hashCode so it is more similar to the code of equals? That way it would be easier to see that the rewritten code does indeed respect the contract. So in other words it would have basically the same conditions in the same order (isIntegral { instanceof BigInteger / getAsNumber().longValue() } etc).
| return (int) (bits ^ (bits >>> 32)); | ||
| // equals compares the remaining numbers by their double values. (For two BigDecimals it | ||
| // uses compareTo, which only considers values equal that also have the same double value.) | ||
| return hashOfDoubleValue(getAsNumber().doubleValue()); |
There was a problem hiding this comment.
I don't think this approach is going to work. Not all BigDecimal or BigInteger values will fit in a double. I suggested logic parallel to the equals logic because I think that is necessary.
For BigDecimal, I think canonicalizing via stripTrailingZeros() should be enough to ensure that two values for which equals returns true will have the same hashCode.
There was a problem hiding this comment.
Thanks — you're right that not every BigDecimal/BigInteger fits in a double, and I think that's exactly the crux, so I want to lay out carefully why the double-based hash is forced by the current equals, and where that leaves your stripTrailingZeros() suggestion.
Why the hash must be a function of the double value (given equals as it stands): every numeric JsonPrimitive is equals() to some primitive whose comparison runs through the double path. Cross-type pairs (BigDecimal vs Double, BigInteger vs Double, etc.) skip both the integral-integral branch and the both-BigDecimal branch, and compare getAsDouble(). So, for example, on current main:
new JsonPrimitive(new BigDecimal("42"))equalsnew JsonPrimitive(42)— via the double path. WithstripTrailingZeros().hashCode()for theBigDecimalbranch these would hash1302vs42, a contract violation on the most ordinary input.new JsonPrimitive(new BigDecimal("0.1"))equalsnew JsonPrimitive(0.1)(same double), butstripTrailingZeroshashes the exact decimal0.1, not the double — again unequal hashes for equal values.
I did check the suggestion empirically before replying: adding stripTrailingZeros().hashCode() for BigDecimal on top of this PR's structure produces 39 equal-but-different-hash ordered pairs on a ~26-value all-pairs matrix plus 300k randomized cross-type pairs (current main scores 75 on the same harness; this PR scores 0). Happy to share the harness or fold its cases into the test.
Beyond-double values can't hash distinctly under the current equals either: BigDecimal("1e400") and BigDecimal("2e400") are each equals() to JsonParser.parseString("1e400") (a LazilyParsedNumber, whose doubleValue() is +Infinity) — so by the contract they're transitively forced to share one hash even though compareTo distinguishes them. That's parser-produced, so it can't be avoided by rejecting infinite Doubles at construction. Under this PR they simply collide (legal for hashCode, and only reachable for magnitudes beyond ~1.8e308).
One behavior note I should make explicit: for the same reason, this PR changes the hashes of Long/BigInteger values above 2^53 (e.g. Long(2^53+1)). That's forced: on main, Long(2^53+1) equals Double(2^53), so they must share a hash.
The fork in the road: if you'd like values beyond double precision to hash distinctly, I believe equals itself has to be refined first (e.g. exact numeric comparison for all Number pairs) — at which point stripTrailingZeros-style hashing becomes exactly right. That would be a visible behavior change (Double(0.1) would stop equaling BigDecimal("0.1")), but it would also fix a pre-existing non-transitivity: today Long(2^53+1) ~ Double(2^53) ~ Long(2^53) while Long(2^53+1) ≠ Long(2^53). I deliberately kept equals untouched in this PR, but I'm happy to take that on here or in a follow-up if you'd prefer that direction — your call on scope.
Fixes #992.
Problem
JsonPrimitive.equalsconsiders numbers with the same value but differentNumberbacking types equal — e.g.new JsonPrimitive(42), a lazily-parsed"42"(the type produced by round-tripping a number through JSON), andnew JsonPrimitive(42.0)are all equal to one another. ButhashCodebranched onisIntegral(this):An integral primitive (
42, hashed as along) and an equal non-integral one (lazily-parsed"42", hashed viadoubleToLongBits) therefore get different hash codes, violating theObject.hashCodecontract. In practice this breaksHashMap/HashSet: a value stored undernew JsonPrimitive(42)can't be found using the equal primitive you get back after parsing the same number from JSON.Fix
Derive the hash code solely from the
doublevalue that every branch ofequalsultimately compares:longrange are hashed as thelong(this keeps the hash unchanged for the common integral types, and normalizes-0.0to0, whichequalstreats as equal to0).doubleToLongBits(doubleValue)(which canonicalizes NaN, matchingequals).equalsis left untouched.Scope / disclosure
hashCodeused two different algorithms (integral → hash thelong; otherwise → hashdoubleToLongBits), which is the root cause of the bug. Now anything integral-valued and inlongrange hashes as thelong. Concretely: an integral-valued floating primitive like42.0now hashes like42;-0.0now hashes like0; and an integral value with magnitude> 2^53(aLong/BigIntegeradoublecan't represent exactly) now hashes by itsdoublevalue, so9007199254740993Lmatches9007199254740992.0. In every one of those cases the new hash agrees with another primitive thatequalsalready considers equal. Purely fractionalDouble/Float/BigDecimalvalues are unchanged. Hash codes aren't part of gson's serialized contract, but I'm flagging the change explicitly.equalsitself is non-transitive around the2^53boundary (two distinctlongs that both round to the samedoubleare each "equal" to thatdouble, but not to each other). This PR does not touchequalsand does not try to fix that; it only makeshashCodeconsistent withequalsas currently defined. Happy to file a separate issue if that's useful.Testing
testEqualsAndHashCodeAcrossNumberTypesasserts equal-implies-equal-hashcode acrossInteger/LazilyParsedNumber/Double/BigDecimal/BigInteger, the JSON round-trip case from the issue,-0.0vs0, and the> 2^53long/double case.gsonmodule test suite passes (mvn test); Spotless clean.Authored with AI assistance; the change was independently reviewed and verified before submission (including a fuzz check confirming no
equals-equal pair produces differing hash codes). I'll sign the Google CLA.