Commit 5d3f461
[SPARK-17506][SQL] Improve the check double values equality rule.
## What changes were proposed in this pull request?
In `ExpressionEvalHelper`, we check the equality between two double values by comparing whether the expected value is within the range [target - tolerance, target + tolerance], but this can cause a negative false when the compared numerics are very large.
Before:
```
val1 = 1.6358558070241E306
val2 = 1.6358558070240974E306
ExpressionEvalHelper.compareResults(val1, val2)
false
```
In fact, `val1` and `val2` are but with different precisions, we should tolerant this case by comparing with percentage range, eg.,expected is within range [target - target * tolerance_percentage, target + target * tolerance_percentage].
After:
```
val1 = 1.6358558070241E306
val2 = 1.6358558070240974E306
ExpressionEvalHelper.compareResults(val1, val2)
true
```
## How was this patch tested?
Exsiting testcases.
Author: jiangxingbo <[email protected]>
Closes #15059 from jiangxb1987/deq.1 parent 3fe630d commit 5d3f461
File tree
2 files changed
+30
-7
lines changed- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions
2 files changed
+30
-7
lines changedLines changed: 3 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
170 | 170 | | |
171 | 171 | | |
172 | 172 | | |
173 | | - | |
174 | | - | |
175 | | - | |
176 | | - | |
177 | | - | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
178 | 176 | | |
179 | 177 | | |
180 | 178 | | |
| |||
Lines changed: 27 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
22 | 23 | | |
23 | 24 | | |
24 | 25 | | |
| |||
289 | 290 | | |
290 | 291 | | |
291 | 292 | | |
292 | | - | |
293 | | - | |
294 | 293 | | |
295 | 294 | | |
| 295 | + | |
| 296 | + | |
296 | 297 | | |
297 | 298 | | |
298 | 299 | | |
299 | 300 | | |
300 | 301 | | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
301 | 326 | | |
0 commit comments