Commit 0496413
[SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations
### What changes were proposed in this pull request?
precompute the `DecisionTreeMetadata` and reuse it for all trees
### Why are the changes needed?
In existing impl, each `DecisionTreeRegressor` needs a pass on the whole dataset to calculate the same `DecisionTreeMetadata` repeatedly.
In this PR, with default depth=5, it is about 8% faster then existing impl
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
existing testsuites
Closes apache#27011 from zhengruifeng/gbt_reuse_instr_meta.
Authored-by: zhengruifeng <ruifengz@foxmail.com>
Signed-off-by: Sean Owen <srowen@gmail.com>1 parent 16e5e79 commit 0496413
File tree
2 files changed
+24
-7
lines changed- mllib/src/main/scala/org/apache/spark/ml/tree/impl
2 files changed
+24
-7
lines changedLines changed: 11 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
| 23 | + | |
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| |||
306 | 306 | | |
307 | 307 | | |
308 | 308 | | |
309 | | - | |
310 | | - | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
311 | 315 | | |
312 | 316 | | |
313 | 317 | | |
| |||
342 | 346 | | |
343 | 347 | | |
344 | 348 | | |
345 | | - | |
346 | | - | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
347 | 353 | | |
348 | 354 | | |
349 | 355 | | |
| |||
Lines changed: 13 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
99 | 99 | | |
100 | 100 | | |
101 | 101 | | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
102 | 111 | | |
103 | 112 | | |
104 | 113 | | |
| |||
113 | 122 | | |
114 | 123 | | |
115 | 124 | | |
116 | | - | |
| 125 | + | |
| 126 | + | |
117 | 127 | | |
118 | 128 | | |
119 | 129 | | |
| |||
122 | 132 | | |
123 | 133 | | |
124 | 134 | | |
125 | | - | |
| 135 | + | |
126 | 136 | | |
| 137 | + | |
127 | 138 | | |
128 | 139 | | |
129 | 140 | | |
| |||
0 commit comments