Skip to content

[SPARK-9478] [ml] Add class weights to Random Forest #13851

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
n-triple-a wants to merge 18 commits into from
Closed

[SPARK-9478] [ml] Add class weights to Random Forest #13851

Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
bc7e824
Modify impurity implementations, NOTICE: a further modification is ne…
Jun 7, 2016
df3b4e7
save changes, but compile error exists
Jun 7, 2016
7bcabda
simple testSuites and run properly
Jun 7, 2016
61c4858
add unbalenced data test case, modify label prediction process s.t. t…
Jun 7, 2016
aeb0856
Make Decision Tree predicions correct without changing the base class…
Jun 7, 2016
4dc3e32
1.put classweight def to the right area 2.change interfaces of getOld…
Jun 10, 2016
ad55f6d
add SetClassWeights to RandomForestClassifier
Jun 15, 2016
c17067f
change code style such that requirements are met
Jun 15, 2016
17635d9
random forest with class weights runs properly
Jun 15, 2016
9d52c1f
minor changes
Jun 16, 2016
bf1acfd
add Strategy new param doc
Jun 17, 2016
2baf814
move classW def to DeTrParam class
Jun 17, 2016
fe3819c
remove @BeanProperty
Jun 17, 2016
fd2eee5
change getOldImpu interfaces
Jun 17, 2016
455c47e
first version that pass all run_test, by adding redundant constructor…
Jun 20, 2016
9c99973
first version that pass all tests
Jun 21, 2016
f53a2cc
minor modifications for code styling
Jun 21, 2016
ec5f6df
Minor code style modificaions
Jun 22, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Minor code style modificaions
  • Loading branch information
Yuewei Na authored and Yuewei Na committed Jun 22, 2016
commit ec5f6df4a5c11053f4a57201a09b631a5f8b81cf
8 changes: 4 additions & 4 deletions mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ class RandomForestSuite extends SparkFunSuite with MLlibTestSparkContext {
val fakeMetadata = new DecisionTreeMetadata(1, 0, 0, 0,
Map(), Set(),
Array(6), Gini, QuantileStrategy.Sort,
0, 0, 0.0, 0, 0, Array()
0, 0, 0.0, 0, 0, Array[Double]()
)
val featureSamples = Array.fill(200000)(math.random)
val splits = RandomForest.findSplitsForContinuousFeature(featureSamples, fakeMetadata, 0)
Expand All @@ -110,7 +110,7 @@ class RandomForestSuite extends SparkFunSuite with MLlibTestSparkContext {
val fakeMetadata = new DecisionTreeMetadata(1, 0, 0, 0,
Map(), Set(),
Array(5), Gini, QuantileStrategy.Sort,
0, 0, 0.0, 0, 0, Array()
0, 0, 0.0, 0, 0, Array[Double]()
)
val featureSamples = Array(1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3).map(_.toDouble)
val splits = RandomForest.findSplitsForContinuousFeature(featureSamples, fakeMetadata, 0)
Expand All @@ -124,7 +124,7 @@ class RandomForestSuite extends SparkFunSuite with MLlibTestSparkContext {
val fakeMetadata = new DecisionTreeMetadata(1, 0, 0, 0,
Map(), Set(),
Array(3), Gini, QuantileStrategy.Sort,
0, 0, 0.0, 0, 0, Array()
0, 0, 0.0, 0, 0, Array[Double]()
)
val featureSamples = Array(2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 4, 5).map(_.toDouble)
val splits = RandomForest.findSplitsForContinuousFeature(featureSamples, fakeMetadata, 0)
Expand All @@ -138,7 +138,7 @@ class RandomForestSuite extends SparkFunSuite with MLlibTestSparkContext {
val fakeMetadata = new DecisionTreeMetadata(1, 0, 0, 0,
Map(), Set(),
Array(3), Gini, QuantileStrategy.Sort,
0, 0, 0.0, 0, 0, Array()
0, 0, 0.0, 0, 0, Array[Double]()
)
val featureSamples = Array(0, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2).map(_.toDouble)
val splits = RandomForest.findSplitsForContinuousFeature(featureSamples, fakeMetadata, 0)
Expand Down