[SPARK-7022][PySpark][ML] Add ML.Tuning.ParamGridBuilder to PySpark

[oefirouz]

No description provided.

[ash211]

Jenkins this is ok to test

[SparkQA]

Test build #30616 has finished for PR 5601 at commit 8b8a6d2.

• This patch fails Python style tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class ParamGridBuilder(object):
• This patch does not change any dependencies.

[mengxr]

Return self so this could be chained.

[SparkQA]

Test build #30630 has finished for PR 5601 at commit 9a8ce22.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class ParamGridBuilder(object):
• This patch does not change any dependencies.

[SparkQA]

Test build #30647 has finished for PR 5601 at commit c9e2481.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class ParamGridBuilder(object):
• This patch does not change any dependencies.

[oefirouz]

Friendly bump for more comments :)

[mengxr]

It looks good to me. We need to run Jenkins again to make sure it doesn't conflict with recent changes to the pipeline API.

[mengxr]

test this please

[SparkQA]

Test build #31312 has finished for PR 5601 at commit c9e2481.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class ParamGridBuilder(object):
• This patch does not change any dependencies.

[mengxr]

test this please

[SparkQA]

Test build #31457 has finished for PR 5601 at commit c9e2481.

• This patch fails MiMa tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class ParamGridBuilder(object):
• This patch does not change any dependencies.

[oefirouz]

Should I rebase and update this? I'm not sure why the build is failing.

[justinuang]

Consider doing this

[dict(zip(self._param_grid.keys(), prod)) for prod in itertools.product(*self._param_grid.values())]

To avoid the overhead of lots of dictionary copies.

[mengxr]

+1 on this. Should we move self._param_grid.keys()/values() out for readability?

keys = self._param_grid.keys()
grid_values = self._param_grid.values()
return [dict(zip(keys, prod)) for prod in itertools.product(*grid_values)]

[justinuang]

Yea, you should try rebasing. It looks like you're not the only one running into this.

https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31639/console

[mengxr]

test this please

[SparkQA]

Test build #31692 has finished for PR 5601 at commit c9e2481.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class ParamGridBuilder(object):

[mengxr]

LGTM. I've merged this into master and I'll send a follow-up PR to implement what @justinuang suggested.