Skip to content

[SPARK-20393][WEBU UI][1.6] Strengthen Spark to prevent XSS vulnerabilities #19528

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ambauma wants to merge 6 commits into from
Closed

[SPARK-20393][WEBU UI][1.6] Strengthen Spark to prevent XSS vulnerabilities #19528

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d9a45aa
Initial Merge of SPARK-20393 to 1.6 branch
n-marion May 10, 2017
630854a
Removing what I believe is extra code never intended for the Spark 1.…
ambauma Oct 10, 2017
ffe3e98
Adding back in DriverPage.scala
ambauma Oct 10, 2017
cafe18d
Added missing XSS strip based on comparison to 2.0 branch.
ambauma Oct 19, 2017
cb1609b
[SPARK-20862][MLLIB][PYTHON] Avoid passing float to ndarray.reshape i…
MrBago May 24, 2017
76ad8c5
Changes based on comments.
ambauma Oct 21, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[SPARK-20862][MLLIB][PYTHON] Avoid passing float to ndarray.reshape i… 
…n LogisticRegressionModel

## What changes were proposed in this pull request?

Fixed TypeError with python3 and numpy 1.12.1. Numpy's `reshape` no longer takes floats as arguments as of 1.12. Also, python3 uses float division for `/`, we should be using `//` to ensure that `_dataWithBiasSize` doesn't get set to a float.

## How was this patch tested?

Existing tests run using python3 and numpy 1.12.

Author: Bago Amirbekian <[email protected]>

Closes #18081 from MrBago/BF-py3floatbug.
  • Loading branch information
@MrBago @ambauma
MrBago authored and ambauma committed Oct 20, 2017
commit cb1609b055dafd78af15d7a1b19658f81df1ebca
2 changes: 1 addition & 1 deletion python/pyspark/mllib/classification.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ def __init__(self, weights, intercept, numFeatures, numClasses):
self._dataWithBiasSize = None
self._weightsMatrix = None
else:
self._dataWithBiasSize = self._coeff.size / (self._numClasses - 1)
self._dataWithBiasSize = self._coeff.size // (self._numClasses - 1)
Copy link
Member

@srowen srowen Oct 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nor this?

Copy link
Author

@ambauma ambauma Oct 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to apply this to get past a python unit test failure. My assumption is that the NewSparkPullRequestBuilder is on a different version of numpy than when the Spark 1.6 branch was last built. The current python unit test failure looks like it has to do with a novel version of SciPy.

Copy link
Author

@ambauma ambauma Oct 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already fixed in the 2.0 branch, btw. Just was never applied to 1.6. [SPARK-20862]

self._weightsMatrix = self._coeff.toArray().reshape(self._numClasses - 1,
self._dataWithBiasSize)

Expand Down