Skip to content

[SPARK-22666][ML][SQL] Spark datasource for image format #22328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
WeichenXu123 wants to merge 7 commits into from
Closed

[SPARK-22666][ML][SQL] Spark datasource for image format #22328

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
5b5aee6
init pr
WeichenXu123 Sep 4, 2018
5164d19
fix scala style
WeichenXu123 Sep 4, 2018
bd6178c
address comments
WeichenXu123 Sep 5, 2018
4d52754
address comments
WeichenXu123 Sep 5, 2018
3fffd7e
nit update
WeichenXu123 Sep 5, 2018
3c8863c
more nit updates
WeichenXu123 Sep 5, 2018
218ce4c
address comments
WeichenXu123 Sep 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix scala style
  • Loading branch information
@WeichenXu123
WeichenXu123 committed Sep 4, 2018
commit 5164d19deb447e49e87d20f5a1efbb0c2ee177ee
2 changes: 1 addition & 1 deletion mllib/src/main/scala/org/apache/spark/ml/source/image/ImageDataSource.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,4 +48,4 @@ package org.apache.spark.ml.source.image
* @note This class is public for documentation purpose. Please don't use this class directly.
* Rather, use the data source API as illustrated above.
Copy link
Member

@HyukjinKwon HyukjinKwon Sep 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WeichenXu123, don't we plan to make a documentation in the site?

Copy link
Contributor

@mengxr mengxr Sep 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't see a section in the doc that lists all built-in data sources. It would be nice if we create a section and link it to this API doc. I think we can do it with a follow-up PR. I want to see if we can get this PR merged before branch cut:)

*/
class ImageDataSource private() {}
class ImageDataSource private() {}
Copy link
Contributor

@cloud-fan cloud-fan Sep 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this class?

Copy link
Contributor Author

@WeichenXu123 WeichenXu123 Sep 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for doc.
similar to LibSVMDataSource

Copy link
Contributor

@cloud-fan cloud-fan Sep 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a convention? AFAIK in the scala world we usually put document in package object.

Copy link
Member

@HyukjinKwon HyukjinKwon Sep 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just simply remove this and make a followup for the doc in the site ..

Copy link
Contributor

@mengxr mengxr Sep 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re: @cloud-fan The Scala package doc doesn't work for Java, which requires a different format.

Re: @HyukjinKwon It would be nice to have some doc in the site, though I didn't find the list of built-in data sources in the doc site. I think it is okay to have docs in both locations for IDE users and for people search on the web.