Skip to content

[SPARK-43849][BUILD] Enable unused imports check for Scala 2.13 #41356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
LuciferYang wants to merge 5 commits into from
Closed

[SPARK-43849][BUILD] Enable unused imports check for Scala 2.13 #41356

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
cb1bab0
fixed
LuciferYang May 29, 2023
3d9dd93
revert
LuciferYang May 29, 2023
5d24b16
fix sbt
LuciferYang May 29, 2023
0e615ef
remove SPARK-33499
LuciferYang May 29, 2023
e80f584
rm empty line
LuciferYang May 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions ...nect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,9 @@ import org.apache.spark.connect.proto
import org.apache.spark.connect.proto.{ExecutePlanResponse, SqlCommand, StreamingQueryCommand, StreamingQueryCommandResult, StreamingQueryInstanceId, WriteStreamOperationStart, WriteStreamOperationStartResult}
import org.apache.spark.connect.proto.ExecutePlanResponse.SqlCommandResult
import org.apache.spark.connect.proto.Parse.ParseFormat
import org.apache.spark.connect.proto.StreamingQueryCommand
Copy link
Contributor Author

@LuciferYang LuciferYang May 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scala 2.13 checked them unused as they have already been imports in line 31

import org.apache.spark.connect.proto.StreamingQueryCommandResult
import org.apache.spark.connect.proto.StreamingQueryInstanceId
import org.apache.spark.connect.proto.StreamingQueryManagerCommand
import org.apache.spark.connect.proto.StreamingQueryManagerCommandResult
import org.apache.spark.connect.proto.StreamingQueryManagerCommandResult.StreamingQueryInstance
import org.apache.spark.connect.proto.WriteStreamOperationStart
import org.apache.spark.connect.proto.WriteStreamOperationStart.TriggerCase
import org.apache.spark.ml.{functions => MLFunctions}
import org.apache.spark.sql.{Column, Dataset, Encoders, ForeachWriter, RelationalGroupedDataset, SparkSession}
Expand Down
9 changes: 8 additions & 1 deletion pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2855,9 +2855,9 @@
<arg>-explaintypes</arg>
<arg>-target:jvm-1.8</arg>
<arg>-Wconf:cat=deprecation:wv,any:e</arg>
<arg>-Wunused:imports</arg>
<!--
TODO(SPARK-33805): Undo the corresponding deprecated usage suppression rule after fixed
<arg>-Wunused:imports</arg>
-->
<arg>-Wconf:cat=scaladoc:wv</arg>
<arg>-Wconf:cat=lint-multiarg-infix:wv</arg>
Expand Down Expand Up @@ -2891,6 +2891,13 @@
<arg>-Wconf:cat=unchecked&amp;msg=outer reference:s</arg>
<arg>-Wconf:cat=unchecked&amp;msg=eliminated by erasure:s</arg>
<arg>-Wconf:msg=^(?=.*?a value of type)(?=.*?cannot also be).+$:s</arg>

Copy link
Member

@dongjoon-hyun dongjoon-hyun May 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit. Redundant empty line.

Copy link
Contributor Author

@LuciferYang LuciferYang May 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dongjoon-hyun , e80f584 removed it

<!--
TODO(SPARK-43850): Remove the following suppression rules and remove `import scala.language.higherKinds`
from the corresponding files when Scala 2.12 is no longer supported.
-->
<arg>-Wconf:cat=unused-imports&amp;src=org\/apache\/spark\/graphx\/impl\/VertexPartitionBase.scala:s</arg>
<arg>-Wconf:cat=unused-imports&amp;src=org\/apache\/spark\/graphx\/impl\/VertexPartitionBaseOps.scala:s</arg>
</args>
<jvmArgs>
<jvmArg>-Xss128m</jvmArg>
Expand Down
8 changes: 6 additions & 2 deletions project/SparkBuild.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -257,7 +257,7 @@ object SparkBuild extends PomBuild {
"-Wconf:cat=deprecation:wv,any:e",
// 2.13-specific warning hits to be muted (as narrowly as possible) and addressed separately
// TODO(SPARK-33499): Enable this option when Scala 2.12 is no longer supported.
// "-Wunused:imports",
"-Wunused:imports",
"-Wconf:cat=lint-multiarg-infix:wv",
"-Wconf:cat=other-nullary-override:wv",
"-Wconf:cat=other-match-analysis&site=org.apache.spark.sql.catalyst.catalog.SessionCatalog.lookupFunction.catalogFunction:wv",
Expand All @@ -283,7 +283,11 @@ object SparkBuild extends PomBuild {
// 4. `fruitless type test: a value of TypeA cannot also be a TypeB`
"-Wconf:cat=unchecked&msg=outer reference:s",
"-Wconf:cat=unchecked&msg=eliminated by erasure:s",
"-Wconf:msg=^(?=.*?a value of type)(?=.*?cannot also be).+$:s"
"-Wconf:msg=^(?=.*?a value of type)(?=.*?cannot also be).+$:s",
// TODO(SPARK-43850): Remove the following suppression rules and remove `import scala.language.higherKinds`
// from the corresponding files when Scala 2.12 is no longer supported.
"-Wconf:cat=unused-imports&amp;src=org\\/apache\\/spark\\/graphx\\/impl\\/VertexPartitionBase.scala:s",
"-Wconf:cat=unused-imports&amp;src=org\\/apache\\/spark\\/graphx\\/impl\\/VertexPartitionBaseOps.scala:s"
)
}
}
Expand Down
2 changes: 1 addition & 1 deletion ...atalyst/src/main/scala-2.13/org/apache/spark/sql/catalyst/expressions/ExpressionSet.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

package org.apache.spark.sql.catalyst.expressions

import scala.collection.{mutable, IterableFactory, IterableOps}
Copy link
Contributor Author

@LuciferYang LuciferYang May 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IterableFactory and IterableOps are unused imports

import scala.collection.mutable
import scala.collection.mutable.ArrayBuffer

object ExpressionSet {
Expand Down