Skip to content

[SPARK-41708][SQL][FOLLOWUP] Add a new replaceAll to SQLQueryTestHelper#replaceNotIncludedMsg to remove @hashCode #39598

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
LuciferYang wants to merge 6 commits into from
Closed

[SPARK-41708][SQL][FOLLOWUP] Add a new replaceAll to SQLQueryTestHelper#replaceNotIncludedMsg to remove @hashCode #39598

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
7bd488b
Override equals and hashCode for TableIdentifier
LuciferYang Jan 16, 2023
104cdac
update golden files
LuciferYang Jan 16, 2023
38be9a0
remove hashCode and revert change
LuciferYang Jan 16, 2023
b0f8c6c
revert change
LuciferYang Jan 16, 2023
f30204e
update golden files
LuciferYang Jan 16, 2023
bfe25b2
remove @hashCode
LuciferYang Jan 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Override equals and hashCode for TableIdentifier
  • Loading branch information
@LuciferYang
LuciferYang committed Jan 16, 2023
commit 7bd488b4d80c70ce51baccc50db1a5f5f22d5d44
7 changes: 7 additions & 0 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/identifiers.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,13 @@ case class TableIdentifier(table: String, database: Option[String], catalog: Opt

def this(table: String) = this(table, None, None)
def this(table: String, database: Option[String]) = this(table, database, None)

override def equals(obj: Any): Boolean = obj match {
Copy link
Contributor Author

@LuciferYang LuciferYang Jan 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @cloud-fan @HyukjinKwon @dongjoon-hyun @ulysses-you

The daily build GA Master Scala 2.13 + Hadoop 3 + JDK 8 failed after #39468 merged.

Are there any suggestions for better solutions? For example, override toString of CatalogFileIndex? What is better to print?

Copy link
Contributor Author

@LuciferYang LuciferYang Jan 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently printing is org.apache.spark.sql.execution.datasources.CatalogFileIndex@hashCode

Copy link
Member

@dongjoon-hyun dongjoon-hyun Jan 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The better solution would be to normalize the output by removing CatalogFileIndex@xxx part.

Copy link
Contributor Author

@LuciferYang LuciferYang Jan 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just print org.apache.spark.sql.execution.datasources.CatalogFileIndex?

Copy link
Contributor

@ulysses-you ulysses-you Jan 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can replace CatalogFileIndex@xxx to CatalogFileIndex in SQLQueryTestHelper#replaceNotIncludedMsg to recover test.

Copy link
Contributor Author

@LuciferYang LuciferYang Jan 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ulysses-you do you mean

spark/sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala

Line 173 in 2062c74

output

change to use SQLQueryTestHelper.replaceNotIncludedMsg(output) ?

Copy link
Contributor

@ulysses-you ulysses-you Jan 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it had already used

case t: TableIdentifier => table == t.table && database == t.database && catalog == t.catalog
case _ => false
}

override def hashCode(): Int = unquotedString.hashCode
}

/** A fully qualified identifier for a table (i.e., database.tableName) */
Expand Down