-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-18217] [SQL] Disallow creating permanent views based on temporary views or UDFs #15764
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
509327e
1b430bb
695110f
4dbd3b6
7100a8f
86e7f9d
a4df82b
1c3899f
fec0066
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,14 +19,13 @@ package org.apache.spark.sql.execution.command | |
|
|
||
| import scala.util.control.NonFatal | ||
|
|
||
| import org.apache.spark.sql.{AnalysisException, Dataset, Row, SparkSession} | ||
| import org.apache.spark.sql.{AnalysisException, Row, SparkSession} | ||
| import org.apache.spark.sql.catalyst.{SQLBuilder, TableIdentifier} | ||
| import org.apache.spark.sql.catalyst.catalog.{CatalogStorageFormat, CatalogTable, CatalogTableType} | ||
| import org.apache.spark.sql.catalyst.expressions.Alias | ||
| import org.apache.spark.sql.catalyst.plans.QueryPlan | ||
| import org.apache.spark.sql.catalyst.plans.logical.{LogicalPlan, Project} | ||
| import org.apache.spark.sql.execution.datasources.{DataSource, LogicalRelation} | ||
| import org.apache.spark.sql.types.{MetadataBuilder, StructType} | ||
| import org.apache.spark.sql.catalyst.plans.logical.{LogicalPlan, Project, SubqueryAlias} | ||
| import org.apache.spark.sql.types.MetadataBuilder | ||
|
|
||
|
|
||
| /** | ||
|
|
@@ -131,6 +130,18 @@ case class CreateViewCommand( | |
| s"specified by CREATE VIEW (num: `${userSpecifiedColumns.length}`).") | ||
| } | ||
|
|
||
| // When creating a permanent view, not allowed to reference temporary objects. For example, | ||
| // temporary views. | ||
| // TODO: Disallow creating permanent views based on temporary UDFs | ||
| if (!isTemporary) { | ||
|
||
| analyzedPlan.collectFirst { | ||
| case s: SubqueryAlias if s.isGeneratedByTempTable => | ||
| throw new AnalysisException(s"Not allowed to create a permanent view $name by " + | ||
| s"referencing a temp view `${s.alias}`. " + | ||
| originalText.map(sql => s"""SQL: "$sql".""").getOrElse("")) | ||
|
||
| } | ||
| } | ||
|
|
||
| val aliasedPlan = if (userSpecifiedColumns.isEmpty) { | ||
| analyzedPlan | ||
| } else { | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is there a way to do it without introducing this?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just updated the PR description. This might be the cleanest way. The reason is explained below.
Also cc @cloud-fan and @liancheng Do you have any better solution?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we can do it from the unanalyzed plan, like what this PR did for temp functions.