Skip to content
Closed
Prev Previous commit
Next Next commit
update
  • Loading branch information
panbingkun committed Jun 25, 2024
commit 960f32a110704028981bcdcf0e09546c926e6d1e
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.spark.sql.execution.command

import org.apache.spark.sql.{Row, SparkSession}
import org.apache.spark.sql.catalyst.analysis.{ResolveCatalogs, ResolvedIdentifier}
import org.apache.spark.sql.catalyst.plans.logical.LogicalPlan
import org.apache.spark.sql.connector.catalog.{NamespaceChange, SupportsNamespaces}
import org.apache.spark.sql.errors.QueryCompilationErrors

/**
* A command that ALTER NAMESPACE UNSET PROPERTIES command.
Copy link
Contributor

@cloud-fan cloud-fan Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please fix the grammar.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

*
* The syntax of this command is:
* {{{
* ALTER (DATABASE|SCHEMA|NAMESPACE) ... UNSET (DBPROPERTIES|PROPERTIES) [IF EXISTS] ...;
* }}}
*/
case class UnsetNamespacePropertiesCommand(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to define UnsetNamespacePropertiesCommand in module sql/core because it inherits UnaryRunnableCommand extends RunnableCommand and cannot be accessed in the module sql/catalyst (sql/core depends on sql/catalyst)

ident: LogicalPlan,
propKeys: Seq[String],
ifExists: Boolean) extends UnaryRunnableCommand {

override def run(sparkSession: SparkSession): Seq[Row] = {
val ResolvedIdentifier(catalog, ident) = child
val ns = ResolveCatalogs.resolveNamespace(
catalog, ident.namespace.toSeq :+ ident.name, fetchMetadata = true)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, we should leave this to the analyzer. We should put UnresolvedNamespace inside UnsetNamespacePropertiesCommand, instead of UnresolvedIdentifier

Copy link
Contributor Author

@panbingkun panbingkun Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, let me fix it.

Also, I found an issue during the migration of Unify v1 and v2 ALTER TABLE .. UNSET TBLPROPERTIES IF EXISTS tests.
The command (AlterTableExec) for ALTER TABLE ... UNSET TBLPROPERTIES ... in v2 seems to have ignored whether the IF EXISTS parameter.
I think we should fix it. WDYH?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, we should leave this to the analyzer. We should put UnresolvedNamespace inside UnsetNamespacePropertiesCommand, instead of UnresolvedIdentifier

Thanks, updated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should fix it. WDYH?

Yea we should fix, but in a separated PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, let me do it in another separate PR.

if (!ifExists) {
val properties = ns.metadata
val nonexistentKeys = propKeys.filter(key => !properties.contains(key))
if (nonexistentKeys.nonEmpty) {
throw QueryCompilationErrors.unsetNonExistentPropertiesError(
nonexistentKeys, ns.namespace)
}
}
val changes = propKeys.map {
NamespaceChange.removeProperty
}
catalog match {
case supportsNS: SupportsNamespaces =>
supportsNS.alterNamespace(ns.namespace.toArray, changes: _*)
case _ =>
QueryCompilationErrors.catalogOperationNotSupported(
catalog, "UNSET (DBPROPERTIES|PROPERTIES)")
}

Seq.empty
}

override def child: LogicalPlan = ident

override protected def withNewChildInternal(
newChild: LogicalPlan): UnsetNamespacePropertiesCommand =
copy(ident = newChild)
}