Skip to content
Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
fix
  • Loading branch information
gatorsmile committed Aug 2, 2018
commit a5762d76cbb12d3da0fd4721cea90456bea2a3ef
Original file line number Diff line number Diff line change
Expand Up @@ -924,23 +924,3 @@ case class Deduplicate(

override def output: Seq[Attribute] = child.output
}

/**
* A logical plan for setting a barrier of analysis.
*
* The SQL Analyzer goes through a whole query plan even most part of it is analyzed. This
* increases the time spent on query analysis for long pipelines in ML, especially.
*
* This logical plan wraps an analyzed logical plan to prevent it from analysis again. The barrier
* is applied to the analyzed logical plan in Dataset. It won't change the output of wrapped
* logical plan and just acts as a wrapper to hide it from analyzer. New operations on the dataset
* will be put on the barrier, so only the new nodes created will be analyzed.
*
* This analysis barrier will be removed at the end of analysis stage.
*/
case class AnalysisBarrier(child: LogicalPlan) extends LeafNode {
override protected def innerChildren: Seq[LogicalPlan] = Seq(child)
override def output: Seq[Attribute] = child.output
override def isStreaming: Boolean = child.isStreaming
override def doCanonicalize(): LogicalPlan = child.canonicalized
}