Skip to content
Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add scaladoc, rename param
  • Loading branch information
peter-toth committed Apr 21, 2023
commit ecbc4b879a8f1c397d200958b1b7ade0f6768ca5
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,19 @@ case class InlineCTE(alwaysInline: Boolean = false) extends Rule[LogicalPlan] {
cteDef.child.exists(_.expressions.exists(_.isInstanceOf[OuterReference]))
}

/**
* Accumulates all the CTEs from a plan into a special map.
*
* @param plan The plan to collect the CTEs from
* @param cteMap A mutable map that accumulates the CTEs and their reference information by CTE
* ids. The value of the map is tuple whose elements are:
* - The CTE definition
* - The number of incoming references to the CTE. This includes references from
* outer CTEs and regular places.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* outer CTEs and regular places.
* inner CTEs and regular places.

Copy link
Contributor Author

@peter-toth peter-toth Apr 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually wanted to write other CTEs and not inner/outer. E.g. in

WITH (
  cte1 AS (SELECT 1),
  cte2 AS (SELECT * FROM cte1)
)
SELECT * FROM cte1 JOIN cte2 ON ...

the reference count of cte1 is 2. 1 is from an "other" CTE (but not "inner"/"outer") (and 1 is from a "regular place").

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in 806c2de

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cloud-fan, please let me know if anything else is needed here.

* - A mutable inner map that tracks outgoing references (counts) to other CTEs.
* @param outerCTEId While collecting the map we use this optional CTE id to identify the
* current outer CTE.
*/
def buildCTEMap(
plan: LogicalPlan,
cteMap: mutable.Map[Long, (CTERelationDef, Int, mutable.Map[Long, Int])],
Expand Down Expand Up @@ -109,15 +122,22 @@ case class InlineCTE(alwaysInline: Boolean = false) extends Rule[LogicalPlan] {
}
}

/**
* Cleans the CTE map by removing those CTEs that are not referenced at all and corrects those
* CTE's reference counts where the removed CTE referred to.
*
* @param cteMap A mutable map that accumulates the CTEs and their reference information by CTE
* ids. Needs to be sorted to speed up cleaning.
*/
private def cleanCTEMap(
cteRefMap: mutable.SortedMap[Long, (CTERelationDef, Int, mutable.Map[Long, Int])]
cteMap: mutable.SortedMap[Long, (CTERelationDef, Int, mutable.Map[Long, Int])]
) = {
cteRefMap.keys.toSeq.reverse.foreach { currentCTEId =>
val (_, currentRefCount, refMap) = cteRefMap(currentCTEId)
cteMap.keys.toSeq.reverse.foreach { currentCTEId =>
val (_, currentRefCount, refMap) = cteMap(currentCTEId)
if (currentRefCount == 0) {
refMap.foreach { case (referencedCTEId, uselessRefCount) =>
val (cteDef, refCount, refMap) = cteRefMap(referencedCTEId)
cteRefMap(referencedCTEId) = (cteDef, refCount - uselessRefCount, refMap)
val (cteDef, refCount, refMap) = cteMap(referencedCTEId)
cteMap(referencedCTEId) = (cteDef, refCount - uselessRefCount, refMap)
}
}
}
Expand Down