[SPARK-21652][SQL] Fix rule confliction between InferFiltersFromConstraints and ConstantPropagation

[jiangxb1987]

What changes were proposed in this pull request?

For the given example below, the predicate added by InferFiltersFromConstraints is folded by ConstantPropagation later, this leads to unconverged optimize iteration:

Seq((1, 1)).toDF("col1", "col2").createOrReplaceTempView("t1")
Seq(1, 2).toDF("col").createOrReplaceTempView("t2")
sql("SELECT * FROM t1, t2 WHERE t1.col1 = 1 AND 1 = t1.col2 AND t1.col1 = t2.col AND t1.col2 = t2.col")

We can fix this by adjusting the indent of the optimize rules.

How was this patch tested?

Add test case that would have failed in SQLQuerySuite.

[jiangxb1987]

cc @gatorsmile

[SparkQA]

Test build #81299 has finished for PR 19099 at commit 7a364a1.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[maropu]

Why you changed this?

[jiangxb1987]

This is not ideal, but it should resolve the specific failure case and could unblock other PR.

@gatorsmile is working on an alter approach that should perfectly resolve the entire issue of add/remove filters, but it takes time and I don't think it will be ready in a few days.

[gatorsmile]

Basically, this is to infer the filters at the beginning of this huge batch, and then remove the useless at the end. When it involves multiple rules, we might still hit the max iteration. Eventually, we need a clean fix to split the whole batch to two separate ones. One is to add the filters/constraints; another is to remove the filters/constraints based on the costs.

[maropu]

aha, thanks for the explanation~

[maropu]

Just catching the exception is not enough for this test?

[SparkQA]

Test build #81322 has finished for PR 19099 at commit 9b1c642.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #81323 has finished for PR 19099 at commit 231ca3f.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jiangxb1987]

retest this please

[SparkQA]

Test build #81418 has finished for PR 19099 at commit 231ca3f.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[gatorsmile]

LGTM

[gatorsmile]

Thanks! Merging to master.