[SPARK-24937][SQL] Datasource partition table should load empty static partitions

[wangyum]

What changes were proposed in this pull request?

How to reproduce:

spark-sql> CREATE TABLE tbl AS SELECT 1;
spark-sql> CREATE TABLE tbl1 (c1 BIGINT, day STRING, hour STRING)
         > USING parquet
         > PARTITIONED BY (day, hour);
spark-sql> INSERT INTO TABLE tbl1 PARTITION (day = '2018-07-25', hour='01') SELECT * FROM tbl where 1=0;
spark-sql> SHOW PARTITIONS tbl1;
spark-sql> CREATE TABLE tbl2 (c1 BIGINT)
         > PARTITIONED BY (day STRING, hour STRING);
spark-sql> INSERT INTO TABLE tbl2 PARTITION (day = '2018-07-25', hour='01') SELECT * FROM tbl where 1=0;
spark-sql> SHOW PARTITIONS tbl2;
day=2018-07-25/hour=01
spark-sql> 

1. Users will be confused about whether the partition data of tbl1 is generated.
2. Inconsistent with Hive table behavior.

This pr fix this issues.

How was this patch tested?

unit tests

[SparkQA]

Test build #93602 has finished for PR 21883 at commit 90f99fc.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #93633 has finished for PR 21883 at commit d684d38.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #93645 has finished for PR 21883 at commit b44e578.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[wangyum]

retest this please

[SparkQA]

Test build #93652 has finished for PR 21883 at commit b44e578.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[wangyum]

cc @gatorsmile @gengliangwang

[gatorsmile]

TablePartitionSpec is a map. The output of this func might be different from the actual schema, right?

For example, the partition schema is [partCol1, partCol2]. Your output could place partCol2 before partCol1

[wangyum]

You are right. Fixed it.

[SparkQA]

Test build #93829 has finished for PR 21883 at commit 46bfcc6.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[gatorsmile]

@wangyum How about our Hive tables? Add a test case for Hive tables? They are created by CREATE TABLE instead of CREATE TABLE USING. You can use csv/json format in the test cases.

[SparkQA]

Test build #93855 has finished for PR 21883 at commit 536346e.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[wangyum]

retest this please.

[SparkQA]

Test build #93868 has finished for PR 21883 at commit 536346e.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[gengliangwang]

retest this please.

[SparkQA]

Test build #93886 has finished for PR 21883 at commit 536346e.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[gatorsmile]

LGTM

Thanks! Merged to master.