[CARBONDATA-4274] Fix create partition table error with spark 3.1
authorShreelekhyaG <shreelu_gampa@yahoo.com>
Thu, 19 Aug 2021 14:56:00 +0000 (20:26 +0530)
committerkunal642 <kunalkapoor642@gmail.com>
Tue, 31 Aug 2021 12:26:49 +0000 (17:56 +0530)
commitca659b5cd3219de7c8a9603784559e20fdff1fda
treececab46c83788f717148c0c9ac8b8446f0d27aa6
parentf52aa20a82b3e9766d34511ac4296b50c5a3ea9e
[CARBONDATA-4274] Fix create partition table error with spark 3.1

Why is this PR needed?
With spark 3.1, we can create a partition table by giving partition
columns from schema.
Like below example:
create table partitionTable(c1 int, c2 int, v1 string, v2 string)
stored as carbondata partitioned by (v2,c2)

When the table is created by SparkSession with CarbonExtension,
catalog table is created with the specified partitions.
But in cluster/ with carbon session, when we create partition
table with above syntax it is creating normal table with no partitions.

What changes were proposed in this PR?
partitionByStructFields is empty when we directly give partition
column names. So it was not creating a partition table. Made
changes to identify the partition column names and get the struct
field and datatype info from table columns.

This closes #4208
integration/spark/src/main/spark3.1/org/apache/spark/sql/SparkVersionAdapter.scala
integration/spark/src/main/spark3.1/org/apache/spark/sql/parser/CarbonSparkSqlParser.scala
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala