[CARBONDATA-4194] Fixed presto read after update/delete from spark
authornihal0107 <nihalnitn97@gmail.com>
Fri, 24 Sep 2021 11:31:29 +0000 (17:01 +0530)
committerakashrn5 <akashnilugal@gmail.com>
Thu, 28 Oct 2021 06:44:39 +0000 (12:14 +0530)
commit07b41a5382f554646f231e192cf39c8f28302a05
tree629fa3b2c4add607b2a40743f2c83ab0c3903456
parent7d94691deb3300624ce4b22c4563cb4b9da776fa
[CARBONDATA-4194] Fixed presto read after update/delete from spark

Why is this PR needed?
After update/delete with spark on the table which contains array/struct column,
when we are trying to read from presto then it is throwing class cast exception.
It is because when we perform update/delete then it contains vector of type
ColumnarVectorWrapperDirectWithDeleteDelta which we are trying to typecast to
CarbonColumnVectorImpl and because of this it is throwing typecast exception.
After fixing this(added check for instanceOf) it started throwing IllegalArgumentException.
It is because:

1. In case of local dictionary enable CarbondataPageSource.load is calling
ComplexTypeStreamReader.putComplexObject before setting the correct number
of rows(doesn't subtrat deleted rows). And it throws IllegalArgument while
block building for child elements.
2. position count is wrong in the case of the struct. It should subtract
the number of deleted rows in LocalDictDimensionDataChunkStore.fillVector.
While this is not required to be changed in the case of the array because
datalength of the array already taking care of deleted rows in
ColumnVectorInfo.getUpdatedPageSizeForChildVector.

What changes were proposed in this PR?
First fixed class cast exception after putting instanceOf condition in if block.
Then subtracted the deleted row count before calling ComplexTypeStreamReader.putComplexObject
in DirectCompressCodec.decodeAndFillVector. Also handle deleted rows in case of struct
in LocalDictDimensionDataChunkStore.fillVector

Does this PR introduce any user interface change?
No

Is any new testcase added?
No

This Closes #4224
core/src/main/java/org/apache/carbondata/core/datastore/chunk/store/impl/LocalDictDimensionDataChunkStore.java
core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/compress/DirectCompressCodec.java
core/src/main/java/org/apache/carbondata/core/scan/result/vector/impl/CarbonColumnVectorImpl.java