Andrew Lamb [Fri, 29 Apr 2022 17:54:38 +0000 (13:54 -0400)] 
Prepare for arrow-rs 13.0.0 release (#1624)
* Update version to 13.0.0
* Draft changelog
* updates
* moar
* cleanup
* remove duplicated entry, update for #1589
* Final updates
Atef Sawaed [Fri, 29 Apr 2022 17:49:43 +0000 (20:49 +0300)] 
Fix decimals min max statistics (#1621)
* Fix incorrect writing of min/max statistics
* Refactor
* Decimals Byte array comparison
* Add Decimals test
* Use slice instead of vector
* Fix build error
* Fix build error
* Coding Style
* More tests
* Refactor
* Improve code readability
Co-authored-by: Atef Sawaed <atefsawaed@microsoft.com>
Remzi Yang [Fri, 29 Apr 2022 17:22:35 +0000 (01:22 +0800)] 
use standard library (#1629)
Signed-off-by: remzi <13716567376yh@gmail.com>
Andrew Lamb [Thu, 28 Apr 2022 18:51:01 +0000 (14:51 -0400)]
Clarify docs on UnionBuilder::append_null (#1628)
Raphael Taylor-Davies [Thu, 28 Apr 2022 17:39:37 +0000 (18:39 +0100)] 
Fix Null Mask Handling in ArrayData And UnionArray (#1589)
* Fix ListArray and StructArray equality (#626)
* Simplify null masking in equality comparisons
Various UnionArray fixes (#1598) (#1596) (#1591) (#1590)
Fix handling of null masks in ArrayData equality (#1599)
* Miscellaneous fixes
* Fix structure null equality
* Review feedback
dependabot[bot] [Wed, 27 Apr 2022 18:59:07 +0000 (14:59 -0400)] 
Update flatbuffers requirement from =2.1.1 to =2.1.2 (#1622)
Updates the requirements on [flatbuffers](https://github.com/google/flatbuffers) to permit the latest version.
- [Release notes](https://github.com/google/flatbuffers/releases)
- [Commits](https://github.com/google/flatbuffers/commits)
---
updated-dependencies:
- dependency-name: flatbuffers
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Remzi Yang [Mon, 25 Apr 2022 20:29:20 +0000 (04:29 +0800)]
Add `substring` support for binary (#1608)
* add substring for binary
fix some test for string array
Signed-off-by: remzi <13716567376yh@gmail.com>
* fix another bug in test
Signed-off-by: remzi <13716567376yh@gmail.com>
* update doc
Signed-off-by: remzi <13716567376yh@gmail.com>
* add with_nulls tests
Signed-off-by: remzi <13716567376yh@gmail.com>
* add without_nulls tests
Signed-off-by: remzi <13716567376yh@gmail.com>
* fix clippy
Signed-off-by: remzi <13716567376yh@gmail.com>
Jörn Horstmann [Mon, 25 Apr 2022 20:27:30 +0000 (22:27 +0200)] 
Don't access and validate offset buffer in ListArray::from(ArrayData) (#1602)
* Don't access and validate offset buffer in ListArray::from(ArrayData)
* Simplify empty buffer creation
* fix clippy
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Andrew Lamb [Mon, 25 Apr 2022 11:18:21 +0000 (07:18 -0400)]
Add example readme (#1615)
* Add example readme
* RAT
* prettier
* Review comments
Andrew Lamb [Mon, 25 Apr 2022 10:24:27 +0000 (06:24 -0400)]
Improve docs and examples links on main readme (#1614)
* Improve docs and examples links on main readme
* markdown
* improve wording
Liang-Chi Hsieh [Mon, 25 Apr 2022 06:15:57 +0000 (23:15 -0700)] 
Read/Write nested dictionaries under FixedSizeList in IPC (#1610)
* Read/Write nested dictionaries under FixedSizeList in IPC
* Fix clippy
tfeda [Sun, 24 Apr 2022 05:42:45 +0000 (01:42 -0400)] 
Update datatypes in parquet::basic::LogicalType (#1612)
* Convert LogicalType::STRING(StringType) to LogicalType::String
* Covert LogicalType::MAP(MapType) to LogicalType::Map
* Convert LogicalType::LIST(ListType) to LogicalType::List
* Converted LogicalType::ENUM(EnumType) to LogicalType::Enum
* Convert LogicalType::DECIMAL(DecimalType) to LogicalType::Decimal { scale, precision }, Fix String proc macro error
* Convert LogicalType::DATE(DateType) to LogicalType::Date
* Convert LogicalType::TIME(TimeType) to LogicalType::Time { is_adjusted_to_u_t_c: bool, unit: TimeUnit }
* Convert LogicalType::TIMESTAMP(TimestampType) to LogicalType::Timestamp { is_adjusted_to_u_t_c: bool, unit: TimeUnit }
* Convert LogicalType::INTEGER(IntType) to LogicalType::Integer { bit_width: i8, is_signed: bool }
* Convert LogicalType::UNKNOWN,JSON,BSON to LogicalType::Unknown,Json,Bson
* Convert LogicalType::UUID to LogicalType::Uuid
* Add ref t to simplify printing in src/arrow/schema from_int32()
Chao Sun [Sat, 23 Apr 2022 06:42:01 +0000 (23:42 -0700)] 
Parquet: schema validation should allow scale == precision for decimal type (#1607)
Liang-Chi Hsieh [Thu, 21 Apr 2022 08:22:37 +0000 (01:22 -0700)]
Fix map nullable flag in `ParquetTypeConverter` (#1592)
* Fix map nullable flag
* Fix tests
* Add another map case
Liang-Chi Hsieh [Wed, 20 Apr 2022 18:58:55 +0000 (11:58 -0700)]
Read/write nested dictionary under map in ipc stream reader/writer (#1583)
* IPC read/write nested dict in map
* For review comments
* update comment
Liang-Chi Hsieh [Wed, 20 Apr 2022 07:25:18 +0000 (00:25 -0700)]
Read/write nested dictionary under large list in ipc stream reader/writer (#1585)
* Read/Write nested dictionaries under LargeList in IPC
* For review comments
Remzi Yang [Tue, 19 Apr 2022 16:26:34 +0000 (00:26 +0800)]
Add utf-8 validation checking for `substring` (#1577)
* add utf-8 validation checking
update doc
add a test for invalid array type
Signed-off-by: remzi <13716567376yh@gmail.com>
* add tests
clean up
Signed-off-by: remzi <13716567376yh@gmail.com>
* test the worst case
Signed-off-by: remzi <13716567376yh@gmail.com>
* update doc and tests
Signed-off-by: remzi <13716567376yh@gmail.com>
* update doc
Signed-off-by: remzi <13716567376yh@gmail.com>
* use std method is_char_boundary
update doc
Signed-off-by: remzi <13716567376yh@gmail.com>
* add 2 substring benches
Signed-off-by: remzi <13716567376yh@gmail.com>
* replace dyn Fn with loop unswitching
Signed-off-by: remzi <13716567376yh@gmail.com>
* Update arrow/src/compute/kernels/substring.rs
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Andrew Lamb [Mon, 18 Apr 2022 05:51:18 +0000 (01:51 -0400)]
Derive `Clone` and `PartialEq` for json `DecoderOptions` (#1581)
* Implement Clone and PartialEq for DecoderOptions
* fmt
DuRipeng [Mon, 18 Apr 2022 05:50:24 +0000 (13:50 +0800)] 
Support casting to/from `DataType::Null` in `cast` kernel (#1572)
* cast null from and to others
* fmt fix
* add more ut
Co-authored-by: duripeng <duripeng@baidu.com>
Raphael Taylor-Davies [Sun, 17 Apr 2022 10:52:55 +0000 (11:52 +0100)]
Remove reference indirection for Copy types in substring (#1576)
tfeda [Sun, 17 Apr 2022 08:37:56 +0000 (04:37 -0400)]
Replace &Option<T> with Option<&T> (#1571)
* Convert &Option<T> arguments to Option<&T>
* fix cargo fmt issues
* Update FileMetaData created_by() to return Option<&str>, Fix integration test
Liang-Chi Hsieh [Sat, 16 Apr 2022 17:23:53 +0000 (10:23 -0700)]
Use littleendian arrow files for projection_should_work (#1573)
Andrew Lamb [Fri, 15 Apr 2022 22:46:09 +0000 (18:46 -0400)]
Prepare for 12.0.0 release: Update version and CHANGELOG (#1569)
* Update version to 12.0.0
* Update changelog script
* Update changelog for 12.0.0
Raphael Taylor-Davies [Fri, 15 Apr 2022 14:09:25 +0000 (15:09 +0100)]
Split out ListArrayReader into separate module (#1483) (#1563)
* Split out ListArrayReader into separate module (#1483)
* Fix merge conflict
Raphael Taylor-Davies [Fri, 15 Apr 2022 14:09:12 +0000 (15:09 +0100)]
fix infinite loop in not fully packed bit-packed runs (#1555)
* fix infinite loop in not fully packed bit-packed runs
* Add test and also fix get_batch_with_dict
Co-authored-by: Andrei Liakhovich <anliakho@microsoft.com>
Andrew Lamb [Fri, 15 Apr 2022 13:46:50 +0000 (09:46 -0400)]
Add test for creating FixedSizeBinaryArray::try_from_sparse_iter failed when given all Nones (#1551)
* Add test for creating FixedSizeBinaryArray::try_from_sparse_iter failed when given all Nones
* fix test
Liang-Chi Hsieh [Fri, 15 Apr 2022 13:05:23 +0000 (06:05 -0700)]
Read/write nested dictionary in ipc stream reader/writer (#1566)
* Read dictionary inside dictionary
* Fix clippy
Chao Sun [Fri, 15 Apr 2022 12:52:03 +0000 (05:52 -0700)]
initial commit (#1564)
Raphael Taylor-Davies [Fri, 15 Apr 2022 12:50:28 +0000 (13:50 +0100)]
Split out MapArray into separate module (#1483) (#1562)
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Raphael Taylor-Davies [Fri, 15 Apr 2022 12:43:42 +0000 (13:43 +0100)]
Support empty projection in ParquetRecordBatchReader (#1560)
* Support empty projection in ParquetRecordBatchReader
* Fix async reader
* Fix RAT
Liang-Chi Hsieh [Thu, 14 Apr 2022 18:42:29 +0000 (11:42 -0700)]
Fix incorrect `into_buffers` for UnionArray (#1567)
* Fix incorrect buffers for UnionArray
* Add test
* Re-enable test_filter_union_array_sparse
Andrew Lamb [Thu, 14 Apr 2022 17:30:16 +0000 (13:30 -0400)]
Add CI check for full validation mode (#1546)
* Add force_validate feature
* Disable some redundant checks
* Add issue link
* Add test with force_validate feature flag
* fix up message
* disable due to https://github.com/apache/arrow-rs/issues/1547
* disable ipc test failure
* fix clippy
* Fix doctest to pass with force_validate enabled
Raphael Taylor-Davies [Thu, 14 Apr 2022 14:45:15 +0000 (15:45 +0100)]
Add option to skip decoding arrow metadata from parquet (#1459) (#1558)
* Add option to skip decoding arrow metadata from parquet (#1459)
Fix inference from null logical type (#1557)
Replace some `&Option<T>` with `Option<&T>` (#1556)
* Update parquet/src/arrow/arrow_reader.rs
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* Fmt
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Andrew Lamb [Wed, 13 Apr 2022 16:46:53 +0000 (12:46 -0400)]
Improve JSON reader documentation (#1559)
* Improve JSON reader documentation
* Apply suggestions from code review
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Andrew Lamb [Wed, 13 Apr 2022 12:51:15 +0000 (08:51 -0400)]
Conslidate JSON reader options (#1539)
Thomas Peiselt [Wed, 13 Apr 2022 11:28:32 +0000 (13:28 +0200)] 
Fix reading dictionaries from nested structs in ipc `StreamReader` (#1550)
* Fix reading dictionaries from nested structs in ipc `StreamReader`
* Fix clippy error
* Apply review comment about field naming in test
Raphael Taylor-Davies [Tue, 12 Apr 2022 23:44:22 +0000 (00:44 +0100)]
Create RecordBatch With Non-Zero Row Count But No Columns (#1536) (#1552)
* Support empty RecordBatch (#1536)
* Placate clippy
* Review feedback
* Fix doc
* Fix create_record_batch_slice_empty_batch test
Sumit [Tue, 12 Apr 2022 13:39:05 +0000 (15:39 +0200)] 
Allow json reader/decoder to work with format_strings for each field (#1451)
* implement parser for remaining types used by json decoder
* added format strings (hashmap) to json reader
the format_string map's key is column name.
The value will be used to parse the date64/date32 types from json
if the read value is of string type
add tests for formatted parser for date{32,64}type for json readers
all-parsers start
fixup! added format strings (hashmap) to json reader
* add DecoderOptions struct for holding options for decoder
that way later extensions to the decoder can be added to this struct
without breaking API.
* Fixup some comments
* added test for string parsing json reader for time{32,64} types
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Yijie Shen [Tue, 12 Apr 2022 13:34:18 +0000 (21:34 +0800)] 
minor: enable Date32/64 to String/LargeString cast (#1534)
Remzi Yang [Sun, 10 Apr 2022 11:15:27 +0000 (19:15 +0800)]
update the doc of `substring` (#1529)
* update doc
Signed-off-by: remzi <13716567376yh@gmail.com>
* update doc
Signed-off-by: remzi <13716567376yh@gmail.com>
* Update arrow/src/compute/kernels/substring.rs
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Andrew Lamb [Fri, 8 Apr 2022 10:19:56 +0000 (06:19 -0400)]
fix clippy errors in 1.60 (#1527)
Liang-Chi Hsieh [Thu, 7 Apr 2022 21:07:45 +0000 (14:07 -0700)]
Add `new_from_strings` to create `MapArrays` (#1507)
* Add new_from_strings
* Fix clippy
* Update arrow/src/array/array_map.rs
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* Fix typo too
* For review
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Liang-Chi Hsieh [Thu, 7 Apr 2022 21:06:39 +0000 (14:06 -0700)]
Fix reading nested lists from parquet files (#1517)
* Fix
* Add test
Jörn Horstmann [Thu, 7 Apr 2022 21:01:21 +0000 (23:01 +0200)]
Decouple buffer deallocation from ffi and allow creating buffers from rust vec (#1494)
* Decouple buffer deallocation from ffi and allow zero-copy buffer creation from rust vectors or strings
* Move allocation owner to alloc module
* Rename and comment Deallocation variants
* Fix doc link
* Explicitly assert that Buffer is UnwindSafe
* fix: doc comment
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Remzi Yang [Thu, 7 Apr 2022 20:31:30 +0000 (04:31 +0800)]
Speed up the `substring` kernel by about 2x (#1512)
* speed up substring
Signed-off-by: remzi <13716567376yh@gmail.com>
* add comments
Signed-off-by: remzi <13716567376yh@gmail.com>
* reformat code
Signed-off-by: remzi <13716567376yh@gmail.com>
* use trait opject to simplify the code
Signed-off-by: remzi <13716567376yh@gmail.com>
* reformat code
Signed-off-by: remzi <13716567376yh@gmail.com>
* fmt code
Signed-off-by: remzi <13716567376yh@gmail.com>
Andrew Lamb [Thu, 7 Apr 2022 19:30:46 +0000 (15:30 -0400)]
chore: Update `prost`, `prost-derive` and `prost-types` to 0.10, `tonic`, and `tonic-build` to `0.7` (#1510)
* chore: Update prost, prost-derive and prost-types to 0.10
* Update tonic requirement from 0.6 to 0.7
Updates the requirements on [tonic](https://github.com/hyperium/tonic) to permit the latest version.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.6.0...v0.7.0)
---
updated-dependencies:
- dependency-name: tonic
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update tonic-build requirement from 0.6 to 0.7
Updates the requirements on [tonic-build](https://github.com/hyperium/tonic) to permit the latest version.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.6.0...v0.7.0)
---
updated-dependencies:
- dependency-name: tonic-build
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update generated code
* Try installing cmake dependencies for flight
* install cmake and protobuf
* Use --experimental_allow_proto3_optional flag
* fix apt-install
* try to install just protobuf compiler
* Add action to configure workspace
* Use prost enabled toolchain
* fixes
* fixups
* fix clippy
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sven Cattell [Wed, 6 Apr 2022 17:39:50 +0000 (13:39 -0400)] 
Fix for missing documentation of `GenericListBuilder` (#1525)
* Should fix issue #1518
* Lint change.
Andrew Lamb [Tue, 5 Apr 2022 19:55:56 +0000 (15:55 -0400)]
Add a diagram to `take` kernel documentation (#1524)
Andrew Lamb [Tue, 5 Apr 2022 13:39:56 +0000 (09:39 -0400)]
Mark remove-old-releases.sh executable (#1522)
Remzi Yang [Tue, 5 Apr 2022 13:32:52 +0000 (21:32 +0800)]
Delete duplicate code in the `sort` kernel (#1519)
* remove repeated code
Signed-off-by: remzi <13716567376yh@gmail.com>
* delete weird file
Signed-off-by: remzi <13716567376yh@gmail.com>
Andrew Lamb [Fri, 1 Apr 2022 15:23:38 +0000 (11:23 -0400)]
Prepare for release version `11.1.0` (#1514)
* Update release version to 11.1.0
* draft: changelog
* more
* update
* Fixup
Liang-Chi Hsieh [Thu, 31 Mar 2022 18:20:28 +0000 (11:20 -0700)]
Implement ArrayEqual for UnionArray (#1469)
* init
* more
* Remove dense/sparse case
* Fix clippy
* For review
* For review
Liang-Chi Hsieh [Thu, 31 Mar 2022 01:07:06 +0000 (18:07 -0700)]
Add FFI for Arrow C Stream Interface (#1384)
* Add FFI for Arrow C Stream Interface
* Add ArrowArrayStreamReader
* Add test
* Fix clippy
* fix format
* define error code
* Regenerate ffi binding using bindgen
* Rewrite test
* Remove CStreamInterface
* Fix clippy error
* Fix more clippy errors
* For review comment.
* Fix clippy error
* Fix clippy error
* not run example code in comment
* ignore doctest
* For review
* Fix clippy
* For review comment
* For review
* Add export_reader_into_raw
* For review
Andrew Lamb [Wed, 30 Mar 2022 17:49:12 +0000 (13:49 -0400)]
Clarify docs that SlicesIterator ignores null values (#1504)
* Clarify docs that SlicesIterator ignores null values
* Update arrow/src/compute/kernels/filter.rs
Co-authored-by: Yijie Shen <henry.yijieshen@gmail.com>
Co-authored-by: Yijie Shen <henry.yijieshen@gmail.com>
Andrew Lamb [Wed, 30 Mar 2022 17:48:33 +0000 (13:48 -0400)]
Update release scripts to automatically clean up old release versions (#1467)
* Automatically clean up old release versions
* Update dev/release/release-tarball.sh
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
* Add message to delete command
* fix submodules
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Remzi Yang [Wed, 30 Mar 2022 17:47:29 +0000 (01:47 +0800)]
Support calculating number of chars for `StringArray` (#1503)
* add functions, no tests yet
Signed-off-by: remzi <13716567376yh@gmail.com>
* add tests
delete unchecked fn
update doc
Signed-off-by: remzi <13716567376yh@gmail.com>
* use lib method
update doc and test
Signed-off-by: remzi <13716567376yh@gmail.com>
Andrew Lamb [Wed, 30 Mar 2022 17:46:20 +0000 (13:46 -0400)]
Implement `size_hint` and `ExactSizedIterator` for `DecimalArray` (#1506)
* Implement size_hint and ExactSizedIterator for DecimalArray
* clippy
* Apply suggestions from code review
Co-authored-by: Liang-Chi Hsieh <viirya@gmail.com>
* Move DecimalIter to iterator.rs
* Bring back doc fixes
Co-authored-by: Liang-Chi Hsieh <viirya@gmail.com>
Remzi Yang [Tue, 29 Mar 2022 18:59:40 +0000 (02:59 +0800)]
update doc (#1491)
Signed-off-by: remzi <13716567376yh@gmail.com>
Liang-Chi Hsieh [Tue, 29 Mar 2022 18:59:00 +0000 (11:59 -0700)]
Use Arrow take kernel within ListArrayReader (#1490)
* Remove remove_indices
* For review
Jörn Horstmann [Tue, 29 Mar 2022 16:48:37 +0000 (18:48 +0200)]
Fix miri error in try_from_trusted_len_iter (#1497)
Remzi Yang [Mon, 28 Mar 2022 20:47:45 +0000 (04:47 +0800)]
Add `length` kernel support for List Array (#1488)
* add fn for list length
code format
Signed-off-by: remzi <13716567376yh@gmail.com>
* add list support into length function
Signed-off-by: remzi <13716567376yh@gmail.com>
* add tests
Signed-off-by: remzi <13716567376yh@gmail.com>
* update doc
Signed-off-by: remzi <13716567376yh@gmail.com>
Yijie Shen [Mon, 28 Mar 2022 20:38:58 +0000 (04:38 +0800)]
Support sort for decimal data type (#1487)
Liang-Chi Hsieh [Sun, 27 Mar 2022 10:46:56 +0000 (03:46 -0700)]
Fix generate_non_canonical_map_case, fix `MapArray` equality (#1476)
* Revamp list_equal for map type
* Canonicalize schema
* Add nullability and metadata
Raphael Taylor-Davies [Fri, 25 Mar 2022 16:43:52 +0000 (16:43 +0000)]
Fix reading/writing nested null arrays (#1480) (#1036) (#1399) (#1481)
Raphael Taylor-Davies [Fri, 25 Mar 2022 12:55:36 +0000 (12:55 +0000)]
Split ArrayReaderBuilder into its own module (#1483) (#1485)
* Split ArrayReaderBuilder into its own module (#1483)
* Add license header
Remzi Yang [Thu, 24 Mar 2022 18:32:34 +0000 (02:32 +0800)]
Support the `length` kernel on Binary Array (#1465)
* support length on binary array (not test)
rewrite unary_offset using macro
Signed-off-by: remzi <13716567376yh@gmail.com>
* add tests
Signed-off-by: remzi <13716567376yh@gmail.com>
* add non-utf8 test cases
Signed-off-by: remzi <13716567376yh@gmail.com>
* fix some doc
Signed-off-by: remzi <13716567376yh@gmail.com>
* update doc
simplify the way to get offsets. No performance penalty
Signed-off-by: remzi <13716567376yh@gmail.com>
Remzi Yang [Wed, 23 Mar 2022 20:53:16 +0000 (04:53 +0800)]
fix doc (#1471)
Signed-off-by: remzi <13716567376yh@gmail.com>
Liang-Chi Hsieh [Tue, 22 Mar 2022 11:11:10 +0000 (04:11 -0700)]
Fix doc (#1463)
Liang-Chi Hsieh [Tue, 22 Mar 2022 11:07:08 +0000 (04:07 -0700)]
Fix generate_map_case (#1457)
jakevin [Tue, 22 Mar 2022 11:06:25 +0000 (19:06 +0800)] 
Improve performance of DictionaryArray::try_new() (#1435)
* improve `DictionaryArray::try_new()` #1313
* *: fix typo
* *: add cheap validate and unit test
* *: polish the error
* Add safety note
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Liang-Chi Hsieh [Tue, 22 Mar 2022 10:09:08 +0000 (03:09 -0700)]
Fix Parquet reader for null list (#1448)
* Fix Parquet reader for null list
* Test on forked parquet-testing
* For review comments
* Fix clippy
Liang-Chi Hsieh [Sat, 19 Mar 2022 06:26:50 +0000 (23:26 -0700)]
Remove Clone and copy source structs internally (#1449)
* Remove Clone and copy source structs internally
* Remove drop_in_place and add more comment
* Add export_into_raw
* Fix format
* Fix clippy
* Move to export_array_into_raw
* Fix clippy
* Fix doc
* Use write_unaligned
Andrew Lamb [Fri, 18 Mar 2022 07:46:56 +0000 (03:46 -0400)]
Prepare for 11.0.0 release (#1461)
* Update version to 11.0.0
* Update changelog
* update changelog
* fixup
* tweak
Liang-Chi Hsieh [Thu, 17 Mar 2022 12:45:56 +0000 (05:45 -0700)]
Fix generate_interval_case in integration test (#1446)
* Fix generate_interval_case
* Fix
Remzi Yang [Thu, 17 Mar 2022 12:43:10 +0000 (20:43 +0800)]
rewrite doc (#1450)
Signed-off-by: remzi <13716567376yh@gmail.com>
jakevin [Thu, 17 Mar 2022 07:35:28 +0000 (15:35 +0800)]
enhancement: remove redundant if/clamp_min/abs (#1428)
Kyle Barron [Wed, 16 Mar 2022 12:23:01 +0000 (06:23 -0600)] 
Set `default-features = false` for `zstd` in the parquet crate to support `wasm32-unknown-unknown` (#1414)
* Update zstd version for wasm support
* Bump to 0.11.1
Liang-Chi Hsieh [Wed, 16 Mar 2022 12:22:49 +0000 (05:22 -0700)]
`filter` kernel should work with FixedSizeListArrays (#1434)
* filter kernel should work with FixedSizeListArrays
* Fix clippy
* Fix clippy
Liang-Chi Hsieh [Wed, 16 Mar 2022 07:40:44 +0000 (00:40 -0700)]
filter kernel should work with UnionArray (#1412)
Remzi Yang [Wed, 16 Mar 2022 06:45:33 +0000 (14:45 +0800)]
Rewrite doc example of ListArray and LargeListArray (#1447)
Liang-Chi Hsieh [Mon, 14 Mar 2022 12:12:14 +0000 (05:12 -0700)]
Fix generate_decimal128_case (#1440)
Liang-Chi Hsieh [Mon, 14 Mar 2022 07:11:02 +0000 (00:11 -0700)]
Fix integration doc (#1438)
Raphael Taylor-Davies [Mon, 14 Mar 2022 07:05:54 +0000 (07:05 +0000)]
Fix DeltaBitPack MiniBlock Bit Width Padding (#1418)
* Consistent DeltaBitPackEncoder bit width padding (#1416)
Ignore non-zero padded bit widths in DeltaBitPackDecoder (#1417)
* chore: review feedback
* Add test of DeltaBitPackDecoder padding
* Revert formatting
Remzi Yang [Sat, 12 Mar 2022 17:51:34 +0000 (01:51 +0800)]
Add doc example for creating `FixedSizeListArray` (#1426)
Jörn Horstmann [Fri, 11 Mar 2022 20:02:05 +0000 (21:02 +0100)]
Support nullable keys in DictionaryArray::try_new (#1430)
* Support nullable keys in DictionaryArray::try_new
* Set null count so it does not have to be recalculated
Jörn Horstmann [Fri, 11 Mar 2022 19:22:18 +0000 (20:22 +0100)]
Fix possibly unaligned writes in MutableBuffer (#1421)
* Fix possibly unaligned writes in MutableBuffer
* Remove debug output and make from_trusted_len_iter follow the same pattern
* Add comment in extend_from_slice
jakevin [Fri, 11 Mar 2022 18:31:08 +0000 (02:31 +0800)]
Add value_unchecked() for FixedSizeBinaryArray (#1420)
dependabot[bot] [Fri, 11 Mar 2022 12:19:41 +0000 (07:19 -0500)]
Update zstd requirement from 0.10 to 0.11 (#1415)
Updates the requirements on [zstd](https://github.com/gyscos/zstd-rs) to permit the latest version.
- [Release notes](https://github.com/gyscos/zstd-rs/releases)
- [Commits](https://github.com/gyscos/zstd-rs/compare/v0.10.0...v0.11.0)
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Wang Fenjin [Fri, 11 Mar 2022 11:19:18 +0000 (19:19 +0800)] 
Implement basic FlightSQL Server (#1386)
* init impl flight sql server mod
Change-Id: I108b2468b078470bb8b6f95c031035cc09227986
* update according to comments
Change-Id: Ibb381e105041b38e6402850a2338403f802568ec
* fix ci error
Change-Id: I9485e510f1a960b6e094e559c3679434f8474ec1
* format code
Change-Id: I7ef4ade3acc81ccf5df088c866d41b538cf6f4f2
* fix clippy issue
Change-Id: I35d108ef43f2c2245444cfd5ea82da00b4f694f9
* add more test
Change-Id: Ic159cea2c76b017e183d2946e2d24e6fd1f9b4c1
* improve error handling
Change-Id: I709c16613092fd42ccff827eed3e3ad3f28368e2
* delete unnecessary Sync
Change-Id: I03ed0f69ddb1203ecd75982815fa72eca4d81160
* add flight_sql_server example
Change-Id: Ia35d697aaac3c72feba9c3aaf380ee3930484c48
* get rid of type annotation in unpack
Change-Id: I6006702d424ac6595f58c66057df267c4fd24476
* fix comments
Change-Id: I740d3d4e5aabbb56219291381e6a6db6506eca28
* add feature flight-sql
Change-Id: I223cf76be10ff379fcc9000c730d99c9773c7c3d
* delete all-features flag as packed_simd_2 no supported
Change-Id: I50915b85b2f806bac5cd3207623e3f4e0e1974a1
* add feature flag for example
Change-Id: I562efcfa89a606b8061d2715ca1b6775e2a952a9
* fix do_put and do_action API
Change-Id: I80bef8c2b0a713a87c43487708ae721f5f8f9da9
* format code
Change-Id: Ie664a5fca965759dbba59ad9e34fc6e33150ddbf
* rename feature
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* rename to flight-sql-experimental
Change-Id: I4de4fe3768b0316e69ba6798406310632933d25d
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Remzi Yang [Fri, 11 Mar 2022 02:40:15 +0000 (10:40 +0800)]
Remove duplicate bound check in the function shift (#1409)
Signed-off-by: remzi <13716567376yh@gmail.com>
Liang-Chi Hsieh [Fri, 11 Mar 2022 02:37:06 +0000 (18:37 -0800)]
Directly write to MutableBuffer in substring (#1423)
Daniël Heres [Wed, 9 Mar 2022 17:40:53 +0000 (18:40 +0100)] 
Implement projection for arrow file / streams (#1339)
* Implement projection for arrow file / streams
* Tests
* Fix
* Fix
* Add test
* Add test
* Add link
* Undo change to existing test
* Update arrow/src/ipc/reader.rs
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* Use project
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Chao Sun [Wed, 9 Mar 2022 07:27:43 +0000 (23:27 -0800)]
Add dictionary support for C data interface (#1407)
* initial commit
* add integration tests for python
* address comments
Remzi Yang [Tue, 8 Mar 2022 15:28:20 +0000 (23:28 +0800)]
fix (#1406)
Signed-off-by: remzi <13716567376yh@gmail.com>
jakevin [Mon, 7 Mar 2022 21:29:59 +0000 (05:29 +0800)]
add unit test to check all none (#1405)
Andrew Lamb [Sun, 6 Mar 2022 18:41:11 +0000 (13:41 -0500)]
Improve integration testing docs (#1403)
Sumit [Sun, 6 Mar 2022 13:36:36 +0000 (14:36 +0100)]
Move csv Parser trait and its implementations to utils module (#1385)
* move Parser trait to utils
this allow the parser capabilities to be re-used for json module
* implement parse_formatted for date32
* remove redundant checks
* Update arrow/src/util/mod.rs
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* make Parser trait pub(crate) only and not pub
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Yijie Shen [Sun, 6 Mar 2022 11:44:54 +0000 (19:44 +0800)]
Introduce `ReadOptions` with builder API, for parquet filter row groups that satisfy all filters, and enable filter row groups by range. (#1389)
* Filter row groups by comparing midpoint with offset range
* lint
* ReadOptions with builder API
* fix comments
* precise range doc
* tab to space
Andrew Lamb [Sun, 6 Mar 2022 11:41:58 +0000 (06:41 -0500)]
Add note in contributing guideline about types of contributions (#1396)
* Add note in contributing guideline about types of contributions
* prettier
Andrew Lamb [Sat, 5 Mar 2022 13:54:34 +0000 (08:54 -0500)]
fix: Fix grpc schema hack in flight integration test (#1402)
projects / arrow-rs.git / log
