Add blob support
authorSteve Lawrence <slawrence@apache.org>
Thu, 15 Aug 2019 17:02:34 +0000 (13:02 -0400)
committerSteve Lawrence <stephen.d.lawrence@gmail.com>
Thu, 5 Sep 2019 14:26:30 +0000 (10:26 -0400)
commit9de0d1a3c3d294600d5c769243efe4f67b03bac1
treeff7f8e5abb1d565a0699293a422f05373842adca
parentebee69f3362b6d418a501927c5f0faaf794c75ef
Add blob support

- Adds support for xs:anyURI primitive type. Elements with this type
  must specifiy the new property dfdl:objectKind, which can be
  "bytes" or "characters". Support for "characters" is not added in this
  patch.
- On parse elements with objectKind="bytes" will have the data read
  and written out to a file with a randomly generated name, with a URI
  to that file being put in the infoset. On unparse, the data is read
  from the URI and written to the data stream. Behavior is very similar
  to xs:hexBinary.
- BLOB Objects must have lengths set with dfdl:lengthKind="explicit".
- New API functions are added to set where to write blob files to and to
  a get list of all blob files that were written.
- TDML Runner is updated to be aware of xsi:type="xs:anyURI" and compare
  the contents of the blob files. It will also delete blob files after a
  test passes.
- The CLI will write blob files to the $PWD/daffodil-blobs/ directory.

DAFFODIL-1735
42 files changed:
.github/workflows/main.yml
daffodil-cli/src/main/scala/org/apache/daffodil/Main.scala
daffodil-core/src/main/scala/org/apache/daffodil/dsom/ElementBase.scala
daffodil-core/src/main/scala/org/apache/daffodil/dsom/RestrictionUnion.scala
daffodil-core/src/main/scala/org/apache/daffodil/dsom/RuntimePropertyMixins.scala
daffodil-core/src/main/scala/org/apache/daffodil/dsom/SimpleTypes.scala
daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementBaseGrammarMixin.scala
daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/Padded.scala
daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/PrimitivesLengthKind.scala
daffodil-io/src/main/scala/org/apache/daffodil/io/DataInputStream.scala
daffodil-io/src/main/scala/org/apache/daffodil/io/InputSourceDataInputStream.scala
daffodil-io/src/main/scala/org/apache/daffodil/io/StringDataInputStreamForUnparse.scala
daffodil-lib/src/main/scala/org/apache/daffodil/util/Misc.scala
daffodil-lib/src/main/scala/org/apache/daffodil/xml/XMLUtils.scala
daffodil-lib/src/test/scala/org/apache/daffodil/xml/test/unit/TestXMLUtils.scala
daffodil-propgen/src/main/resources/org/apache/daffodil/xsd/DFDL_part2_attributes.xsd
daffodil-propgen/src/main/resources/org/apache/daffodil/xsd/DFDL_part3_model.xsd
daffodil-propgen/src/main/resources/org/apache/daffodil/xsd/dafext.xsd
daffodil-propgen/src/main/resources/org/apache/daffodil/xsd/dfdlx.xsd
daffodil-runtime1-unparser/src/main/scala/org/apache/daffodil/processors/unparsers/BlobLengthUnparser.scala [new file with mode: 0644]
daffodil-runtime1/src/main/scala/org/apache/daffodil/dpath/NodeInfo.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/infoset/InfosetImpl.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/infoset/InfosetOutputter.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/DataProcessor.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/BlobLengthParsers.scala [new file with mode: 0644]
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/PState.scala
daffodil-tdml-lib/src/main/scala/org/apache/daffodil/tdml/TDMLRunner.scala
daffodil-tdml-lib/src/main/scala/org/apache/daffodil/tdml/processor/TDMLDFDLProcessor.scala
daffodil-tdml-processor/src/main/scala/org/apache/daffodil/tdml/TDMLInfosetInputter.scala
daffodil-tdml-processor/src/main/scala/org/apache/daffodil/tdml/processor/DaffodilTDMLDFDLProcessor.scala
daffodil-test/src/test/resources/org/apache/daffodil/section05/simple_types/Blobs.tdml [new file with mode: 0644]
daffodil-test/src/test/resources/org/apache/daffodil/section05/simple_types/SimpleTypes.tdml
daffodil-test/src/test/resources/org/apache/daffodil/section05/simple_types/blobs/blob_01.bin [new file with mode: 0644]
daffodil-test/src/test/resources/org/apache/daffodil/section05/simple_types/blobs/blob_02.bin [new file with mode: 0644]
daffodil-test/src/test/resources/org/apache/daffodil/section05/simple_types/blobs/blob_03.bin [new file with mode: 0644]
daffodil-test/src/test/resources/org/apache/daffodil/section05/simple_types/blobs/blob_04.bin [new file with mode: 0644]
daffodil-test/src/test/resources/org/apache/daffodil/section05/simple_types/blobs/blob_07.bin [new file with mode: 0644]
daffodil-test/src/test/resources/org/apache/daffodil/section05/simple_types/blobs/blob_13a.bin [new file with mode: 0644]
daffodil-test/src/test/resources/org/apache/daffodil/section05/simple_types/blobs/blob_13b.bin [new file with mode: 0644]
daffodil-test/src/test/resources/org/apache/daffodil/section05/simple_types/blobs/blob_13c.bin [new file with mode: 0644]
daffodil-test/src/test/resources/org/apache/daffodil/section05/simple_types/blobs/blob_13d.bin [new file with mode: 0644]
daffodil-test/src/test/scala/org/apache/daffodil/section05/simple_types/TestBlobs.scala [new file with mode: 0644]