Fix multiple discriminator behavior and free up marks when resolved
authorSteve Lawrence <slawrence@apache.org>
Tue, 11 Aug 2020 16:06:41 +0000 (12:06 -0400)
committerSteve Lawrence <stephen.d.lawrence@gmail.com>
Mon, 17 Aug 2020 15:51:35 +0000 (11:51 -0400)
commit1763fb2d13eba1df5dacb44c560bd24010f35881
tree15b268b17ea48fcaa8ef2aa385ae1aeb0fbf34b5
parentcf57e0d04ed7414081febb0992e6ebdbb6cd708d
Fix multiple discriminator behavior and free up marks when resolved

Currently we maintain a separator discriminator stack, and parser would
push, pop, and modify this stack appropriately to make parse decisions.

One problem with this implementation is that discriminators only ever
modify the state on the top of the stack, which means that multiple
discriminators are essentially no-ops, rather than resolving points of
uncertainty up the stack.

Another problem is that discriminators are not tied to Marks in any way,
so resolving a discriminator does not immediately discard the Mark, and
thus memory associated with that Mark cannot be freed until the parse
unwinds.

To resolve both issues, this patch completely removes the discriminator
stack, and instead replace it with a stack of Marks, each of which
represents a points of uncertainties. Resolving a point of uncertainty
(e.g. from a discriminator) simply discards the mark on the top of the
stack. This frees up the Mark and associated memory, as well as allows
multiple discriminator behavior to discriminate multiple points of
uncertainty.

To determine if a point of uncertainty was resolved, parsers must simply
check if the associated Mark is at the top of the stack. If it is not,
then something discriminated the point of uncertainty. To simplify this
logic, multiple helper functions and methods are created to ensure
points of uncertainty are created/destroyed/handled correctly.

DAFFODIL-2371
20 files changed:
build.sbt
daffodil-cli/src/it/scala/org/apache/daffodil/debugger/TestCLIDebugger.scala
daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/PrimitivesExpressions.scala
daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/SequenceCombinator.scala
daffodil-macro-lib/src/main/scala/org/apache/daffodil/processors/parsers/PointOfUncertaintyMacros.scala [new file with mode: 0644]
daffodil-runtime1/src/main/scala/org/apache/daffodil/debugger/InteractiveDebugger.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/ProcessorStateBases.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/AssertPatternParsers.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/ElementCombinator1.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/ElementKindParsers.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/ExpressionEvaluatingParsers.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/InitiatedContentParsers.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/NilEmptyCombinatorParsers.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/PState.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/Parser.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/SeparatedParseHelper.scala
daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/SequenceParserBases.scala
daffodil-test/src/test/resources/org/apache/daffodil/section15/choice_groups/ChoiceGroupInitiatedContent.tdml
daffodil-test/src/test/scala/org/apache/daffodil/section07/discriminators/TestDiscriminators.scala
daffodil-test/src/test/scala/org/apache/daffodil/section15/choice_groups/TestChoiceGroupInitiatedContent.scala