3 hours agoNIFI-5951 Fix error logging with rollback on failure master
Koji Kawamura [Fri, 11 Jan 2019 02:46:07 +0000 (11:46 +0900)] 
NIFI-5951 Fix error logging with rollback on failure

Signed-off-by: Matthew Burgess <>
This closes #3264

6 days agoNIFI-5944: If processor is no longer scheduled to run after a failed trigger of ...
Mark Payne [Thu, 10 Jan 2019 21:33:55 +0000 (16:33 -0500)] 
NIFI-5944: If processor is no longer scheduled to run after a failed trigger of @OnScheduled, ensure that we set scheduled status to STOPPED

This closes #3263

7 days agoNIFI-5945 Add support for password login to kerberos code in nifi-security-utils
Bryan Bende [Wed, 9 Jan 2019 22:37:10 +0000 (17:37 -0500)] 
NIFI-5945 Add support for password login to kerberos code in nifi-security-utils

Fixing solr test

Signed-off-by: Matthew Burgess <>
This closes #3256

7 days agoNIFI-5790 removed the last test as it's causing a race condition intermittently ...
SavtechSolutions [Thu, 10 Jan 2019 19:13:35 +0000 (14:13 -0500)] 
NIFI-5790 removed the last test as it's causing a race condition intermittently (#3260)

Signed-off-by: Peter Wicks <>
7 days agoNIFI-5944: When components are started on NiFi startup, if they are invalid, don...
Mark Payne [Thu, 10 Jan 2019 14:53:05 +0000 (09:53 -0500)] 
NIFI-5944: When components are started on NiFi startup, if they are invalid, don't fail immediately and give up. Instead, keep attempting to start the component when it becomes valid.

This closes #3259

7 days agoNIFI-5909 added optional settings for date, time, and timestamp formats used to write...
Alex Savitsky [Wed, 19 Dec 2018 16:08:32 +0000 (11:08 -0500)] 
NIFI-5909 added optional settings for date, time, and timestamp formats used to write Records to Elasticsearch

NIFI-5909 added content checks to the unit tests

NIFI-5937 use explicit long value for test dates/times (to not depend on the timezone of test executor)

NIFI-5937 tabs to spaces

Fixing checkstyle violations introduced by PR)

NIFI-5937 adjusted property descriptions for consistency; limited EL scope to variable registry; added an appropriate validator along with its Maven dependency; moved format initialization to @OnScheduled

NIFI-5909 tabs to spaces

Signed-off-by: Ed <>
This closes #3227

8 days agoNIFI-5937 use processor-configured encoding instead of the system default
Alex Savitsky [Tue, 8 Jan 2019 15:15:45 +0000 (10:15 -0500)] 
NIFI-5937 use processor-configured encoding instead of the system default

NIFI-5937 added tests to verify that accented characters are preserved correctly

NIFI-5937 unfolding starred imports

NIFI-5937 unfolding starred imports (now with statics)

Signed-off-by: Ed <>
This closes #3250

8 days agoNIFI-5841 Fix memory leak of PutHive3Streaming.
kei miyauchi [Tue, 8 Jan 2019 00:38:46 +0000 (09:38 +0900)] 
NIFI-5841 Fix memory leak of PutHive3Streaming.

This closes #3249.

Signed-off-by: Koji Kawamura <>
9 days agoNIFI-5826 Fix back-slash escaping at Lexers
Koji Kawamura [Wed, 5 Dec 2018 06:03:21 +0000 (15:03 +0900)] 
NIFI-5826 Fix back-slash escaping at Lexers

Adding unit test cases for escaped characters

Signed-off-by: Ed <>
This closes #3200

9 days agoNIFI-5935:
Matt Gilman [Mon, 7 Jan 2019 19:02:42 +0000 (14:02 -0500)] 
- Ensuring exceptions are handled in the ldap user/group sync background thread.
- Adding additional logging around what users/groups were discovered.

Signed-off-by: Pierre Villard <>
This closes #3247.

9 days agoNIFI-5920: Tagging an object in S3
Stephen Goodman [Sun, 30 Dec 2018 00:10:38 +0000 (19:10 -0500)] 
NIFI-5920: Tagging an object in S3

Unit tests and functionality for tagging an object in S3.

Set FlowFile attributes directly from tags retrieved from S3.

Add guard clauses to ensure evaluated properties are not blank.

This closes #3239.

Signed-off-by: Koji Kawamura <>
10 days agoNIFI-5879: Fixed bug in FileSystemRepository that can occur if an InputStream is...
Mark Payne [Thu, 6 Dec 2018 21:22:29 +0000 (16:22 -0500)] 
NIFI-5879: Fixed bug in FileSystemRepository that can occur if an InputStream is obtained, then more data is written to the Content Claim - the InputStream would end before allowing the sequential data to be read. Also fixed bugs in LimitedInputStream related to available(), mark(), and reset() and the corresponding unit tests. Additionally, found that one call to was not properly flushing the output of any Content Claim that has been written to before attempting to read it.

Signed-off-by: Matthew Burgess <>
This closes #3207

10 days agoNIFI-5854 Added skeleton logic to convert decimal time units.
Andy LoPresto [Fri, 30 Nov 2018 04:17:58 +0000 (20:17 -0800)] 
NIFI-5854 Added skeleton logic to convert decimal time units.
Added helper methods.
Added unit tests.

NIFI-5854 [WIP] Cleaned up logic.
Resolved failing unit tests due to error message change.

NIFI-5854 [WIP] All helper method unit tests pass.

NIFI-5854 [WIP] FormatUtils#getPreciseTimeDuration() now handles all tested inputs correctly.
Added unit tests.

NIFI-5854 [WIP] FormatUtils#getTimeDuration() still using long.
Added unit tests.
Renamed existing unit tests to reflect method under test.

NIFI-5854 FormatUtils#getTimeDuration() returns long but now accepts decimal inputs.
Added @Deprecation warnings (will update callers where possible).
All unit tests pass.

NIFI-5854 Fixed unit tests (ran in IDE but not Maven) due to int overflows.
Fixed checkstyle issues.

NIFI-5854 Fixed typo in Javadoc.

NIFI-5854 Fixed typo in Javadoc.

Signed-off-by: Matthew Burgess <>
This closes #3193

13 days agoNIFI-5927 Added to project root.
Andy LoPresto [Fri, 4 Jan 2019 05:01:29 +0000 (21:01 -0800)] 
NIFI-5927 Added to project root.
Removed reference to Jasypt from cryptography section of
Fixed JCraft link for Chrome browser.

This closes #3243.

Signed-off-by: Andrew Lim <>
13 days agoNIFI-5926 Added certificate commands to toolkit guide.
Andy LoPresto [Fri, 4 Jan 2019 00:36:28 +0000 (16:36 -0800)] 
NIFI-5926 Added certificate commands to toolkit guide.
Applied formatting and organizational changes from PR review.

This closes #3242.

Signed-off-by: Andrew Lim <>
13 days agoNIFI-5921 - Timeout property for ConsumeJMS
Pierre Villard [Fri, 4 Jan 2019 09:51:19 +0000 (10:51 +0100)] 
NIFI-5921 - Timeout property for ConsumeJMS

Signed-off-by: Pierre Villard <>
This closes #3245.

13 days agoNIFI-5921 - Added property to allow a user to define a timeout on the ConsumeJMS...
Ryan Whittington [Wed, 2 Jan 2019 16:17:06 +0000 (16:17 +0000)] 
NIFI-5921 - Added property to allow a user to define a timeout on the ConsumeJMS processor

Co-Authored-By: rwhittington <>
Signed-off-by: Pierre Villard <>
This closes #3240.

2 weeks agoNIFI-5908 Inject locale info into JsonInjector object to make sure we don't lose...
Mike Thomsen [Tue, 18 Dec 2018 17:53:29 +0000 (12:53 -0500)] 
NIFI-5908 Inject locale info into JsonInjector object to make sure we don't lose that information.

Signed-off-by: Pierre Villard <>
This closes #3226.

2 weeks agoNIFI-5919: Addressed a race condition that can exist if adding FlowFiles to a FlowFil...
Mark Payne [Fri, 28 Dec 2018 15:35:48 +0000 (10:35 -0500)] 
NIFI-5919: Addressed a race condition that can exist if adding FlowFiles to a FlowFileQueue before adjusting the size of the queue to account for the FlowFiles

Signed-off-by: Pierre Villard <>
This closes #3238.

2 weeks agoNIFI-5917 Fix TestSelectHiveQL.testNoTimeLimit
Koji Kawamura [Fri, 28 Dec 2018 04:09:50 +0000 (13:09 +0900)] 
NIFI-5917 Fix TestSelectHiveQL.testNoTimeLimit

Signed-off-by: Pierre Villard <>
This closes #3237.

2 weeks agoNIFI-5892 Wait timestamp lingers, potentially messing up downstream wait-notify pairs
Otto Fowler [Fri, 21 Dec 2018 14:15:24 +0000 (09:15 -0500)] 
NIFI-5892 Wait timestamp lingers, potentially messing up downstream wait-notify pairs
Clear the wait timestamp when transferring to failur or success

replace explicit attribute clear with function call, refactor and integrate into existing tests per review

This closes #3233.

Signed-off-by: Koji Kawamura <>
3 weeks agoMerge pull request #3232 from patricker/NIFI-5913
Pierre Villard [Fri, 21 Dec 2018 13:33:16 +0000 (14:33 +0100)] 
Merge pull request #3232 from patricker/NIFI-5913

NIFI-5913 - Standardize Definition of UUID in Documentation

4 weeks agoNIFI-5913 - Standardize Definition of UUID in Documentation 3232/head
Peter Wicks [Thu, 20 Dec 2018 17:11:07 +0000 (10:11 -0700)] 
NIFI-5913 - Standardize Definition of UUID in Documentation

4 weeks agoNIFI-5871 ignore UUID attribute when copying flow file attributes (#3203)
SavtechSolutions [Thu, 20 Dec 2018 17:05:25 +0000 (12:05 -0500)] 
NIFI-5871 ignore UUID attribute when copying flow file attributes (#3203)

NIFI-5871 ignore UUID attribute when copying flow file attributes

Signed-off-by: Peter Wicks <>
4 weeks agoMerge pull request #3219 from zenfenan/NIFI-5893
Pierre Villard [Wed, 19 Dec 2018 09:50:05 +0000 (10:50 +0100)] 
Merge pull request #3219 from zenfenan/NIFI-5893

NIFI-5893: AWS Endpoint Overriding now functions properly

4 weeks agoNIFI-4892 - ValidateCSV: no doublequote escaping in invalid output
Pierre Villard [Tue, 20 Feb 2018 09:36:31 +0000 (10:36 +0100)] 
NIFI-4892 - ValidateCSV: no doublequote escaping in invalid output

NIFI-4892 - ValidateCSV: no doublequote escaping in invalid output

NIFI-5907 - unit test

This closes #2481.

Signed-off-by: Koji Kawamura <>
4 weeks agoMerge pull request #3225 from ijokarumawak/nifi-5905
Pierre Villard [Tue, 18 Dec 2018 09:21:29 +0000 (10:21 +0100)] 
Merge pull request #3225 from ijokarumawak/nifi-5905

NIFI-5905 Secure S2S fails with NullPointerException

4 weeks agoMerge pull request #3224 from mattyb149/NIFI-5904
Pierre Villard [Tue, 18 Dec 2018 08:50:25 +0000 (09:50 +0100)] 
Merge pull request #3224 from mattyb149/NIFI-5904

NIFI-5904: Fix PutHive3Streaming handling of RecordReaderFactoryException

4 weeks agoNIFI-3988: Add fragment attributes to SplitRecord
Matthew Burgess [Thu, 13 Dec 2018 19:02:14 +0000 (14:02 -0500)] 
NIFI-3988: Add fragment attributes to SplitRecord

This closes #3217.

Signed-off-by: Koji Kawamura <>
4 weeks agoNIFI-5905 Secure S2S fails with NullPointerException 3225/head
Koji Kawamura [Tue, 18 Dec 2018 04:57:28 +0000 (13:57 +0900)] 
NIFI-5905 Secure S2S fails with NullPointerException

The constructor arguments should be set to instance variables so that
subsequent code can refer "this.*".

4 weeks agoNIFI-5748 Improved Proxy Header Support
Jeff Storck [Mon, 29 Oct 2018 17:29:28 +0000 (13:29 -0400)] 
NIFI-5748 Improved Proxy Header Support

- Fixed proxy header support to use X-Forwarded-Host instead of X-ForwardedServer
- Added support for the context path header used by Traefik when proxying a service (X-Forwarded-Prefix)
- Added tests to ApplicationResourceTest for X-Forwarded-Context and X-Forwarded-Prefix
- Updated administration doc to include X-Forwarded-Prefix
- Added NIFI_WEB_PROXY_CONTEXT_PATH env var to dockerhub and dockermaven scripts
- Added documentation for NIFI_WEB_PROXY_CONTEXT_PATH to dockerhub
- Updated ApplicationResource to handle a port specified in X-ProxyPort and X-Forwarded-Port headers

This closes #3129.

Signed-off-by: Kevin Doran <>
4 weeks agoMerge pull request #3220 from zenfenan/NIFI-5898
Andy LoPresto [Mon, 17 Dec 2018 19:39:09 +0000 (11:39 -0800)] 
Merge pull request #3220 from zenfenan/NIFI-5898

NIFI-5898: Updated the display name for ACCESS_KEY & SECRET_KEY

Signed-off-by: Andy LoPresto <>
4 weeks agoNIFI-5904: Fix PutHive3Streaming handling of RecordReaderFactoryException 3224/head
Matthew Burgess [Mon, 17 Dec 2018 19:12:18 +0000 (14:12 -0500)] 
NIFI-5904: Fix PutHive3Streaming handling of RecordReaderFactoryException

4 weeks agoNIFI-5898: Updated the display name for ACCESS_KEY & SECRET_KEY 3220/head
zenfenan [Sun, 16 Dec 2018 11:35:52 +0000 (17:05 +0530)] 
NIFI-5898: Updated the display name for ACCESS_KEY & SECRET_KEY

4 weeks agoNIFI-5893: AWS Endpoint Overriding now functions properly 3219/head
zenfenan [Sat, 15 Dec 2018 14:07:31 +0000 (19:37 +0530)] 
NIFI-5893: AWS Endpoint Overriding now functions properly

5 weeks agoNIFI-4579: Fix ValidateRecord type coercing
Koji Kawamura [Thu, 14 Jun 2018 06:39:17 +0000 (15:39 +0900)] 
NIFI-4579: Fix ValidateRecord type coercing

Signed-off-by: Matthew Burgess <>
This closes #2794

5 weeks agoNIFI-5891 fix handling of null logical types in Hive3Streaming processor
gkkorir [Thu, 13 Dec 2018 14:25:37 +0000 (17:25 +0300)] 
NIFI-5891 fix handling of null logical types in Hive3Streaming processor

NIFI-5891: Fixed Checkstyle issues
Signed-off-by: Matthew Burgess <>
This closes #3216

5 weeks agoNIFI-5621: Added Cassandra connection provider service
zenfenan [Sun, 21 Oct 2018 17:41:07 +0000 (23:11 +0530)] 
NIFI-5621: Added Cassandra connection provider service
NIFI-5621: Improvements on LICENSE & NOTICE files

This closes #3105

Signed-off-by: Mike Thomsen <>
5 weeks agoNIFI-5889 - changed the wording about Destination URL in S2S RTs
Pierre Villard [Tue, 11 Dec 2018 16:36:45 +0000 (17:36 +0100)] 
NIFI-5889 - changed the wording about Destination URL in S2S RTs

This closes #3215.

Signed-off-by: Koji Kawamura <>
5 weeks agoNIFI-5890 Added a unit test that proves that 1.9 fixes a bug in the date conversion...
Mike Thomsen [Tue, 11 Dec 2018 16:18:52 +0000 (11:18 -0500)] 
NIFI-5890 Added a unit test that proves that 1.9 fixes a bug in the date conversion behavior in 1.8.

This closes #3214.

Signed-off-by: Koji Kawamura <>
5 weeks agoNIFI-5875 Improve docs around the PriorityAttributePrioritizer
Wil Selwood [Thu, 6 Dec 2018 10:17:19 +0000 (10:17 +0000)] 
NIFI-5875 Improve docs around the PriorityAttributePrioritizer

Clear up the documentation around the PriorityAttributePrioritizer to make it clearer that if the attributes are both number the largest wins but if they are not numbers they sort the other way around.

I had to go and look at the code to work out what was going on after reading the existing documentation.

Simplify PriorityAttributePrioritzer docs

Simplify the main section of documentation for PriorityAttributePrioritzer and then add a notes section with more details about the ordering.

Fix example.

Put back comment setting the priority attribute

Further improvements to the Priority ordering

This closes #3205.

Signed-off-by: Koji Kawamura <>
5 weeks agoNIFI-5859: Added XML-based documentation writer that can be used to document a component.
Mark Payne [Thu, 15 Nov 2018 17:07:08 +0000 (12:07 -0500)] 
NIFI-5859: Added XML-based documentation writer that can be used to document a component.
- Found several instances of nifi-framework-api's ProviderException being thrown from processors. Changed those to IllegalStateException, as ProviderException is not an appropriate Exception in those cases, and extensions should not depend on nifi-framework-api.
- Performed some cleanup, moving Property Descriptors from Controller Service API's/specs into the implementations. Adding to the Service API results in bringing in nifi-utils to the nifi-standard-services-api-nar, which is a bad practice. The 'main' service api nar should not have a dependency on a util class.

NIFI-5859: Added javadocs. Fixed pom.xml that was left pointing to snapshot version of nar maven plugin

NIFI-5859: Addressing review feedback: adding component type, multiple additional details into separate file(s)

This closes #3192.

Signed-off-by: Bryan Bende <>
5 weeks agoNIFI-5885 ArrayOutOfBoundsException at EL 'or' and 'and' functions
Koji Kawamura [Mon, 10 Dec 2018 05:31:16 +0000 (14:31 +0900)] 
NIFI-5885 ArrayOutOfBoundsException at EL 'or' and 'and' functions

EL 'or' and 'and' functions can be called multiple times within the same context using the same evaluator instance.
That happens if their subject is derived from an IteratingEvaluator such as 'anyDelineatedValues'.

And if the right hand side expression for such 'or' and 'and' contains another IteratingEvaluator,
then it can be evaluated more than the number of its candidates, ultimately an ArrayOutOfBoundsException is thrown.

This commit makes Or/AndEvaluator caching its right hand side result to prevent that happens.
For 'or' and 'and' functions, the right hand side expression is independant from their subject boolean value.
It's enough evaluating right hand side once, because it returns the same result even with different subjects.

Signed-off-by: Pierre Villard <>
This closes #3212.

5 weeks agoNIFI-5872 - Added compression option to JsonRecordSetWriter
Pierre Villard [Thu, 6 Dec 2018 08:50:20 +0000 (09:50 +0100)] 
NIFI-5872 - Added compression option to JsonRecordSetWriter

This closes #3208.

Signed-off-by: Koji Kawamura <>
5 weeks agoNIFI-5884 Bumping hbase-client version from 1.1.2 to 1.1.13
Bryan Bende [Fri, 7 Dec 2018 15:50:43 +0000 (10:50 -0500)] 
NIFI-5884 Bumping hbase-client version from 1.1.2 to 1.1.13

This closes #3211.

Signed-off-by: Koji Kawamura <>
5 weeks agoNIFI-5881: Enable to export the template for non-ascii name
Kemix Koo [Fri, 7 Dec 2018 08:31:36 +0000 (16:31 +0800)] 
NIFI-5881: Enable to export the template for non-ascii name

This closes #3210.

Signed-off-by: Koji Kawamura <>
5 weeks agoNIFI-5883 - Update
Dan Levy [Fri, 7 Dec 2018 06:06:59 +0000 (23:06 -0700)] 
NIFI-5883 - Update

Signed-off-by: Pierre Villard <>
This closes #3209.

6 weeks agoNIFI-5830 - RedisConnectionPoolService does not work with Standalone Redis using...
Alexander Bukarev [Tue, 4 Dec 2018 19:15:37 +0000 (22:15 +0300)] 
NIFI-5830 - RedisConnectionPoolService does not work with Standalone Redis using non-localhost deployment

Signed-off-by: Pierre Villard <>
This closes #3176.

6 weeks agoNIFI-5877 Protecting against null values when obtaining master address
Bryan Bende [Thu, 6 Dec 2018 16:45:26 +0000 (11:45 -0500)] 
NIFI-5877 Protecting against null values when obtaining master address

Signed-off-by: Pierre Villard <>
This closes #3206.

6 weeks agoNIFI-5868: Added instrumentation around ListFile such that all disk accesses are...
Mark Payne [Wed, 5 Dec 2018 14:37:20 +0000 (09:37 -0500)] 
NIFI-5868: Added instrumentation around ListFile such that all disk accesses are timed and any unusually long listing times or disk access operations can be logged. Additionally, information is logged at a debug level including significant amounts of troubleshooting information when configured to do so

This closes #3202.

Signed-off-by: Bryan Bende <>
6 weeks agoNIFI-5863 Fixed AbstractFlowFileQueue logging
Koji Kawamura [Wed, 5 Dec 2018 01:23:57 +0000 (10:23 +0900)] 
NIFI-5863 Fixed AbstractFlowFileQueue logging

Signed-off-by: Pierre Villard <>
This closes #3199.

6 weeks agoNIFI-5862 MockRecordParser Has Bad Logic for failAfterN
Peter Wicks [Mon, 3 Dec 2018 17:44:01 +0000 (10:44 -0700)] 
NIFI-5862 MockRecordParser Has Bad Logic for failAfterN

This closes #3195.

Signed-off-by: Koji Kawamura <>
6 weeks agoNIFI-5838 - Improve the schema validation method in Kite processors
Pierre Villard [Thu, 22 Nov 2018 17:50:11 +0000 (18:50 +0100)] 
NIFI-5838 - Improve the schema validation method in Kite processors


Add empty check

This closes #3182.

Signed-off-by: Koji Kawamura <>
6 weeks agoNIFI-5795 RedisDistributedMapCacheClientService put missing option
luup2k [Tue, 6 Nov 2018 19:05:23 +0000 (20:05 +0100)] 
NIFI-5795 RedisDistributedMapCacheClientService put missing option

Signed-off-by: Pierre Villard <>
This closes #3135.

6 weeks agoNIFI-5850: Replaced custom AWS regions enum with the one from AWS Java SDK
zenfenan [Thu, 29 Nov 2018 09:29:23 +0000 (14:59 +0530)] 
NIFI-5850: Replaced custom AWS regions enum with the one from AWS Java SDK

Signed-off-by: Pierre Villard <>
This closes #3190. This closes #3187.

6 weeks agoNIFI-5855: Remove unnecessary ORDER BY clause in GenerateTableFetch when Partition...
Matthew Burgess [Thu, 29 Nov 2018 22:46:24 +0000 (17:46 -0500)] 
NIFI-5855: Remove unnecessary ORDER BY clause in GenerateTableFetch when Partition Size is zero

Signed-off-by: Pierre Villard <>
This closes #3191.

7 weeks agoNIFI-5846: Redirect URL is incorrect after logout
Kotaro Terada [Wed, 28 Nov 2018 05:23:39 +0000 (14:23 +0900)] 
NIFI-5846: Redirect URL is incorrect after logout

This closes #3185.

Signed-off-by: Peter Wicks <>
7 weeks agoNIFI-5849: ListXXX can lose cluster state on processor restart
Koji Kawamura [Thu, 29 Nov 2018 08:44:44 +0000 (17:44 +0900)] 
NIFI-5849: ListXXX can lose cluster state on processor restart

NIFI-5406 introduced the issue by trying to use the resetState variable for
different purposes. AbstractListProcessor should have had a different variable
to control whether to clear state for tracking entity strategy.

Signed-off-by: Pierre Villard <>
This closes #3189.

7 weeks agoNIFI-5845: Add support for OTHER and SQLXML JDBC types to SQL/Hive processors
Matthew Burgess [Tue, 27 Nov 2018 23:35:29 +0000 (18:35 -0500)] 
NIFI-5845: Add support for OTHER and SQLXML JDBC types to SQL/Hive processors

NIFI-5845: Incorporated review comments

This closes #3184.

Signed-off-by: Koji Kawamura <>
7 weeks agoNIFI-5843 added subjects to the error message when neither script body nor script...
Alex Savitsky [Wed, 28 Nov 2018 15:10:08 +0000 (10:10 -0500)] 
NIFI-5843 added subjects to the error message when neither script body nor script text are provided

Signed-off-by: Matthew Burgess <>
This closes #3186

7 weeks agoNIFI-5834: Restore default PutHiveQL error handling behavior
Matthew Burgess [Tue, 20 Nov 2018 22:58:59 +0000 (17:58 -0500)] 
NIFI-5834: Restore default PutHiveQL error handling behavior

NIFI-5834: Incorporated review comments

This closes #3179.

Signed-off-by: Koji Kawamura <>
7 weeks agoNIFI-5333 Added GetMongoRecord.
Mike Thomsen [Sun, 2 Sep 2018 19:58:33 +0000 (15:58 -0400)] 
NIFI-5333 Added GetMongoRecord.

Signed-off-by: zenfenan <>
This closes #3011

8 weeks agoNIFI-5812: Marked Database processors as PrimaryNodeOnly
zenfenan [Mon, 12 Nov 2018 10:16:26 +0000 (15:46 +0530)] 
NIFI-5812: Marked Database processors as PrimaryNodeOnly

Signed-off-by: Pierre Villard <>
This closes #3167.

8 weeks agoNIFI-5836 This closes #3181. changed the brittle timing based tests to integration...
joewitt [Wed, 21 Nov 2018 17:26:13 +0000 (12:26 -0500)] 
NIFI-5836 This closes #3181. changed the brittle timing based tests to integration tests

8 weeks agoNIFI-5560 This closes #3000. Added Follow SYMLINK support for listFTP & listSFTP...
Kumara M S Hemantha [Wed, 12 Sep 2018 20:49:32 +0000 (02:19 +0530)] 
NIFI-5560 This closes #3000. Added Follow SYMLINK support for listFTP & listSFTP and getFTP & getSFTP processors

Signed-off-by: joewitt <>
8 weeks agoNIFI-4260 This closes #3166. Add avro-ipc dependency to hadoop-libraries to support...
Aaron Leon [Sun, 11 Nov 2018 01:50:06 +0000 (19:50 -0600)] 
NIFI-4260 This closes #3166. Add avro-ipc dependency to hadoop-libraries to support Flume avro source

Signed-off-by: joewitt <>
8 weeks agoNIFI-4731 This closes #3019. This closes #2682. This closes #2420.
Pierre Villard [Tue, 18 Sep 2018 21:19:03 +0000 (23:19 +0200)] 
NIFI-4731 This closes #3019.  This closes #2682.  This closes #2420.
NIFI-4933 BigQuery PR Review

Signed-off-by: joewitt <>
8 weeks agoNIFI-4731: BQ Processors and GCP library update.
Daniel Jimenez [Sun, 29 Apr 2018 08:01:34 +0000 (10:01 +0200)] 
NIFI-4731: BQ Processors and GCP library update.

Signed-off-by: joewitt <>
8 weeks agoNIFI-5833 This closes #3180. Marked GetTwitter Consumer Key and Access Token processo...
Andy LoPresto [Wed, 21 Nov 2018 02:15:40 +0000 (18:15 -0800)] 
NIFI-5833 This closes #3180. Marked GetTwitter Consumer Key and Access Token processor properties as sensitive.
NIFI-5833 Added unit test to demonstrate arbitrary decryption of sensitive values regardless of processor property sensitive status.
NIFI-5833 Updated GetTwitter documentation with note about 1.9.0+ marking Consumer Key and Access Token as sensitive.

Signed-off-by: joewitt <>
8 weeks agoNIFI-5744: Put exception message to attribute while ExecuteSQL fail
yjhyjhyjh0 [Wed, 24 Oct 2018 13:22:06 +0000 (21:22 +0800)] 
NIFI-5744: Put exception message to attribute while ExecuteSQL fail

This closes #3107.

Signed-off-by: Peter Wicks <>
8 weeks agoNIFI-5814: Addressed issue in DatabaseReader class that was attempting to set values...
Mark Payne [Mon, 12 Nov 2018 17:12:16 +0000 (12:12 -0500)] 
NIFI-5814: Addressed issue in DatabaseReader class that was attempting to set values on the JSON returned by MaxMind. Instead of modifying the object directly, we should use an Injectable in the Reader so that the value read will have the appropriate values but we don't need to modify those objects returned by MaxMind

Signed-off-by: Pierre Villard <>
This closes #3168.

8 weeks agoNIFI-5828: Documents behavior of ExecuteSQL attrs when Max Rows Per Flow File is...
Colin Dean [Mon, 19 Nov 2018 04:14:55 +0000 (23:14 -0500)] 
NIFI-5828: Documents behavior of ExecuteSQL attrs when Max Rows Per Flow File is enabled

This closes #3177.

Signed-off-by: Koji Kawamura <>
2 months agoNIFI-5662 - Support for generic fixed when using decimal logical type
gkkorir [Fri, 16 Nov 2018 12:49:24 +0000 (15:49 +0300)] 
NIFI-5662 - Support for generic fixed when using decimal logical type

Signed-off-by: Pierre Villard <>
This closes #3175.

2 months agoNIFI-5823: Fixes typo in min idle connections property name
Colin Dean [Thu, 15 Nov 2018 18:46:41 +0000 (13:46 -0500)] 
NIFI-5823: Fixes typo in min idle connections property name

So many people missed it :-(

This closes #3172.

Signed-off-by: Koji Kawamura <>
2 months agoNIFI-5604: Added property to allow empty FlowFile when no SQL generated by GenerateTa...
Matthew Burgess [Fri, 2 Nov 2018 02:09:13 +0000 (22:09 -0400)] 
NIFI-5604: Added property to allow empty FlowFile when no SQL generated by GenerateTableFetch

co-authored by: Peter Wicks <>
Signed-off-by: Peter Wicks <>
This closes #3075.

2 months agoNIFI-5780 Add pre and post statements to ExecuteSQL and ExecuteSQLRecord
yjhyjhyjh0 [Wed, 7 Nov 2018 16:25:50 +0000 (00:25 +0800)] 
NIFI-5780 Add pre and post statements to ExecuteSQL and ExecuteSQLRecord

Signed-off-by: Peter Wicks <>
This closes #3156.

2 months agoNIFI-5824: Added unit test to FlowController to ensure that the ProcessScheduler...
Mark Payne [Thu, 15 Nov 2018 19:26:36 +0000 (14:26 -0500)] 
NIFI-5824: Added unit test to FlowController to ensure that the ProcessScheduler that it creates is properly initialized. Also updated the properties file used by TestFlowController to use a VolatileContentRepository instead of FileSystemRepository, and fixed EventDrivenWorkerQueue to return if calls to poll() are interrupted (via Thread.interrupt) - making these minor fixes resulted in the unit test TestFlowController running in 2 seconds instead of 30+ seconds on my machine

This closes #3173.

Signed-off-by: Bryan Bende <>
2 months agoNIFI-5822: Ensure that we don't call FlowController.getControllerServiceProvider...
Mark Payne [Thu, 15 Nov 2018 17:33:03 +0000 (12:33 -0500)] 
NIFI-5822: Ensure that we don't call FlowController.getControllerServiceProvider() before the ControllerServiceProvider has been initialized

This closes #3171.

Signed-off-by: Bryan Bende <>
2 months agoNIFI-5788: Introduce batch size limit in PutDatabaseRecord processor
vadimar [Mon, 5 Nov 2018 11:15:12 +0000 (13:15 +0200)] 
NIFI-5788: Introduce batch size limit in PutDatabaseRecord processor

NIFI-5788: Introduce batch size limit in PutDatabaseRecord processor

        Renamed 'batch size' to 'Maximum Batch Size'.
        Changed default value of max_batch_size to zero (INFINITE)
        Fixed parameter validation.
        Added unit tests

Signed-off-by: Matthew Burgess <>
This closes #3128

2 months agoNIFI-5652: Fixed LogMessage when logging level is disabled
Matthew Burgess [Wed, 14 Nov 2018 15:50:37 +0000 (10:50 -0500)] 
NIFI-5652: Fixed LogMessage when logging level is disabled

This closes #3170.

Signed-off-by: Koji Kawamura <>
2 months agoNIFI-4130 Add lookup controller service in TransformXML to define XSLT from the UI
Pierre Villard [Mon, 26 Jun 2017 15:48:06 +0000 (17:48 +0200)] 
NIFI-4130 Add lookup controller service in TransformXML to define XSLT from the UI

addressed review comments

Signed-off-by: Matthew Burgess <>
This closes #1953

2 months agoNIFI-5815 - PutORC processor 'Restricted' still requires access to restricted compone...
Pierre Villard [Mon, 12 Nov 2018 19:22:03 +0000 (20:22 +0100)] 
NIFI-5815 - PutORC processor 'Restricted' still requires access to restricted components regardless of restriction

This closes #3169.

Signed-off-by: Koji Kawamura <>
2 months agoNIFI-5805: Pool the BinaryEncoders used by the WriteAvroResultWithExternalSchema...
Mark Payne [Thu, 8 Nov 2018 19:54:05 +0000 (14:54 -0500)] 
NIFI-5805: Pool the BinaryEncoders used by the WriteAvroResultWithExternalSchema writer. Unfortunately, the writer that embeds schemas does not allow for this optimization due to the Avro API

This closes #3160.

2 months agoNIFI-5809: If QueryRecord has a single-column projection and that results in a null...
Mark Payne [Fri, 9 Nov 2018 16:40:59 +0000 (11:40 -0500)] 
NIFI-5809: If QueryRecord has a single-column projection and that results in a null value, do not confuse that with a null value being returned from the Record Reader

This closes #3163.

Signed-off-by: Koji Kawamura <>
2 months agoNIFI-5810 Add UserName EL support to JMS processors
Ed B [Sat, 10 Nov 2018 00:27:10 +0000 (19:27 -0500)] 
NIFI-5810 Add UserName EL support to JMS processors

Signed-off-by: Pierre Villard <>
This closes #3164.

2 months agoNIFI-5718: Added performance-based unit test (Ignored) for LineDemarcator
Mark Payne [Fri, 9 Nov 2018 17:04:43 +0000 (12:04 -0500)] 
NIFI-5718: Added performance-based unit test (Ignored) for LineDemarcator

Signed-off-by: Peter Wicks <>
This closes #3100.

2 months agoNIFI-5718: Implemented LineDemarcator and removed NLKBufferedReader in order to impro...
Mark Payne [Thu, 18 Oct 2018 16:05:16 +0000 (12:05 -0400)] 
NIFI-5718: Implemented LineDemarcator and removed NLKBufferedReader in order to improve performance

2 months agoNIFI-5757 Using Caffeine instead of slow synchronization on LinkedHashMap for caches...
Arek Burdach [Fri, 26 Oct 2018 14:49:50 +0000 (16:49 +0200)] 
NIFI-5757 Using Caffeine instead of slow synchronization on LinkedHashMap for caches - mainly avro schema caches

This closes #3111.

Signed-off-by: Mark Payne <>
2 months agoNIFI-5790: Exposes 6 commons-dbcp options in DBCPConnectionPool
Colin Dean [Mon, 5 Nov 2018 21:18:21 +0000 (16:18 -0500)] 
NIFI-5790: Exposes 6 commons-dbcp options in DBCPConnectionPool

Signed-off-by: Peter Wicks <>
This Closes #3133

2 months agoNIFI-5724 make database connection autocommit configurable
Vish Uma [Fri, 26 Oct 2018 19:32:46 +0000 (15:32 -0400)] 
NIFI-5724 make database connection autocommit configurable

making the database session autocommit value a configurable property

adding custom validation to PutSQL processor so as to disallow 'supports transaction' and 'rollback on failure' to be true when the autocommit value has been set to true

fixing some style issues to conform to standards

This closes #3113.

Signed-off-by: Koji Kawamura <>
2 months agoNIFI-5794 Allow empty string demarcator in ConsumeKafka processors
Pierre Villard [Wed, 7 Nov 2018 09:55:10 +0000 (10:55 +0100)] 
NIFI-5794 Allow empty string demarcator in ConsumeKafka processors

This closes #3159.

Signed-off-by: Koji Kawamura <>
2 months agoIncrease timeouts in all HandleHttpRequest unit tests from 3 seconds to 20 seconds...
Mark Payne [Thu, 8 Nov 2018 21:20:28 +0000 (16:20 -0500)] 
Increase timeouts in all HandleHttpRequest unit tests from 3 seconds to 20 seconds to avoid frequent unit test failures

2 months agoIncrease timeouts in unit tests to avoid frequent failures
Mark Payne [Thu, 8 Nov 2018 20:41:50 +0000 (15:41 -0500)] 
Increase timeouts in unit tests to avoid frequent failures

2 months agoNIFI-5802: Add QueryRecord nullable field support
Koji Kawamura [Thu, 8 Nov 2018 03:10:36 +0000 (12:10 +0900)] 
NIFI-5802: Add QueryRecord nullable field support

Signed-off-by: Pierre Villard <>
This closes #3158.

2 months agoNIFI-5800: Do not recursively call hashCode on child schema for Record Field Types
Mark Payne [Wed, 7 Nov 2018 22:00:51 +0000 (17:00 -0500)] 
NIFI-5800: Do not recursively call hashCode on child schema for Record Field Types

Signed-off-by: Pierre Villard <>
This closes #3157.

2 months agoNIFI-5784 Edit Admin Guide to remove duplicate content that is in new Toolkit Guide
Andrew Lim [Wed, 7 Nov 2018 16:20:59 +0000 (11:20 -0500)] 
NIFI-5784 Edit Admin Guide to remove duplicate content that is in new Toolkit Guide

Signed-off-by: Pierre Villard <>
This closes #3155.

2 months agoNIFI-5798: Fixed bug in FlattenJson that was escaping text as Java instead of escapin...
Mark Payne [Wed, 7 Nov 2018 14:08:54 +0000 (09:08 -0500)] 
NIFI-5798: Fixed bug in FlattenJson that was escaping text as Java instead of escaping as JSON

Signed-off-by: Pierre Villard <>
This closes #3138.

2 months agoNIFI-5752: Load balancing fails with wildcard certs
Kotaro Terada [Thu, 25 Oct 2018 10:46:06 +0000 (19:46 +0900)] 
NIFI-5752: Load balancing fails with wildcard certs

NIFI-5752: Remove an unnecessary String.format

NIFI-5752: Remove an unnecessary block

This closes #3110.

Signed-off-by: Koji Kawamura <>
2 months agoNIFI-5796 Addressed bug in subtract() method for keeping running total of counters...
Mark Payne [Tue, 6 Nov 2018 22:38:46 +0000 (17:38 -0500)] 
NIFI-5796 Addressed bug in subtract() method for keeping running total of counters for status history

This closes #3136.

Signed-off-by: Bryan Bende <>
2 months agoNIFI-5796: Addressed issue that caused Counters' values to show the wrong value in...
Mark Payne [Tue, 6 Nov 2018 21:20:57 +0000 (16:20 -0500)] 
NIFI-5796: Addressed issue that caused Counters' values to show the wrong value in Status History

Signed-off-by: Bryan Bende <>