Ran Ziv [Thu, 8 Jun 2017 15:26:29 +0000 (18:26 +0300)]
ARIA-166 Update README file
Tal Liron [Fri, 2 Jun 2017 19:20:28 +0000 (14:20 -0500)]
ARIA-199 Add "services outputs" CLI command
* Also add an output to hello world example
max-orlov [Tue, 6 Jun 2017 14:05:21 +0000 (17:05 +0300)]
ARIA-213 sporadic tests failures over locked database issue
Increased the timeout for acquiring database lock (for sqlite based db).
max-orlov [Wed, 31 May 2017 18:07:49 +0000 (21:07 +0300)]
ARIA-262 Inconsistent node attributes behavior
Inroduced a more comprehensive way to instrument relationship attributes.
Old behavior instrumented attributes only if they were accessed directly from the
parent model. Traversing the storage made the access to an attribute inconsistent.
The new solution enables encapsulating the attributes disregarding the way they
were retrieved.
Tal Liron [Fri, 2 Jun 2017 17:39:00 +0000 (12:39 -0500)]
ARIA-64 Remove PyYAML dependency
Avia Efrat [Sun, 21 May 2017 13:28:30 +0000 (16:28 +0300)]
ARIA-180 Separate Parameter models and change relationships
First, the Parameter model was separated in 6 individual models:
Input, Output, Property, Attribute, Configuration, Argument.
The first four models are TOSCA-related, and we needed to separate them
as according to the TOSCA spec, all four of these represent different
things, and have different fields.
The other two models (Configuration, Argument) are
orchestration-related, as and such are not supposed to represent the
same object as the TOSCA-related objects.
Second, up until now, the relationships of the then-Parameter models
were many-to-many to any other object related to them. This was a problem,
as logically the relationships were one-to-many (from the other models
to the Parameter models).
In this commit, we changed them to one-to-many. In addition, separating
the Parameter models allowed us to more accurately represent the relationship of each individual Parameter-model with its related models.
Tal Liron [Thu, 20 Apr 2017 22:54:47 +0000 (17:54 -0500)]
ARIA-149 Enhance operation configuration
* Also fixes ARIA-121, ARIA-190
* Parse special "dependencies" configuration parameters as YAML and
treat as Parameter models, allowing them full use of intrinsic
functions, type coersions, and validations
* Rename various functions that process "properties" to more generically
process "parameters" (properties, inputs, attributes, arguments, etc.)
* The "configuration" field in OperationTemplate and Operation models
is now now a dict of Parameter models
* Add "function" and "arguments" fields to Operation model to preserve
user data (in "implementation" and "inputs") and to clearly demarcate
orchestration data from user data; update task API accordingly
* Some cleanup of parser code touched by this commit
* Rename "create_parameters" to "merge_parameter_values" and improve
Ran Ziv [Mon, 29 May 2017 14:11:04 +0000 (17:11 +0300)]
ARIA-268 Add NOTICE file
Ran Ziv [Thu, 25 May 2017 09:18:58 +0000 (12:18 +0300)]
ARIA-261 Single-source ARIA version
max-orlov [Sun, 14 May 2017 19:38:39 +0000 (22:38 +0300)]
ARIA-258 Convert runtime_properties to attributes
max-orlov [Mon, 22 May 2017 15:28:12 +0000 (18:28 +0300)]
NullPool logging messages appear during execution
Ran Ziv [Mon, 22 May 2017 13:44:00 +0000 (16:44 +0300)]
ARIA-162 Upgrade Colorama library version
Upgraded the Colorama library version - This should
take care of the closed-stream error that appeared
sporadically after test runs.
Ran Ziv [Sun, 7 May 2017 11:58:46 +0000 (14:58 +0300)]
ARIA-208 Fix models relationships
Alexandru Zbarcea [Thu, 18 May 2017 20:23:13 +0000 (23:23 +0300)]
ARIA-259 Updated README install section for ArchLinux
Avia Efrat [Mon, 15 May 2017 11:26:34 +0000 (14:26 +0300)]
ARIA-171 service_template.services type inconsistency
As opposed to service_template.node_templates and service.nodes
which have a dict interface, service_template.services
had a list interface. That seemed inconsistent, so the former
interface was changed to a dict interface as well.
Tal Liron [Thu, 20 Apr 2017 22:54:47 +0000 (17:54 -0500)]
ARIA-148 Enhance CLI show commands
* Allow "--full" flag to provide a complete dump
* Allow "--json" and "--yaml" flags for dump in those formats
* Support for node graph and type hierarchies
* Some fixes for YAML dump for our custom types
* Also closes ARIA-186: "aria services show" command
Tal Liron [Thu, 20 Apr 2017 01:07:33 +0000 (20:07 -0500)]
ARIA-139 Support attributes
* Fully implement attribute support in parser
* New intrinsic function evaluation mechanism
* Implemented more intrinsic functions, including get_attribute
* Fix to one-on-one relationship back population
* Fixes to TOSCA use case examples
* Indirectly related: re-enabled node_filter mechanism and reworked
filter constraints in order to make them serializable
* utils/type is much more robust now and consolidates all conversions
and names
* Moved dsl_specification to new utils/specification (because utils/type
uses it)
max-orlov [Tue, 9 May 2017 14:24:31 +0000 (17:24 +0300)]
ARIA-213 Sporadic tests failures over locked database issue
Move from 2 different sessions - one for the log, and the other for general model operations,
to one single session, while utilizing the keep tracking of changes mechanism for both logs and node/task states.
Avia Efrat [Wed, 10 May 2017 14:39:49 +0000 (17:39 +0300)]
ARIA-157 Failing CLI service-templates store tests on Windows
Three tests from `aria service-templates store` failed on Windows, but
not on Linux.
The reason for this failures was differing implementation of
os.path.dirname across the platforms.
Python implements os.path.dirname in the ntpath module. There, somewhere
down the line of calls, (a part of) the argument of dirname is tested
for membership in a string (using `in`). In these three tests, the
argument of dirname is of type MagicMock, and an error is raised since
only a string can be tested for membership in a string.
The solution was to mock the dirname calls.
Ran Ziv [Wed, 10 May 2017 09:08:01 +0000 (12:08 +0300)]
ARIA-62 Apply pylint on extensions
Tal Liron [Fri, 14 Apr 2017 18:39:02 +0000 (13:39 -0500)]
ARIA-140 Version utils
Provided mainly to support version comparisons for plugins in order to
select the highest version of available plugins that would match a
plugin specification. In the future may be useful for other version
comparisons.
Avia Efrat [Mon, 8 May 2017 14:45:23 +0000 (17:45 +0300)]
ARIA-210 Handle relative paths in CLI service-templates
This was a rather simple change, mainly involving adding absolute path
references.
The problems were only in `service-templates store` and in
`service-templates create-archive`.
`service-templates validate` was not affected.
max-orlov [Mon, 8 May 2017 11:46:58 +0000 (14:46 +0300)]
ARIA-160 Tests fail spordically over parameter binding
max-orlov [Mon, 8 May 2017 12:25:37 +0000 (15:25 +0300)]
ARIA-165 Make node name suffix UUIDs become more readable
max-orlov [Mon, 8 May 2017 14:29:29 +0000 (17:29 +0300)]
ARIA-230 Dry execution doesn't log empty operations
max-orlov [Sun, 7 May 2017 13:12:56 +0000 (16:12 +0300)]
ARIA-214 Dry execution changes the state of non implemented operations
Ran Ziv [Sun, 7 May 2017 13:36:39 +0000 (16:36 +0300)]
ARIA-215 Refactor plugin-related code into PluginManager
Refactored plugin-related code from ProcessExecutor into PluginManager.
Additionally, renamed plugin_prefix to plugin_dir in PluginManager.
max-orlov [Wed, 3 May 2017 09:42:27 +0000 (12:42 +0300)]
ARIA-160 Operation toolbelt unit tests fail spordically
Inroduced a filesystem based data holder
Avia Efrat [Sun, 7 May 2017 08:42:58 +0000 (11:42 +0300)]
ARIA-156 Better handle exceptions in the process executor
Previously, if an exception was raised during the starting of a task,
the task's process was permanently blocked on receiving a message.
The reason was that the exception caused the 'listener thread' to
not send a response to the task's process, as the exception was not
handled inside the 'with' block of the listener thread.
The first change I introduced was to wrap the yielding of the message and
the response inside a try-except-finally block, so the exception will be
handled within the 'with' scope, and to ensure a response is sent to the
task's process.
The second change is to move the sending of the 'task started' message in
the task's process to a place where encountering an exception will be
handled via sending a 'task failed' message back to the listener thread.
Ran Ziv [Thu, 4 May 2017 14:07:50 +0000 (17:07 +0300)]
ARIA-209 Fix create-archive CLI command help
max-orlov [Sun, 30 Apr 2017 13:05:27 +0000 (16:05 +0300)]
ARIA-163 Update node state for empty tasks
Additional changes:
* removed `for_node` and `for_relationship` from the api OperationTask.
* api based OperationTask could also have an empty implementation.
* each core task wields its own executor.
* Reordered some of the helper functions for creating tasks.
* intoduced 2 new executors: StubTaskExecutor (for stub tasks) and EmptyOperationExecutor (for empty tasks)
Ran Ziv [Sun, 30 Apr 2017 12:40:57 +0000 (15:40 +0300)]
ARIA-161 Disable end2end tests on Windows
Ran Ziv [Sun, 30 Apr 2017 12:15:37 +0000 (15:15 +0300)]
ARIA-155 Clean models from unused fields
max-orlov [Thu, 27 Apr 2017 14:09:14 +0000 (17:09 +0300)]
ARIA-158 Restore support for relationship ordering
max-orlov [Thu, 20 Apr 2017 15:51:46 +0000 (18:51 +0300)]
ARIA-146 Support colorful execution logging
Ran Ziv [Thu, 27 Apr 2017 12:11:15 +0000 (15:11 +0300)]
ARIA-154 Create end-to-end test for the Nodecellar example
Ran Ziv [Wed, 26 Apr 2017 17:21:19 +0000 (20:21 +0300)]
ARIA-153 Write end-to-end tests for ARIA
Created infrastructure for end-to-end tests,
plus a test for the hello-world example.
Avia Efrat [Thu, 20 Apr 2017 10:23:32 +0000 (13:23 +0300)]
Improve execution cancelling
Unhandled execution status transitions resulting from cancelling an
execution via the CLI, that we indentified and tried to address:
1. TERMINATED -> CANCELLING
You cancel the execution, but by the time we try to set the status to
CANCELLING, the execution thread had already finished, and therefore,
in SUCCEEDED status.
2. FAILED -> CANCELLING
You cancel the execution, but by the time we try to set the status to
CANCELLING, the execution thread had already encountered an error,
and therefore, in FAILED state.
3. TERMINATED -> CANCELLED
Similar to #1, but with CANCELLED instead of CANCELLING.
4. FAILED -> CANCELLED
Similar to #1, but with CANCELLED instead of CANCELLING.
In all of the above cases (#1-#4), we skip updating the execution
status, and log that the execution already succeeded/failed before we
were able to cancel it.
5. CANCELLING -> STARTED
You cancel the execution while it is still in pending state. Meanwhile,
while the execution status was already set to CANCELLING, we try to set
the execution status
6. CANCELLED -> STARTED
Similar to #5, but after the status is set to CANCELLING, it also gets
set to CANCELLED before attempting to set it to STARTED.
In cases #5-#6, we skip updtating the execution status, and nothing
is logged.
Ran Ziv [Sun, 23 Apr 2017 12:39:57 +0000 (15:39 +0300)]
ARIA-150 Fixed simple hello-world example
The simple hello-world example had a missing host node,
which caused an error at service creation phase.
The example now uses custom types which do not require
the extra host node.
max-orlov [Wed, 19 Apr 2017 14:14:15 +0000 (17:14 +0300)]
ARIA-138-Make-logging-more-informative
Ran Ziv [Thu, 20 Apr 2017 12:00:30 +0000 (15:00 +0300)]
ARIA-145 deploying CLI config file at installation
max-orlov [Wed, 19 Apr 2017 18:31:08 +0000 (21:31 +0300)]
ARIA-142-defaults-missing-for-execution-plugin
Ran Ziv [Tue, 28 Mar 2017 09:17:46 +0000 (12:17 +0300)]
ARIA-48 Revamped ARIA CLI
This is a large commit which revolves mostly around creating the
new CLI, but is also tying ARIA's various components together
for real for the first time - allowing a complete run of the
"hello-world" example and more.
This commit introduces a few other important modules:
- aria/core.py - used for managing service-templates and services.
- aria/orchestator/workflow_runner.py - used for managing
a workflow execution on a service.
- aria/orchestrator/dry.py - a "dry executor", used for
dry-executing workflows and printing the tasks that would run.
Other fixes that were required for the successful usage of
ARIA end-to-end have also been introduced in this commit, but
there have been too many to list; Review the commit for more info.
Tal Liron [Fri, 24 Mar 2017 21:33:11 +0000 (16:33 -0500)]
ARIA-92 Automatic operation task configuration
Tal Liron [Tue, 11 Apr 2017 23:19:38 +0000 (18:19 -0500)]
ARIA-127 Make use of in-memory sqlite more robust
Tal Liron [Tue, 11 Apr 2017 20:26:43 +0000 (15:26 -0500)]
ARIA-128 Make test_engine more robust
Tal Liron [Fri, 31 Mar 2017 19:34:41 +0000 (14:34 -0500)]
ARIA-134 Parser populates service workflows
max-orlov [Tue, 4 Apr 2017 17:41:59 +0000 (20:41 +0300)]
ARIA-137-Support-for-predicate-based-queries-in-the-SQL-mapi
max-orlov [Tue, 4 Apr 2017 09:22:12 +0000 (12:22 +0300)]
ARIA-132-fixed-support-for-cascading-deletion
max-orlov [Sun, 2 Apr 2017 16:24:20 +0000 (19:24 +0300)]
ARIA-136-ctx-binary-doesnt-get-installed-via-pip-install
max-orlov [Sun, 2 Apr 2017 11:53:46 +0000 (14:53 +0300)]
ARIA-125-Filtering-returns-the-wrong-models
max-orlov [Thu, 30 Mar 2017 14:29:17 +0000 (17:29 +0300)]
ARIA-86-Create-a-basic-Hello-World-blueprint-example
max-orlov [Thu, 30 Mar 2017 10:56:36 +0000 (13:56 +0300)]
ARIA-135-Add-plugin-field-to-MockTask
Avia Efrat [Tue, 28 Mar 2017 09:51:42 +0000 (12:51 +0300)]
ARIA-133 Add status-related properties to the Execution, Task and Node models
We are adding these properties so it will be easier to filter those models
from storage according to their status, and to not make use of the their
`status` constants outside of the models themselves.
It should be noted, since we currently have the concept of `StubTasks`
(that are not part of the model), we had to implement the same logic for
them, although they don't direct access to the state constants of the
task model. We did this since they are treated as 'regular' model tasks
in some parts of the code.
max-orlov [Sun, 26 Mar 2017 11:13:47 +0000 (14:13 +0300)]
ARIA-132-Models-cascading-deletion-raises-constraint-errors
Additional fixes:
- The relationships are now defined on both sides. we no longer use backref, but back_populates.
max-orlov [Mon, 13 Mar 2017 14:35:10 +0000 (16:35 +0200)]
ARIA-120-Builtin-workflows-relationship-operations-execution-order
Avia Efrat [Wed, 22 Mar 2017 15:19:58 +0000 (17:19 +0200)]
ARIA-126 Add node states
1. The states are described in section 3.3.1 of the TOSCA spec.
2. The state is changed if a standard lifecycle operation runs of the
node, as described in sections 5.7.4.1, 5.7.4.4.1, and 5.7.4.4.2 of the TOSCA spec.
3. We did not address the 'error' state yet.
This state is defined as part of the possible node states in the model, but currently no execution path leads to setting a node's state to 'error'.
4. No validation of state transiontions.
For example, we do not validate if a node goes from 'created'
to 'started' without going through the 'configured' state in between.
Tal Liron [Wed, 22 Mar 2017 20:07:57 +0000 (15:07 -0500)]
ARIA-83 Support ARIA profile for TOSCA
max-orlov [Thu, 23 Mar 2017 10:48:44 +0000 (12:48 +0200)]
ARIA-131-Make-demand-for-networkx-more-strict
Avia Efrat [Thu, 9 Mar 2017 10:02:06 +0000 (12:02 +0200)]
ARIA-96 Improve our requirements handling
1. Currently, don't vendor any library.
2. Use pip-tools (specifically, pip-compile) to manage our dependencies
requirement files.
3. requirements.in:
The requirements.in will be read into `install_requires`, and includes
loose dependencies (as possible, and only direct dependencies), as is
common when installing using a setup.py file.
Since we haven't tested ARIA with many versions of our dependencies, our
dependencies are not very loose, but we are hoping to improve this as
our project matures.
Currently, when we provide an upper bound, it is either because of python
2.6 compatibility, or according to a semantic versioning (i.e. future
versions that could break the current API). Lower boundaries are
usually the lowest version that we tested with ARIA, or because version
before are lacking functionality we are using.
4. requirements.txt:
The requirements.txt is generated from requirements.in via pip-compile,
and includes fixed-version dependencies, including all transitive
dependencies, in order to provide an stable environment that ARIA is
ensured to work on.
max-orlov [Wed, 22 Mar 2017 17:19:53 +0000 (19:19 +0200)]
ARIA-130-Passing-inputs-from-the-workflow-to-the-operation-changes-the-inputs
Tal Liron [Fri, 17 Feb 2017 22:00:40 +0000 (16:00 -0600)]
ARIA-105 Integrate parser and orchestrator models
max-orlov [Mon, 6 Mar 2017 14:43:58 +0000 (16:43 +0200)]
ARIA-117-Log-model-should-have-an-Task-field
max-orlov [Sun, 5 Mar 2017 17:18:21 +0000 (19:18 +0200)]
ARIA-115-Log-model-should-have-an-Execution-field
max-orlov [Sun, 5 Mar 2017 16:31:36 +0000 (18:31 +0200)]
ARIA-116-Missing-type_hierarchy-in-relationship
mxmrlv [Mon, 13 Feb 2017 10:28:27 +0000 (12:28 +0200)]
ARIA-106-Create-sqla-logging-handler
Ran Ziv [Thu, 16 Feb 2017 14:38:51 +0000 (16:38 +0200)]
ARIA-108 Add API for deleting a resource for resource storage
Dan Kilman [Mon, 30 Jan 2017 14:49:00 +0000 (16:49 +0200)]
ARIA-79-concurrent-modifications
Tal Liron [Wed, 15 Feb 2017 18:48:56 +0000 (12:48 -0600)]
ARIA-88 Add TOSCA use-case examples
mxmrlv [Thu, 19 Jan 2017 09:39:36 +0000 (11:39 +0200)]
ARIA-44 Merge parser and storage model
Additional Changes:
* As a part of this task the name to address the model has been changes to the __tablename__ (or __mapiname__ if specified)
* All of the relationships return a query, so accessing the entire list should be done via all()
mxmrlv [Wed, 1 Feb 2017 14:16:01 +0000 (16:16 +0200)]
ARIA-42-Generic-ctx-serialization-mechanism
Tal Liron [Fri, 3 Feb 2017 19:32:46 +0000 (13:32 -0600)]
ARIA-99 Straightforward end-to-end tests for parser and built-in workflow
Ran Ziv [Thu, 2 Feb 2017 12:58:37 +0000 (14:58 +0200)]
ARIA-98 Updated two dependencies to support a range of versions
mxmrlv [Wed, 1 Feb 2017 17:26:27 +0000 (19:26 +0200)]
ARIA-95-pip-wheel-for-aria-ignores-dependencies
mxmrlv [Mon, 30 Jan 2017 16:24:15 +0000 (18:24 +0200)]
ARIA-81-Install-execution-plugin-ctx-entry-point-by-default
mxmrlv [Sun, 15 Jan 2017 17:07:29 +0000 (19:07 +0200)]
ARIA-66-Convert-custom-parser-fields-into-sqla-based-fields
Tal Liron [Wed, 25 Jan 2017 14:30:16 +0000 (16:30 +0200)]
ARIA-61 Initial version of CLI "workflow" command
Dan Kilman [Wed, 25 Jan 2017 14:22:41 +0000 (16:22 +0200)]
ARIA-36 plugin workdir
Tal Liron [Wed, 25 Jan 2017 13:25:33 +0000 (15:25 +0200)]
ARIA-73 Transfer parser's TOSCA model to storage
Tal Liron [Tue, 24 Jan 2017 15:57:50 +0000 (17:57 +0200)]
ARIA-12 New "Runner" class for simple workflow executions
Dan Kilman [Tue, 24 Jan 2017 14:58:10 +0000 (16:58 +0200)]
ARIA-68 Update runtime properties during operation
Tal Liron [Tue, 24 Jan 2017 07:52:36 +0000 (09:52 +0200)]
ARIA-72 Rework built-in workflows to match TOSCA normative lifecycle
Dan Kilman [Mon, 23 Jan 2017 14:55:41 +0000 (16:55 +0200)]
ARIA-74 Add process executor extension registration hooks
Dan Kilman [Sun, 22 Jan 2017 18:43:40 +0000 (20:43 +0200)]
ARIA-70 fix task plugin_name
Tal Liron [Sun, 22 Jan 2017 10:49:12 +0000 (12:49 +0200)]
ARIA-65 Add validation for relationship assignment type
Dan Kilman [Tue, 3 Jan 2017 15:00:46 +0000 (17:00 +0200)]
ARIA-46 Execution plugin
Dan Kilman [Sun, 15 Jan 2017 15:42:23 +0000 (17:42 +0200)]
ARIA-63 Implement attribute tracking for subprocesses
Tal Liron [Sun, 15 Jan 2017 14:45:35 +0000 (16:45 +0200)]
ARIA-60 Fix use of thread locals in parser context
Tal Liron [Sun, 8 Jan 2017 21:12:12 +0000 (23:12 +0200)]
ARIA-59 Inherit default interfaces in relationships
mxmrlv [Tue, 27 Dec 2016 14:42:57 +0000 (16:42 +0200)]
ARIA-52-Support-order-management-in-relationships
Dan Kilman [Thu, 5 Jan 2017 13:01:16 +0000 (15:01 +0200)]
ARIA-56 Implement ip property on node instance model
Dan Kilman [Thu, 5 Jan 2017 12:15:40 +0000 (14:15 +0200)]
ARIA-34 Adding get/download resource and render API (as jinja template)
Dan Kilman [Thu, 5 Jan 2017 11:00:01 +0000 (13:00 +0200)]
ARIA-55 Implement task retry and abort mechanism
mxmrlv [Thu, 5 Jan 2017 09:30:30 +0000 (11:30 +0200)]
ARIA-58-Storage-model-to_dict-creates-symbolic-links
As part of this commits, ARIA will use the builtin MutableDict and List
mxmrlv [Thu, 5 Jan 2017 09:32:08 +0000 (11:32 +0200)]
ARIA-57-Custom executions state transition support
As part of this commit, a new valid transition was added between cancelling
and force cancelling
Ran Ziv [Tue, 3 Jan 2017 12:41:01 +0000 (14:41 +0200)]
ARIA-53 Add badges to README file
mxmrlv [Sun, 11 Dec 2016 22:50:09 +0000 (00:50 +0200)]
ARIA-39-Genericize-storage-models
Dan Kilman [Mon, 19 Dec 2016 15:05:54 +0000 (17:05 +0200)]
ARIA-41 Provide (initial) means for serializing an operation context object
Dan Kilman [Sun, 27 Nov 2016 14:31:29 +0000 (16:31 +0200)]
ARIA-26 Implement operation plugin mechanism
Dan Kilman [Mon, 5 Dec 2016 13:28:29 +0000 (15:28 +0200)]
ARIA-31 Add registry mechanism for extensions