GitLab

* Remove util/dictconfig.py as it's a duplicate of lib/dictconfig.py

Networks API is already documented in a separate file.

Until now, burnin was using a 'feature' version of kamaki
where the 'create_floatingip' method had a 'project' parameter.
This parameter was renamed to 'project_id' for uniformity with the
other kamaki methods.

These changes have been merged to 'develop' branch of kamaki and the API
is not expected to be changed again. This patch changes the way burnin
invokes 'create_floatingip' to use the 'project_id' parameter instead of
the 'project' one.

Rename snf-dispatcher's default PID file from 'snf_dispatcher.pid' to
'snf-dispatcher.pid'.

All Synnefo's management commands are written as custom django-admin commands.
This means that every management command is in fact a class that extends
Django's BaseCommand class.

Django's BaseCommand provides the attributes self.stdout and
self.stderr and Django's documentation encourages the users to use these
attributes if they wish to write to the console. Django doesn't provide an
option to write the output to files and the user has to implement this
explicitly when implementing the handle method.

We would like to extend the above mechanism to allow every snf-manage
command to log all stdout and stderr output on a unique filename under a given
directory. The implementation should change nothing in the way that users write
management commands (only acceptable change is that the new commands may have
to inherit a new class and not the BaseCommand one). This means that
existing management commands should play out of the box and also that the
logging mechanism will globally apply to all of them.

Commands that do not alter the state of the server (i.e. *-list and *-show
commands) will be excluded from the logging mechanism.

Add an entry to the Changelog about the 'Logging mechanism for Synnefo
management commands'.

Closes #3: Log all stdout/stderr for snf-manage invocations

Create 'SynnefoOutputWrapper' which replaces Django's 'OutputWrapper'
and logs the command and its output in a file.

Ref #3: Log all stdout/stderr for snf-manage invocations

'LOG_DIR' is the directory where log files are saved and is going to be
used to determine where to store the management commands' output.

Ref #3: Log all stdout/stderr for snf-manage invocations

When StreamHandler writes a formatted log message to its stream, it adds
a newline terminator. This behavior is inherited by FileHandler and the
other classes which derive from it (such as the rotating file handlers).

Starting with Python 3.2, the message terminator will be configurable.
This has been done by adding a terminator attribute to StreamHandler,
which when emitting an event now writes the formatted message to its
stream first, and then writes the terminator. If you don't want newline
termination for a handler, just set the handler instance's terminator
attribute to the empty string.

This class implements python's 3.2 StreamHandler.

Ref #3: Log all stdout/stderr for snf-manage invocations

Instead of django's BaseCommand class, use out SynnefoCommand which
takes care of logging the command and its output.

Ref #3: Log all stdout/stderr for snf-manage invocations

From Django's documentation:
  When you are using management commands and wish to provide console
  output, you should write to self.stdout and self.stderr, instead of
  printing to stdout and stderr directly.

This patch fixes the managements commands to use the proper
stdout/stderr objects.

Ref #3: Log all stdout/stderr for snf-manage invocations

Design doc for the implementation of a logging mechanism for the
Synnefo's management commands.

Ref #3: Log all stdout/stderr for snf-manage invocations

Update 'MANIFEST.in' files, and fix a bug where 'recursive-include'
doesn't recognize directories with a trailing '/'.

Unify the string listing that is used to describe the arguments that are
accepted by snf-manage commands, and are displayed in the 'usage' of the
commands' help message. Also, add the argument in some commands that it
was missing.

Tool for checking status of Cyclades update path.

This patch series implements a tool for checking the status of Cyclades update path, which
includes Ganeti, AMQP, snf-ganeti-eventd and snf-dispatcher. The tool is
created as part of the snf-dispatcher and can be used by passing the
'--status-check' option. Also, snf-ganeti-eventd is extended in order to send
heartbeat messages.

Create a tool for checking the status of Cyclades update path, which
includes Ganeti, AMQP, snf-ganeti-eventd and snf-dispatcher. The tool is
created as part of the snf-dispatcher and can be used by passing the
'--status-check' option.

The tool sends a 'status-check' message to the running snf-dispatcher
process (via AMQP) and then waits to get the status report from an
exclusive queue. Respectively, the snf-dispatcher uses the Ganeti RAPI
client to add a special tag to each Ganeti cluster (in dry-run mode),
which will trigger snf-ganeti-eventd to send a heartbeat message to the
AMQP broker. Dispatcher collects all these message and sends the status
report back to the tool that triggered the status check.

Closes #23

Add a description to snf-dispatcher's help message and improve the help
message of some options.

Make 'snf-ganeti-eventd' daemon to send heartbeat messages which can
be used to check that the daemon is working correctly, i.e. that it is watching
the Ganeti queue and that it can send messages to the AMQP broker. The
daemon is triggered to send a heartbeat message by setting a special tag
to the Ganeti cluster in 'dry-run' mode.

Heartbeat messages are send with the 'eventd.heartbeat' routing key.

Refs Issue #23

Enable the Synnefo AMQP clients to consume messages from the queues
without requiring acknowledgments.

Also, add the missing 'exclusive' argument in 'AMQPHaighaClient'.

Extend the AMQP clients to support RabbitMQ's 'Queue TTL' extension, by
passing the 'x-expires' argument when declaring a queue. This argument
controls for how long a queue can be unused before it is automatically
deleted.

* Remove inflect as a dependency from snf-astakos-app
* Remove unused code that imported inflect

Closes #20

* pep8 fixes on astakos views

* Remove python-prctl as a dependency from snf-cyclades-gtools
* Remove pycurl as a dependency from snf-cyclades-app
* Update date on copyright notice on setup.py

* pep8 fixes in setup.py of snf-cyclades-gtools
* Update date on copyright notice

Add option '--pull-request' to snf-ci. This option gets a Github
pull-request url (e.g. 'https://github.com/grnet/synnefo/pull/id') and
runs the testsuite in a sophisticated way.

Sophisticated means that it will not just check the remote branch from
which the pull request originated. Instead it will checkout the branch
for which the pull request is indented (e.g. grnet:develop) and apply
the pull request over it. This way it checks the pull request against
the branch this pull request targets.

snf-ci used to find the schema files in the local repo and upload them
to the testing server. This is wrong since one can choose to test a
different branch that the one he has currently checked out, so snf-ci
will use the wrong schema files.

This patch fixes the above problem by instructing snf-ci to use the
schema files it finds in the repo cloned inside the testing server.

..and replace it with the `fabfile2.py`.

Update Copyright dates.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>

In case `keygen` founds the keys it does nothing. If `--force` is
passed then it re-creates the keys.

Currently ci installs the deploy package during `build` command.
Move keygen action from ci's `deploy_synnefo` phase to the
`build_synnefo` phase.

Do not use the `--force` flag so that deploy can be re-entrant on
any level.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>

Introduce new setting in [dirs] section of deploy.conf: `state`.
Override this setting with `--state-dir` option (useful when running
deploy from source).

This dir is currently used to store snf-deploy's `snf_deploy_status`
file, that shows which components on which nodes have been
installed. Do not update this file if `--dry-run` is used.

Additionally introduce `--templates-dir` option that overrides
`template` setting in [dirs] section of `deploy.conf`.

Note that override options do not modify the configuration files.
Changes are performed in the execution context of each snf-deploy
command.

To run snf-deploy from source use:

$ python setup.py develop
$ snf-deploy keygen -c conf -t files -s /tmp
$ snf-deploy all --autoconf -c conf -t files -s /tmp

Add the above setting to snf-ci's schema files.

Update Copyright dates.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>

These dirs are needed to store the ddns and .ssh files created by
snf-deploy keygen. All these files should reside in the template dir
since are going to be moved to target nodes.

Add those dirs in .gitignore since we don't want to track any temp
files created if we run snf-deploy from source.

Update Copyright dates.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>

The API provides port translation for every server. Use this to
determine the ssh port of 'demo.synnefo.org' and replace the hard
coded values in the code.

Closes #15

The uninstall option uses the "--uninstall" switch of `python setup.py
develop`. The behavior of the "--uninstall" switch can be found in the
setuptools docs [1].

[1] https://pythonhosted.org/setuptools/setuptools.html#develop

Add an option to connect to a specific ssh port of the created VM.
It is mainly useful when running ci from a VM in demo.synnefo.org. In
this case, we don't want to deduct the ssh port from the returned
server IP since the demo's DNAT applies only to connections out of the
demo's private network.