GitLab

Remove the ARCHIPELAGO_BACKENDS setting. Instead allocate VMs based on
the disk-templates that are enabled and allowed in ipolicy for each
backend. The command 'snf-manage backend-update-status' can be used to
update the list of enabled disk templates for each backend.

The command has been changed to no longer take any arguments or options.
Instead, this command always updates the resources and the enabled disk
templates of all online Ganeti backends.

Extend 'Backend' models with 'disk_template' attribute, that contains
the list of disk templates that are available in each backend.

Add custom field for storing a list of values in DB. The values
are stored as a TextField separated by a delimiter.

The server_command decorator commits the just created object and retrieves
it back from the database. Return the latter, because the former is stale
in memory.

Catch exception when talking to Ganeti inside 'server_command',
otherwise the pending commission will be rejected. Also, accept the
commission in case of build, because the VM will be created in DB.

Commit #f2080d1 mocked an OP_INSTANCE_REMOVE when enqueuing an
OP_INSTANCE_CREATE job to Ganeti failed. However, this may result in
releasing IPs that may get reserved in Ganeti, because we can never
really know if the job has reached Ganeti or not (e.g.  timeout).  For
this reason, we should *never* delete the VM. Instead we mark the VM as
being in error state.  The user must explicitly delete the server.  This
has the downside, that we return 202 with the resulting VM being in
ERROR state. However, this is the only way to be really secure.

Create 'quotas.handle_resource_commission' function for handling
commissions for VMs and Networks.

Split the server creation command in more steps in order to be
decorated with the 'server_command' function and follow the logic of all
other server commands. The main change is that the commission job is not
accepted when the VM is stored in DB, but when the OP_INSTANCE_CREATE
job finishes in Ganeti, no matter whether it succeeded or not. Finally,
this commit includes a transaction testcase that checks instance
creation.

Code from file 'api/servers.py' has been moved to 'logic/servers.py'.
During the merge of hotfix-0.14.6 the develop, this code was lost. This
commit applies commits #f2080d1, #ea9b0fb, #32a0b85  to the new file.

when helpdesk users search using ip address

In case creating a VM fails before the OP_INSTANCE_CREATE job is
enqueued in Ganeti, the VM and its NICs must be deleted, and the
resources must be released from Quotaholder, which is exactly what is
done when a VM is deleted from Ganeti. Instead of duplicating the code,
this commit uses the same function that is used by snf-dispatcher, by
mocking an successfully OP_INSTANCE_REMOVE.

Commit #32a0b85, modified server creation, by refetching the object from
DB after issuing commission to quotaholder. However, the new object is
not having the 'config_url' attributed, which is added to the vm object
by the 'vmapi'. This commit, fixes this by moving the signal dispatching
code after refetching the vm. Also, this commit makes more clear the
exception handling code, when the creation job can not be enqueued to
Ganeti.

GanetiRapiClient uses PyCurl, a Python interface to libcurl, and
performs blocking requests, so does not work well with gevent. This
commit makes GanetiRapiClient to use 'Requests' HTTP library that is
greenlet-safe.

Set backend to None if not specified.

FOOTER_EXTRA_MESSAGE setting appears above Copyright message at the
Compute templates and the Dashboard UI. Accepts html tags.

Simplify code by removing unnecessary temp variable.

Call to issue_and_accept_commission saves the resource object (eg a vm)
and commits the transaction. This has the side-effect that the resource
object becomes stale: its lock is released and another thread may acquire
it and update the db record. It is thus not safe to further update and then
save the stale object, since this would override others' changes. Instead,
lock and fetch the object again, where needed.

Ensure that a commission to release vm resources is issued only once.
If a vm is marked `deleted', then resources must have already been
released.

Remove trailing slash from the endpoints' publicURL field in service
definitions. These values end up in the response of POST /tokens, and,
according to the keystone API, should not end in slash. A trailing slash
breaks some clients, which do no proper checking before appending a
suffix.

Note that these URLs are not meant to act as a base path in the strict
semantics of http://tools.ietf.org/html/rfc3986#section-5.2.3 . Suffixes
should be merged into by a simple append (modulo catering for the joining
slash).

URLs are still configured as groups per prefix ending in a slash, eg:
'^account/' and '^ui/'. Permanent redirects are added for all non-API
endpoints (ui, helpdesk, admin, etc), so that `BASE_PATH/path' leads to
`BASE_PATH/path/'.

VMs at building error that do not exist in Ganeti must not be reported
as stale. The VM is stale only if the user has asked to destroy the VM,
and so the action is DESTROY.

Make snf-ganeti-eventd send the NICs of the instance not only on
success, but also if the job has failed or has been canceled. Also,
update snf-dispatcher to handle those messages. This step is required to
clear BUILDING NICs that fail.

Create NICs in building state before enqueuing corresponding jobs to
Ganeti. This has already been done when creating the instance, but not
when connecting it to other networks.

Change 'index' field of NIC model to be optional, because the index of
building NICs is unknown. Also, change all API responses to return only
the active NICs.

This commit reverts commits #07602322 and #30514662 which introduced a
bug in dispatcher's logic. These commit fixed the issue that, in a case
the Cyclades DB is unsynced with Ganeti, a resource that does not exist
in the Ganeti backend, it  will not be deleted from DB in case the
remove job fails.

This commits fixes this issue, by quering Ganeti each time a failed
OP_INSTANCE_REMOVE or OP_NETWORK_REMOVE is processed.

'type' attribute of reboot OS API call was directly mapped to the
'reboot_type' option of Ganeti reboot job. However these two are
semantically different. The 'SOFT' and 'HARD' reboot types of OS API are
mapped to 'shutdown_timeout' option of Ganeti jobs. However, setting
this option from Ganeti RAPI is not supported. Until implemented, we
will map both types to 'hard' ganeti reboot, and fallback to default
shutdown timeout of Ganeti (120s).

Also, modify UI to not set the 'HARD' reboot type and fallback to
default option of API which is 'SOFT'.

Fix passing of hotplug argument to GanetiRAPIClient, in case the
GANETI_USE_HOTPLUG is unset. If the Ganeti version does not support
hotplug, passing hotplug=False, will make the jobs fail at Ganeti opcode
validation. This commit fixes this by passing the argument, only when
the setting is set to True.

Commit #c172f4a used the "size" attribute of images. This commit
fixes mocked images of tests to contain this attribute.

Update Cyclades, to pass as 'img_id' the hashmap('checksum') of the
image, instead of the location URL. In that way, snf-image does not any
access to the Pithos database.

In case hotplug is disabled, via GANETI_USE_HOTPLUG setting, network
actions should be allowed only on STOPPED vms. This commit adds this
check and a unittest for this case. Also it makes some other checks
regarding actions and states of vms more strict.

Finally, this commit changes the default value of GANETI_USE_HOTPLUG to
True.