GitLab

Since the size of IP pools is not automatically computed by the size
of corresponding subnet, the data migration must add sizes to all
existing IP pools.

Before implementing the Neutron API, each network had one IP pool that
contained the whole subnet CIDR. This changed since now we have multiple
IP Pools(with base and offset attributes) per network and this IP pools
do not contain the network, broadcast and gateway addreeses. This commit
fixes an old migration to add 'base' and 'offset' attribute to all IP
pool objects, and adds an extra step to old and not-initialized pools in
order to externally reserve those addresses.

Migrate attributes from old Network model to new Subnet model.

Add Subnet model containing Layer-3 characteristics for networks. Also,
extend IPPoolTable object to hold a reference to the Subnet that the
pool belongs to.
Signed-off-by: Dionysis Grigoropoulos <dgrig@grnet.gr>

Add a unique constraint to NetworkInterface model. This constraint
asserts that the same IPv4 address from the same network will no be
assigned to more than one NICs.

Extend 'Backend' models with 'disk_template' attribute, that contains
the list of disk templates that are available in each backend.

Change 'index' field of NIC model to be optional, because the index of
building NICs is unknown. Also, change all API responses to return only
the active NICs.

Add migration file for floating IPs and VM task state.

Extend Network model with the 'drained' attribute, to indicate whether a
network should be used in public IP allocation when creating servers.
This option is only useful for public networks.

Support Ganeti backends with Xen hypervisor:
* Extend 'Backend' model with the 'hypervisor' attribute, which
  currently can be 'kvm' or 'xen'. Use 'kvm' as default value to
  easy database migration. Also add the corresponding schema migration.
* Do not use hotplug at Xen backends, since it is not
  implemented.
* Support different 'hvparams' for Xen and KVM backends. Update
  GANETI_CREATEINSTNACE_KWARGS setting to hold 'hvparams' for both
  Xen and KVM.
* When getting instance console, check 'serial_console' hvparam only
  for Xen instances.

Refs #3772

* Merge 'accepted' and 'rejected' field to 'accept' field.
* Add 'resolved' field, indicating whether this commission has been
  accepted/rejected to Quotaholder
* Remove custom save() method. Handle 'pending' field manually.

Also add the corresponding database migrations.

Use "ACTIVE" as NIC's default state, to avoid addding a data migration.

Until now, a server's public NetworkInterface was not created in the DB
until the arrival of the message from Ganeti hooks. However, the IPv4
address of the NIC is reserved before sending the OP_INSTANCE_CREATE
job to Ganeti.

This drawback of this approach is that if the Ganeti hooks never run
(building ERROR), Cyclades will never associate the IPv4 address with
a machine and therefore this address will never be released.

This commit fixes this issue by creating the VM's public NIC in DB when
creating the server. Also the NetworkInterface model is extended with
a 'state' field to indicate if the NIC is active or still building.
Finally, the corresponding API calls are modified in order to return
only 'ACTIVE' NICs.

Initial commit for VirtualMachineDiagnostics model and the corresponding
south migrations script.

Fix south migration files. In data migrations, access the Models from
the orm, instead importing Models from synnefo.db. This is necessary as
the orm provides access to the version of the models that existed when
the migration file was created.

- Extend the Network model with fields for holding the IPv6 subnet and
  gateway.
- Update network API methods.
- Modify ganeti hooks to return IPv6 address for all network interfaces.
  IPv6 is now derived from the IPv6 subnet of the network in Ganeti,
  instead of the old setting (PUBLIC_IPV6_PREFIX)
- Remove obsolete setting (`PUBLIC_IPV6_PREFIX`)
- Update management commands for IPv6

Use cipher to encrypt the passwords of the Ganeti backends in the DB.
Encryption/decryption is based on a configurable key, defined by the
ENCRYPTION_KEY setting. After modifying this setting, synnefo will be
unable to decrypt already stored passwords, and passwords must be
re-entered, using 'snf-manage backend-modify' command and restaring the
server. A migration file is responsible for initially encrypting already
stored passwords.

Extend NetworkInterface model with a dirty bit, indicating whether
the index of the NIC is consistent with the one in Ganeti. This flag is
set when deleting a NIC, and is switched off after processing the
notification from Ganeti. Removing a NIC with the dirty bit on is not
allowed.

Since Ganeti version 2.6, Ganeti is aware of private networks. We take
advantage of this by propagating network handling to Ganeti. Network
model is expanded to include information about the type of network
(public routed, physical vlan, private filtered), subnet, gateway, dhcp,
link and mac_prefix.

- Update Network Model and delete NetworkLink model
- Create MacPrefixPool and BridgePool models
- Default constant settings for MacPrefix management

- Create Backend model holding info about credentials, status and resources
- Add Backend as foreign key to VirtualMachine
- Create a cache of Backend clients (RAPI) for better performance

Update all modules which are interacting with the RabbitMQ messaging system to
use the AMQPClient in order to use highly available queues in active/active
mode.

Extend messages to include a unique identifier in order to handle
duplication of messages and out of order delivery. The timestamp of the event
that the message is describing is chosen for this.

Extend VirtualMachine model with 'backendtime' field to hold the
timestamp described above.

Fixes some flavor constrains issues

* Added middleware to authenticate with Astakos
* Removed more legacy code
* Deleted Image and ImageMetadata models

Tests and admin still pending.

Also remove aai and helpdesk apps

since more than one packages exist in repository root

list_images now returns images that either match the owner, or
that are marked as public.

Fixes #555

Database migration needed.

Merge branches api-current and ui-0.4:
    * Fix merge conflicts
    * Fix minor bug in reconciliation mgmt command
    * Add comments in settings.py.dist
    * minor PEP8 fixes

Refs #409