Commit 97fae162 authored by Christos Stavrakakis's avatar Christos Stavrakakis
Browse files

Merge branch 'master' into develop

Conflicts:
	ci/schemas/one_node_wheezy/wheezy.conf
	docs/admin-guide.rst
	snf-cyclades-app/synnefo/api/actions.py
	snf-cyclades-app/synnefo/api/servers.py
	snf-cyclades-app/synnefo/logic/backend.py
	snf-cyclades-app/synnefo/logic/callbacks.py
	snf-cyclades-app/synnefo/logic/rapi.py
	snf-cyclades-app/synnefo/logic/reconciliation.py
	snf-cyclades-gtools/synnefo/ganeti/eventd.py
	snf-deploy/conf/packages.conf
	snf-deploy/conf/wheezy.conf
	snf-deploy/snfdeploy/__init__.py
	version
parents b7bdf1ea 11e79c1c
......@@ -153,7 +153,7 @@ objects by domain attribute. This is used by Plankton for listing VM images.
v0.14.10
=======
Released: UNRELEASED
Released: Tue Nov 26 11:03:37 EET 2013
Cyclades
-------
......@@ -165,6 +165,7 @@ Cyclades
enable the use of this feature.
* Fix warning message while getting object permissions to appear only when
path is None and the object has permissions
* Add name to newly created NICs and the corresponding firewall tags.
.. _Changelog-0.14.9:
......
......@@ -5,6 +5,15 @@ Unified NEWS file for Synnefo versions >= 0.13
Since v0.13 all Synnefo components have been merged into a single repository.
.. _NEWS-0.14.10:
v0.14.10
=======
Released: Tue Nov 26 11:03:37 EET 2013
* Support for Ganeti 2.8
.. _NEWS-0.14.9:
v0.14.9
......
......@@ -40,12 +40,11 @@ snf-tools = wheezy
[ganeti]
# use latest package from wheezy repo:
# 2.8.2+snapshot1+b64v1+hotplug2+ippoolfix+rapifix-1~wheezy
snf-ganeti = wheezy
ganeti-htools = wheezy
ganeti-haskell = wheezy
[other]
snf-cloudcms = wheezy
snf-vncauthproxy = wheezy
......
......@@ -1710,6 +1710,7 @@ Upgrade Notes
v0.14 -> v0.14.2 <upgrade/upgrade-0.14.2>
v0.14.5 -> v0.14.6 <upgrade/upgrade-0.14.6>
v0.14.7 -> v0.14.8 <upgrade/upgrade-0.14.8>
v0.14.9 -> v0.14.10 <upgrade/upgrade-0.14.10>
v0.14 -> v0.15 <upgrade/upgrade-0.15>
......@@ -1717,6 +1718,7 @@ Changelog, NEWS
===============
* v0.14.10 :ref:`Changelog <Changelog-0.14.10>`, :ref:`NEWS <NEWS-0.14.10>`
* v0.14.9 :ref:`Changelog <Changelog-0.14.9>`, :ref:`NEWS <NEWS-0.14.9>`
* v0.14.8 :ref:`Changelog <Changelog-0.14.8>`, :ref:`NEWS <NEWS-0.14.8>`
* v0.14.7 :ref:`Changelog <Changelog-0.14.7>`, :ref:`NEWS <NEWS-0.14.7>`
......
Upgrade to Synnefo v0.14.10
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Synnefo v0.14.10 supports both Debian Squeeze and Wheezy. However, since
v0.14.10, Synnefo supports only Ganeti >= 2.8. This means that at least the
Ganeti nodes of a Synnefo deployment should run on Wheezy.
To upgrade to Synnefo v0.14.10 one needs to upgrade both Synnefo and Ganeti
during the same upgrade cycle, so some minimal service downtime is needed.
As always, VMs, Networks and Files will remain usable during the upgrade.
Since this is an upgrade to a minor version, no special upgrade operations
are needed except from the package upgrades.
1. Bring down the services
==========================
First, bring all services (Synnefo and Ganeti) down:
.. code-block:: console
root@astakos-host# /etc/init.d/gunicorn stop
root@cyclades-host# /etc/init.d/gunicorn stop
root@pithos-host# /etc/init.d/gunicorn stop
root@ganeti-master-host# /etc/init.d/ganeti stop
root@ganeti-master-host# /etc/init.d/snf-ganeti-eventd stop
root@ganeti-nodeX-host# /etc/init.d/ganeti stop
root@cyclades-host# /etc/init.d/snf-dispatcher stop
2. Upgrade Ganeti
=================
Once, everything is stopped, upgrade Ganeti following the official upgrade notes
found here:
`Ganeti upgrade notes <http://docs.ganeti.org/ganeti/2.8/html/upgrade.html>`_
In a nutshell:
Install packages
----------------
Install the new Ganeti packages. To be able to use hotplug (which will be part
of the official Ganeti 2.10), we recommend using our Ganeti packages with
version: ``snf-ganeti=2.8.2+snapshot1+b64v1+hotplug3+ippoolfix+rapifix+netxen+lockfix2-1~wheezy``
.. code-block:: console
root@ganeti-master-host# apt-get install snf-ganeti ganeti-htools ganeti-haskell
root@ganeti-nodeX-host# apt-get install snf-ganeti ganeti-htools ganeti-haskell
.. note:: Make sure you install all three Ganeti packages to all hosts.
Also all packages should have the same version.
Upgrade
-------
Upgrade Ganeti's configuration (make sure you do all backup and dry-run steps as
described in the official guide):
.. code-block:: console
root@ganeti-master-host# /usr/share/ganeti/cfgupgrade
Start Ganeti
------------
Start Ganeti and re-distribute the configuration to all Ganeti master candidates:
.. code-block:: console
root@ganeti-master-host# /etc/init.d/ganeti start
root@ganeti-nodeX-host# /etc/init.d/ganeti start
root@ganeti-master-host# gnt-cluster redist-conf
root@ganeti-master-host# /etc/init.d/ganeti stop
root@ganeti-nodeX-host# /etc/init.d/ganeti stop
3. Upgrade Synnefo
==================
Install packages
----------------
Install the new v0.14.10 packages on all hosts according to your deployment.
4. Start all services
=====================
Once, everything is installed successfully, start all services
(Synnefo and Ganeti):
.. code-block:: console
root@cyclades-host# /etc/init.d/snf-dispatcher start
root@ganeti-master-host# /etc/init.d/snf-ganeti-eventd start
root@ganeti-master-host# /etc/init.d/ganeti start
root@ganeti-nodeX-host# /etc/init.d/ganeti start
root@astakos-host# /etc/init.d/gunicorn start
root@cyclades-host# /etc/init.d/gunicorn start
root@pithos-host# /etc/init.d/gunicorn start
Upgrade to Synnefo v0.14.8
^^^^^^^^^^^^^^^^^^^^^^^^^^
Since v0.14.8, Synnefo ships an example Gunicorn configuration file, that gets
installed automatically at ``/etc/gunicorn.d/synnefo.example``.
To use it you need to do two things:
Synnefo v0.14.8 release introduced support for Debian Wheezy (and Django 1.4).
To upgrade from Squeeze to Wheezy you should make sure to change the
``'ENGINE'`` option in ``/etc/synnefo/10-snf-webproject-databases.conf`` to
``'django.db.backends.postgresql_psycopg2``. Replace ``postgresql_psycopg2``
with the DB engine you are using.
If you're upgrading to Synnefo v0.14.8 on Squeeze, you should also make sure to
have the Squezee backports repository installed, since ``snf-webproject``
depends on ``>=python-django-south-0.7.3``, which on Squeeze is only available
from the backports repository.
Since v0.14.8, Synnefo also ships an example Gunicorn configuration file, that
gets installed automatically at ``/etc/gunicorn.d/synnefo.example``. To use it
you need to do two things:
1. Disable your old configuration file by removing it.
[if you are upgrading from an older version and you had such a file]
......
......@@ -847,6 +847,11 @@ class SecurityGroup(models.Model):
name = models.CharField('group name',
max_length=SECURITY_GROUP_NAME_LENGTH)
@property
def backend_uuid(self):
"""Return the name of NIC in Ganeti."""
return "%snic-%s" % (settings.BACKEND_PREFIX_ID, str(self.id))
class PoolTable(models.Model):
available_map = models.TextField(default="", null=False)
......
......@@ -951,7 +951,7 @@ def connect_to_network(vm, nic):
"depends": depends,
}
if vm.backend.use_hotplug():
kwargs["hotplug"] = True
kwargs["hotplug_if_possible"] = True
if settings.TEST:
kwargs["dry_run"] = True
......@@ -967,7 +967,7 @@ def disconnect_from_network(vm, nic):
"nics": [("remove", nic.backend_uuid, {})],
}
if vm.backend.use_hotplug():
kwargs["hotplug"] = True
kwargs["hotplug_if_possible"] = True
if settings.TEST:
kwargs["dry_run"] = True
......
......@@ -37,7 +37,7 @@ from functools import wraps
from django.db import transaction
from synnefo.db.models import (Backend, VirtualMachine, Network,
BackendNetwork, pooled_rapi_client)
from synnefo.logic import utils, backend as backend_mod
from synnefo.logic import utils, backend as backend_mod, rapi
from synnefo.lib.utils import merge_time
......@@ -178,35 +178,41 @@ def update_db(vm, msg, event_time):
logmsg = msg["logmsg"]
nics = msg.get("instance_nics", None)
job_fields = msg.get("job_fields", {})
result = msg.get("result", [])
# Special case: OP_INSTANCE_CREATE with opportunistic locking may fail
# if all Ganeti nodes are already locked. Retry the job without
# opportunistic locking..
if (operation == "OP_INSTANCE_CREATE" and status == "error" and
job_fields.get("opportunistic_locking", False)):
if vm.backendjobid != jobID: # The job has already been retried!
try:
error_code = result[1][1]
except IndexError:
error_code = None
if error_code == rapi.ECODE_TEMP_NORES:
if vm.backendjobid != jobID: # The job has already been retried!
return
# Remove extra fields
[job_fields.pop(f) for f in ("OP_ID", "reason")]
# Remove 'pnode' and 'snode' if they were set by Ganeti iallocator.
# Ganeti will fail if both allocator and nodes are specified.
allocator = job_fields.pop("iallocator")
if allocator is not None:
[job_fields.pop(f) for f in ("pnode", "snode")]
name = job_fields.pop("name", job_fields.pop("instance_name"))
# Turn off opportunistic locking before retrying the job
job_fields["opportunistic_locking"] = False
with pooled_rapi_client(vm) as c:
jobID = c.CreateInstance(name=name, **job_fields)
# Update the VM fields
vm.backendjobid = jobID
# Update the task_job_id for commissions
vm.task_job_id = jobID
vm.backendjobstatus = None
vm.save()
log.info("Retrying failed creation of instance '%s' without"
" opportunistic locking. New job ID: '%s'", name, jobID)
return
# Remove extra fields
[job_fields.pop(f) for f in ("OP_ID", "reason")]
# Remove 'pnode' and 'snode' if they were set by Ganeti iallocator.
# Ganeti will fail if both allocator and nodes are specified.
allocator = job_fields.pop("iallocator")
if allocator is not None:
[job_fields.pop(f) for f in ("pnode", "snode")]
name = job_fields.pop("name", job_fields.pop("instance_name"))
# Turn off opportunistic locking before retrying the job
job_fields["opportunistic_locking"] = False
with pooled_rapi_client(vm) as c:
jobID = c.CreateInstance(name=name, **job_fields)
# Update the VM fields
vm.backendjobid = jobID
# Update the task_job_id for commissions
vm.task_job_id = jobID
vm.backendjobstatus = None
vm.save()
log.info("Retrying failed creation of instance '%s' without"
" opportunistic locking. New job ID: '%s'", name, jobID)
return
backend_mod.process_op_status(vm, event_time, jobID,
operation, status,
......
......@@ -95,6 +95,11 @@ _INST_REINSTALL_REQV1 = INST_REINSTALL_REQV1
_NODE_MIGRATE_REQV1 = NODE_MIGRATE_REQV1
_NODE_EVAC_RES1 = NODE_EVAC_RES1
#: Not enough resources (iallocator failure, disk space, memory, etc.)
ECODE_NORES = "insufficient_resources"
#: Temporarily out of resources; operation can be tried again
ECODE_TEMP_NORES = "temp_insufficient_resources"
class Error(Exception):
......
#!/usr/bin/env python
"""Tool to update Ganeti instances:
* add unique name to the NICs of all Ganeti instances
* rename all instance tags related with network firewall profiles to include
the unique name of the corresponding NIC.
The name for each NIC is based on the PK of the NIC in Cyclades DB.
"""
FIREWALL_TAGS_PREFIX = "synnefo:network:"
FIREWALL_TAGS = {"ENABLED": "synnefo:network:%s:protected",
"DISABLED": "synnefo:network:%s:unprotected",
"PROTECTED": "synnefo:network:%s:limited"}
# Gevent patching
import gevent
from gevent import monkey
monkey.patch_all()
import sys
import subprocess
from optparse import OptionParser, TitledHelpFormatter
# Configure Django env
from synnefo import settings
from django.core.management import setup_environ
setup_environ(settings)
from django.db import close_connection
from synnefo.db.models import Backend, pooled_rapi_client
from synnefo.management.common import get_backend
import logging
logger = logging.getLogger("migrate_nics")
handler = logging.StreamHandler()
formatter = logging.Formatter("[%(levelname)s] %(message)s")
handler.setFormatter(formatter)
logger.setLevel(logging.DEBUG)
logger.addHandler(handler)
logger.propagate = False
DESCRIPTION = """\
Tool to update all Ganeti instances in order to add a unique name to NICs of
all instances and rename the instance firewall tags to include the NIC name.
"""
def main():
parser = OptionParser(description=DESCRIPTION,
formatter=TitledHelpFormatter())
parser.add_option("--backend-id", dest="backend_id",
help="Update instances only of this Ganeti backend."),
parser.add_option("--dry-run", dest="dry_run", default=False,
action="store_true",
help="Do not send any jobs to Ganeti backend.")
parser.add_option("--ganeti-dry-run", dest="ganeti_dry_run", default=False,
action="store_true",
help="Pass --dry-run option to Ganeti jobs.")
parser.add_option("--parallel", dest="parallel", default=False,
action="store_true",
help="Use a seperate process for each backend.")
parser.add_option("-d", "--debug", dest="debug", default=False,
action="store_true",
help="Display debug information.")
options, args = parser.parse_args()
if options.backend_id:
backends = [get_backend(options.backend_id)]
else:
if Backend.objects.filter(offline=True).exists():
msg = "Can not update intances. An 'offline' backend exists."
raise Exception(msg)
backends = Backend.objects.all()
if options.debug:
logger.setLevel(logging.DEBUG)
if len(backends) > 1 and options.parallel:
cmd = sys.argv
processes = []
for backend in backends:
p = subprocess.Popen(cmd + ["--backend-id=%s" % backend.id])
processes.append(p)
for p in processes:
p.wait()
return
else:
[upgrade_backend(b, options.dry_run, options.ganeti_dry_run)
for b in backends]
return
def upgrade_backend(backend, dry_run, ganeti_dry_run):
jobs = []
instances_ids = get_instances_with_anonymous_nics(backend)
for vm in backend.virtual_machines.filter(id__in=instances_ids):
jobs.append(gevent.spawn(upgrade_vm, vm, dry_run, ganeti_dry_run))
if jobs:
for job_chunk in [jobs[x:x+25] for x in range(0, len(jobs), 25)]:
gevent.joinall(jobs)
else:
logger.info("No anonymous NICs in backend '%s'. Nothing to do!",
backend.clustername)
return
def get_instances_with_anonymous_nics(backend):
"""Get all Ganeti instances that have NICs without names."""
with pooled_rapi_client(backend) as rc:
instances = rc.GetInstances(bulk=True)
# Filter snf- instances
instances = filter(lambda i:
i["name"].startswith(settings.BACKEND_PREFIX_ID),
instances)
# Filter instances with anonymous NICs
instances = filter(lambda i: None in i["nic.names"], instances)
# Get IDs of those instances
instances_ids = map(lambda i:
i["name"].replace(settings.BACKEND_PREFIX_ID, "", 1),
instances)
return instances_ids
def upgrade_vm(vm, dry_run, ganeti_dry_run):
"""Add names to Ganeti NICs and update firewall Tags."""
logger.info("Updating NICs of instance %s" % vm.backend_vm_id)
index_to_uuid = {}
new_tags = []
# Compute new NICs names and firewall tags
for nic in vm.nics.all():
if nic.index is None:
msg = ("Cannot update NIC '%s'. The index of the NIC is unknown."
" Please run snf-manage reconcile-servers --fix-all and"
" retry!")
logger.critical(msg)
continue
uuid = nic.backend_uuid
# Map index -> UUID
index_to_uuid[nic.index] = uuid
# New firewall TAG with UUID
firewall_profile = nic.firewall_profile
if firewall_profile and firewall_profile != "DISABLED":
firewall_tag = FIREWALL_TAGS[nic.firewall_profile] % uuid
new_tags.append(firewall_tag)
renamed_nics = [("modify", index, {"name": name})
for index, name in index_to_uuid.items()]
instance = vm.backend_vm_id
with pooled_rapi_client(vm) as rc:
# Delete old Tags
tags = rc.GetInstanceTags(instance)
delete_tags = [t for t in tags if t.startswith(FIREWALL_TAGS_PREFIX)]
if delete_tags:
logger.debug("Deleting tags '%s' from instance '%s'",
delete_tags, vm.backend_vm_id)
if not dry_run:
rc.DeleteInstanceTags(instance, delete_tags,
dry_run=ganeti_dry_run)
# Add new Tags
if new_tags:
logger.debug("Adding new tags '%s' to instance '%s'",
new_tags, vm.backend_vm_id)
if not dry_run:
rc.AddInstanceTags(instance, new_tags, dry_run=ganeti_dry_run)
# Add names to NICs
logger.debug("Modifying NICs of instance '%s'. New NICs: '%s'",
vm.backend_vm_id, renamed_nics)
if not dry_run:
rc.ModifyInstance(vm.backend_vm_id,
nics=renamed_nics, dry_run=ganeti_dry_run)
close_connection()
if __name__ == "__main__":
main()
sys.exit(0)
......@@ -45,6 +45,11 @@ import sys
import os
path = os.path.normpath(os.path.join(os.getcwd(), '..'))
sys.path.append(path)
# Since Ganeti 2.7, debian package ships the majority of the python code in
# a private module under '/usr/share/ganeti'. Add this directory to path
# in order to be able to import ganeti. Also, add it to the start of path
# to allow conflicts with Ganeti RAPI client.
sys.path.insert(0, "/usr/share/ganeti")
import json
import logging
......@@ -120,7 +125,7 @@ def get_instance_nics(instance, logger):
nics = map(lambda x: dict(zip(nic_keys, x)), nics)
except ganeti_errors.OpPrereqError:
# Not running on master! Load the conf file
raw_data = utils.ReadFile(constants.CLUSTER_CONF_FILE)
raw_data = utils.ReadFile(pathutils.CLUSTER_CONF_FILE)
config = serializer.LoadJson(raw_data)
i = config["instances"][instance]
nics = []
......@@ -236,6 +241,7 @@ class JobFileHandler(pyinotify.ProcessEvent):
"status": op.status,
"cluster": self.cluster_name,
"logmsg": logmsg,
"result": op.result,
"jobId": job_id})
if op.status == "success":
......
......@@ -40,8 +40,6 @@ snf-tools = wheezy
[ganeti]
# use latest package from wheezy repo:
# 2.8.2+snapshot1+b64v1+hotplug2+ippoolfix+rapifix-1~wheezy
snf-ganeti = wheezy
ganeti-htools = wheezy
ganeti-haskell = wheezy
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment