Commit 547a6802 authored by Giorgos Korfiatis's avatar Giorgos Korfiatis
Browse files

Merge branch 'feature-component-base-url' into develop

parents 3023abfd b9c39885
......@@ -30,6 +30,18 @@ Astakos
* Implement API calls for projects.
* Store the base URL of a component. Deployer should provide it when adding
a new component. Service endpoints originating from a component are
expected to match its base URL; otherwise, a warning is issued.
Re-registration with `snf-component-register' affects both the base and
the ui URL.
* Management commands:
* Introduced new commands:
* component-show
* Changed commands:
* component-add got options --base-url and --ui-url
Cyclades
--------
......
......@@ -1066,6 +1066,7 @@ commission-show Show details for a pending commission
component-add Register a component
component-list List components
component-modify Modify component attributes
component-show Show component details
project-control Manage projects and applications
project-list List projects
project-show Show project details
......@@ -1839,6 +1840,7 @@ Upgrade Notes
v0.13 -> v0.14 <upgrade/upgrade-0.14>
v0.14 -> v0.14.2 <upgrade/upgrade-0.14.2>
v0.14.5 -> v0.14.6 <upgrade/upgrade-0.14.6>
v0.14 -> v0.15 <upgrade/upgrade-0.15>
Changelog, NEWS
......
......@@ -917,9 +917,9 @@ offered by the services.
.. code-block:: console
astakos-host$ snf-manage component-add astakos astakos_ui_url
astakos-host$ snf-manage component-add cyclades cyclades_ui_url
astakos-host$ snf-manage component-add pithos pithos_ui_url
astakos-host$ snf-manage component-add astakos --base-url astakos_base_url --ui-url astakos_ui_url
astakos-host$ snf-manage component-add cyclades --base-url cyclades_base_url --ui-url cyclades_ui_url
astakos-host$ snf-manage component-add pithos --base-url pithos_base_url --ui-url pithos_ui_url
astakos-host$ snf-manage service-export-astakos > astakos.json
astakos-host$ snf-manage service-import --json astakos.json
cyclades-host$ snf-manage service-export-cyclades > cyclades.json
......
Upgrade to Synnefo v0.15
^^^^^^^^^^^^^^^^^^^^^^^^
The upgrade to v0.15 consists in the following steps:
1. Bring down services and backup databases.
2. Upgrade packages, migrate the databases and configure settings.
3. Re-register components and services in astakos.
4. Bring up all services.
.. warning::
It is strongly suggested that you keep separate database backups
for each service after the completion of each step.
1. Bring web services down, backup databases
============================================
1. All web services must be brought down so that the database maintains a
predictable and consistent state during the migration process::
$ service gunicorn stop
$ service snf-dispatcher stop
$ service snf-ganeti-eventd stop
2. Backup databases for recovery to a pre-migration state.
3. Keep the database servers running during the migration process.
2. Upgrade Synnefo and configure settings
=========================================
2.1 Install the new versions of packages
----------------------------------------
::
astakos.host$ apt-get install \
python-objpool \
snf-common \
python-astakosclient \
snf-django-lib \
snf-webproject \
snf-branding \
snf-astakos-app
cyclades.host$ apt-get install \
python-objpool \
snf-common \
python-astakosclient \
snf-django-lib \
snf-webproject \
snf-branding \
snf-pithos-backend \
snf-cyclades-app
pithos.host$ apt-get install \
python-objpool \
snf-common \
python-astakosclient \
snf-django-lib \
snf-webproject \
snf-branding \
snf-pithos-backend \
snf-pithos-app \
snf-pithos-webclient
ganeti.node$ apt-get install \
python-objpool \
snf-common \
snf-cyclades-gtools \
snf-pithos-backend
.. note::
Make sure `snf-webproject' has the same version with snf-common
.. note::
Installing the packages will cause services to start. Make sure you bring
them down again (at least ``gunicorn``, ``snf-dispatcher``)
2.2 Sync and migrate the database
---------------------------------
.. note::
If you are asked about stale content types during the migration process,
answer 'no' and let the migration finish.
::
astakos-host$ snf-manage syncdb
astakos-host$ snf-manage migrate
cyclades-host$ snf-manage syncdb
cyclades-host$ snf-manage migrate
pithos-host$ pithos-migrate upgrade head
3. Re-register components and services in astakos
=================================================
Component registration has changed; you will thus need to repeat the
process. On the astakos node, run::
astakos-host$ snf-component-register
This will detect that the Synnefo components are already registered and ask
to re-register. Answer positively. You need to enter the base URL and the UI
URL for each component, just like during the initial registration.
4. Bring all services up
========================
After the upgrade is finished, we bring up all services:
.. code-block:: console
astakos.host # service gunicorn start
cyclades.host # service gunicorn start
pithos.host # service gunicorn start
cyclades.host # service snf-dispatcher start
......@@ -31,34 +31,44 @@
# interpreted as representing official policies, either expressed
# or implied, of GRNET S.A.
from optparse import make_option
from django.core.management.base import BaseCommand, CommandError
from astakos.im.models import Component
class Command(BaseCommand):
args = "<name> <component URL>"
args = "<name>"
help = "Register a component"
option_list = BaseCommand.option_list + (
make_option('--ui-url',
dest='ui_url',
default=None,
help="Set UI URL"),
make_option('--base-url',
dest='base_url',
default=None,
help="Set base URL"),
)
def handle(self, *args, **options):
if len(args) < 2:
if len(args) != 1:
raise CommandError("Invalid number of arguments")
name = args[0]
url = args[1]
base_url = options['base_url']
ui_url = options['ui_url']
try:
s = Component.objects.get(name=name)
Component.objects.get(name=name)
m = "There already exists a component named '%s'." % name
raise CommandError(m)
except Component.DoesNotExist:
pass
components = list(Component.objects.filter(url=url))
if components:
m = "Component URL '%s' is registered for another service." % url
raise CommandError(m)
try:
c = Component.objects.create(name=name, url=url)
c = Component.objects.create(
name=name, url=ui_url, base_url=base_url)
except BaseException:
raise CommandError("Failed to register component.")
else:
......
......@@ -42,9 +42,10 @@ class Command(ListCommand):
FIELDS = {
"id": ("id", "Component ID"),
"name": ("name", "Component Name"),
"url": ("url", "Component URL"),
"base url": ("base_url", "Component base URL"),
"ui url": ("url", "Component UI URL"),
"token": ("auth_token", "Authentication token"),
"token created": ("auth_token_created", "Token creation date"),
}
fields = ["id", "name", "url", "token"]
fields = ["id", "name", "base url"]
# Copyright 2012 GRNET S.A. All rights reserved.
# Copyright 2012, 2013 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
......@@ -43,10 +43,13 @@ class Command(BaseCommand):
help = "Modify component attributes"
option_list = BaseCommand.option_list + (
make_option('--url',
dest='url',
make_option('--ui-url',
dest='ui_url',
default=None,
help="Set component url"),
help="Set UI URL"),
make_option('--base-url',
dest='base_url',
help="Set base URL"),
make_option('--auth-token',
dest='auth_token',
default=None,
......@@ -79,16 +82,21 @@ class Command(BaseCommand):
"Component does not exist. You may run snf-manage "
"component-list for available component IDs.")
url = options.get('url')
ui_url = options.get('ui_url')
base_url = options.get('base_url')
auth_token = options.get('auth_token')
renew_token = options.get('renew_token')
purge_services = options.get('purge_services')
if not any([url, auth_token, renew_token, purge_services]):
if not any([ui_url, base_url, auth_token, renew_token,
purge_services]):
raise CommandError("No option specified.")
if url:
component.url = url
if ui_url:
component.url = ui_url
if base_url:
component.base_url = base_url
if auth_token:
component.auth_token = auth_token
......
# Copyright 2013 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
# conditions are met:
#
# 1. Redistributions of source code must retain the above
# copyright notice, this list of conditions and the following
# disclaimer.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials
# provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY GRNET S.A. ``AS IS'' AND ANY EXPRESS
# OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL GRNET S.A OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
# AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
# The views and conclusions contained in the software and
# documentation are those of the authors and should not be
# interpreted as representing official policies, either expressed
# or implied, of GRNET S.A.
from django.core.management.base import CommandError
from astakos.im.models import Component
from synnefo.lib.ordereddict import OrderedDict
from snf_django.management.commands import SynnefoCommand
from snf_django.management import utils
class Command(SynnefoCommand):
args = "<component name or ID>"
help = "Show component details"
def handle(self, *args, **options):
if len(args) != 1:
raise CommandError("Please provide a component name or ID.")
identifier = args[0]
if identifier.isdigit():
try:
component = Component.objects.get(id=int(identifier))
except Component.DoesNotExist:
raise CommandError('No component found with ID %s.' %
identifier)
else:
try:
component = Component.objects.get(name=identifier)
except Component.DoesNotExist:
raise CommandError('No component found named %s.' % identifier)
kv = OrderedDict(
[
('id', component.id),
('name', component.name),
('base url', component.base_url),
('ui url', component.url),
('token', component.auth_token),
('token created', component.auth_token_created),
('token expires', component.auth_token_expires),
])
utils.pprint_table(self.stdout, [kv.values()], kv.keys(),
options["output_format"], vertical=True)
services = component.service_set.all()
service_data = []
for service in services:
service_data.append((service.id, service.name, service.type))
if service_data:
self.stdout.write('\n')
labels = ('id', 'name', 'type')
utils.pprint_table(self.stdout, service_data, labels,
options["output_format"],
title='Registered services')
......@@ -90,7 +90,8 @@ class Command(BaseCommand):
raise CommandError(m)
try:
existed = add_service(component, name, service_type, endpoints)
existed = add_service(component, name, service_type, endpoints,
out=self.stdout)
except RegisterException as e:
raise CommandError(e.message)
......
......@@ -124,6 +124,7 @@ class Component(models.Model):
db_index=True)
url = models.CharField(_('Component url'), max_length=1024, null=True,
help_text=_("URL the component is accessible from"))
base_url = models.CharField(max_length=1024, null=True)
auth_token = models.CharField(_('Authentication Token'), max_length=64,
null=True, blank=True, unique=True)
auth_token_created = models.DateTimeField(_('Token creation date'),
......
......@@ -121,14 +121,19 @@ def get_resources(resources=None, services=None):
return resource_dict
def add_endpoint(service, endpoint_dict):
def add_endpoint(component, service, endpoint_dict, out=None):
endpoint = Endpoint.objects.create(service=service)
for key, value in endpoint_dict.iteritems():
base_url = component.base_url
if key == "publicURL" and not value.startswith(base_url):
warn = out.write if out is not None else logger.warning
warn("Warning: Endpoint URL '%s' does not start with "
"assumed component base URL '%s'.\n" % (value, base_url))
EndpointData.objects.create(
endpoint=endpoint, key=key, value=value)
def add_service(component, name, service_type, endpoints):
def add_service(component, name, service_type, endpoints, out=None):
defaults = {'component': component,
'type': service_type,
}
......@@ -141,12 +146,11 @@ def add_service(component, name, service_type, endpoints):
(name, service.component.name))
raise RegisterException(m)
service.endpoints.all().delete()
else:
service.component = component
service.type = service_type
for key, value in defaults.iteritems():
setattr(service, key, value)
service.save()
for endpoint in endpoints:
add_endpoint(service, endpoint)
add_endpoint(component, service, endpoint, out=out)
return not created
......@@ -11,9 +11,9 @@ desc[cyclades]='compute component'
desc[pithos]='file storage component'
declare -A ex_url
ex_url[astakos]='https://accounts.example.synnefo.org/astakos/'
ex_url[cyclades]='https://compute.example.synnefo.org/cyclades/'
ex_url[pithos]='https://storage.example.synnefo.org/pithos/'
ex_url[astakos]='https://accounts.example.synnefo.org/astakos'
ex_url[cyclades]='https://compute.example.synnefo.org/cyclades'
ex_url[pithos]='https://storage.example.synnefo.org/pithos'
register_services () {
......@@ -22,7 +22,7 @@ register_services () {
}
ex_ui_url () {
echo "$(echo $1 | sed -e 's/\/*$//g')/ui/"
echo "$(echo $1 | sed -e 's/\/*$//g')/ui"
}
changed=0
......@@ -42,6 +42,7 @@ decide () {
register_component () {
component=$1
exists=$2
component_desc=${desc[$component]}
component_ex_url=${ex_url[$component]}
echo "Registering the $component_desc ($component):"
......@@ -55,29 +56,18 @@ register_component () {
read ui_url
decide "Register $component with the given URLs (y/n)? "
if [ $? -eq 0 ]; then
snf-manage component-add $component $ui_url
if [ $? -eq 0 ]; then
read -p "Please write down the token and press Enter to continue. "
register_services $component $base_url
changed=1
echo
if [ $exists -eq 0 ]; then
snf-manage component-add $component --base-url $base_url \
--ui-url $ui_url
if [ $? -eq 0 ]; then
read -p "Please write down the token and press Enter to continue. "
changed=1
fi
else
snf-manage component-modify $component --base-url $base_url \
--ui-url $ui_url --purge-services
fi
fi
}
register_comp_serv () {
component=$1
component_desc=${desc[$component]}
component_ex_url=${ex_url[$component]}
echo "Registering services for $component:"
echo "Give the URL of $component base installation" \
"(e.g. $component_ex_url)"
echo -n 'Base URL: '
read base_url
decide "Register ${component}'s services with the given URL (y/n)? "
if [ $? -eq 0 ]; then
register_services $component $base_url
echo
fi
}
......@@ -91,13 +81,13 @@ register_one () {
if [ $? -ne 0 ]; then
decide "Register the ${desc[$component]} ($component) (y/n)? "
if [ $? -eq 0 ]; then
register_component $component
register_component $component 0
fi
else
echo "The ${desc[$component]} ($component) is registered."
decide "Update its registered services (y/n)? "
decide "Re-register (y/n)? "
if [ $? -eq 0 ]; then
register_comp_serv $component
register_component $component 1
fi
fi
}
......
......@@ -600,15 +600,15 @@ def astakos_loaddata():
def astakos_register_components():
debug(env.host, " * Register services in astakos...")
cyclades_base_url = "https://%s/cyclades/" % env.env.cyclades.fqdn
pithos_base_url = "https://%s/pithos/" % env.env.pithos.fqdn
astakos_base_url = "https://%s/astakos/" % env.env.accounts.fqdn
cyclades_base_url = "https://%s/cyclades" % env.env.cyclades.fqdn
pithos_base_url = "https://%s/pithos" % env.env.pithos.fqdn
astakos_base_url = "https://%s/astakos" % env.env.accounts.fqdn
cmd = """
snf-manage component-add "home" https://{0} home-icon.png
snf-manage component-add "cyclades" {1}ui/
snf-manage component-add "pithos" {2}ui/
snf-manage component-add "astakos" {3}ui/
snf-manage component-add "home" --ui-url https://{0}
snf-manage component-add "cyclades" --base-url {1} --ui-url {1}/ui
snf-manage component-add "pithos" --base-url {2} --ui-url {2}/ui
snf-manage component-add "astakos" --base-url {3} --ui-url {3}/ui
""".format(env.env.cms.fqdn, cyclades_base_url,
pithos_base_url, astakos_base_url)
try_run(cmd)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment