Commit 598f4e2f authored by Christos Stavrakakis's avatar Christos Stavrakakis

Merge branch 'hotfix-0.14.2' into develop

Conflicts:
	Changelog
	snf-astakos-app/astakos/im/messages.py
	snf-astakos-app/astakos/im/views/target/__init__.py
	snf-cyclades-app/synnefo/logic/backend.py
	snf-cyclades-app/synnefo/logic/management/commands/reconcile-servers.py
	snf-pithos-backend/pithos/backends/lib/sqlalchemy/node.py
	snf-pithos-backend/pithos/backends/lib/sqlite/node.py
parents d40f74ea 9ececcf9
......@@ -13,6 +13,12 @@ v0.14next
Released: UNRELEASED
Synnefo-wide
------------
* Integrate Pithos tests in continuous integration.
Cyclades
--------
......@@ -27,10 +33,19 @@ objects by domain attribute. This is used by Plankton for listing VM images.
* Enforce container-level atomicity in (most) Pithos API calls.
Synnefo-wide
------------
.. _Changelog-0.14.2:
Released: Fri Jul 12 13:13:32 EEST 2013
v0.14.2
=======
Cyclades
--------
* Add new setting PITHOS_BACKEND_POOL_SIZE, which configures the size
of the pool of Pithos backends that are used by plankton.
* Integrate Pithos tests in continuous integration.
.. _Changelog-0.14:
......
......@@ -6,6 +6,13 @@ Unified NEWS file for Synnefo versions >= 0.13
Since v0.13 all Synnefo components have been merged into a single repository.
.. _NEWS-0.14.2:
v0.14.2
=======
Released: Fri Jul 12 13:13:32 EEST 2013
.. _NEWS-0.14:
v0.14
......
......@@ -1802,9 +1802,13 @@ Upgrade Notes
v0.12 -> v0.13 <upgrade/upgrade-0.13>
v0.13 -> v0.14 <upgrade/upgrade-0.14>
v0.14 -> v0.14.2 <upgrade/upgrade-0.14.2>
Changelog, NEWS
===============
* v0.14.2 :ref:`Changelog <Changelog-0.14.2>`, :ref:`NEWS <NEWS-0.14.2>`
* v0.14 :ref:`Changelog <Changelog-0.14>`, :ref:`NEWS <NEWS-0.14>`
* v0.13 :ref:`Changelog <Changelog-0.13>`, :ref:`NEWS <NEWS-0.13>`
......@@ -1760,8 +1760,6 @@ It can be retrieved by running on the Astakos node (node1 in our case):
The token has been generated automatically during the :ref:`Cyclades service
registration <services-reg>`.
TODO: Document the Network Options here
Edit ``/etc/synnefo/20-snf-cyclades-app-cloudbar.conf``:
.. code-block:: console
......@@ -1800,19 +1798,6 @@ The above settings denote the Message Queue. Those settings should have the same
values as in ``/etc/synnefo/10-snf-cyclades-gtools-backend.conf`` file, and
reflect our :ref:`Message Queue setup <rabbitmq-setup>`.
Edit ``/etc/synnefo/20-snf-cyclades-app-ui.conf``:
.. code-block:: console
UI_LOGIN_URL = "https://node1.example.com/ui/login"
UI_LOGOUT_URL = "https://node1.example.com/ui/logout"
The ``UI_LOGIN_URL`` option tells the Cyclades Web UI where to redirect users,
if they are not logged in. We point that to Astakos.
The ``UI_LOGOUT_URL`` option tells the Cyclades Web UI where to redirect the
user when he/she logs out. We point that to Astakos, too.
Edit ``/etc/synnefo/20-snf-cyclades-app-vmapi.conf``:
.. code-block:: console
......@@ -2107,13 +2092,12 @@ installation. We do this by running:
.. code-block:: console
$ kamaki config set user.url "https://node1.example.com"
$ kamaki config set compute.url "https://node1.example.com/api/v1.1"
$ kamaki config set image.url "https://node1.example.com/image"
$ kamaki config set file.url "https://node2.example.com/v1"
$ kamaki config set token USER_TOKEN
$ kamaki config set cloud.default.url \
"https://node1.example.com/astakos/identity/v2.0"
$ kamaki config set cloud.default.token USER_TOKEN
The USER_TOKEN appears on the user's `Profile` web page on the Astakos Web UI.
Both the Authentication URL and the USER_TOKEN appear on the user's
`API access` web page on the Astakos Web UI.
You can see that the new configuration options have been applied correctly,
either by checking the editable file ``~/.kamakirc`` or by running:
......
......@@ -33,12 +33,13 @@ following line in your ``/etc/apt/sources.list`` file:
.. code-block:: console
deb http://apt.dev.grnet.gr unstable/
deb http://apt.dev.grnet.gr stable/
Then run:
.. code-block:: console
# curl https://dev.grnet.gr/files/apt-grnetdev.pub | apt-key add -
# apt-get update
# apt-get install snf-deploy
......@@ -105,6 +106,8 @@ installed Synnefo on a single node.
Caveats
=======
Certificates
------------
To be able to view all web pages make sure you have accepted all certificates
for domains:
......@@ -115,6 +118,38 @@ for domains:
* cms.synnefo.live
Spawning VMs
------------
By default, snf-deploy can't spawn VMs. To be able to do so, edit
``/etc/synnefo/cyclades.conf`` and change line 29 from:
.. code-block:: console
'no_install': True,
to:
.. code-block:: console
'no_install': False,
Networks
--------
In order to create private networks, you have to edit
``/etc/synnefo/cyclades.conf`` and change line 3 from:
.. code-block:: console
PRIVATE_MAC_FILTERED_BRIDGE = 'br0'
to:
.. code-block:: console
DEFAULT_MAC_FILTERED_BRIDGE = 'br0'
Using the installation
======================
......
......@@ -71,8 +71,6 @@ In `/etc/synnefo/cyclades.conf` add:
FEEDBACK_CONTACTS = (
('feedback@example.com', 'feedback@example.com'),
)
UI_LOGIN_URL = "https://accounts.example.com/im/login"
UI_LOGOUT_URL = "https://accounts.example.com/im/logout"
UI_FLAVORS_DISK_TEMPLATES_INFO = {
'rbd': {'name': 'Rbd',
'description': 'Volumes residing inside a RADOS cluster'},
......
Upgrade to Synnefo v0.14.2
^^^^^^^^^^^^^^^^^^^^^^^^^^
The upgrade from v0.14 to v0.14.2 consists in three steps:
1. Bring down services and backup databases.
2. Upgrade packages and migrate Pithos database.
3. Bring up all services.
1. Bring web services down, backup databases
============================================
1. All web services must be brought down so that the database maintains a
predictable and consistent state during the migration process::
$ service gunicorn stop
$ service snf-dispatcher stop
$ service snf-ganeti-eventd stop
2. Backup databases for recovery to a pre-migration state.
3. Keep the database servers running during the migration process
2. Upgrade Synnefo and configure settings
=========================================
2.1 Install the new versions of packages
----------------------------------------
::
astakos.host$ apt-get install \
python-objpool \
snf-common \
python-astakosclient \
snf-django-lib \
snf-webproject \
snf-branding \
snf-astakos-app
cyclades.host$ apt-get install \
python-objpool \
snf-common \
python-astakosclient \
snf-django-lib \
snf-webproject \
snf-branding \
snf-pithos-backend \
snf-cyclades-app
pithos.host$ apt-get install \
python-objpool \
snf-common \
python-astakosclient \
snf-django-lib \
snf-webproject \
snf-branding \
snf-pithos-backend \
snf-pithos-app \
snf-pithos-webclient
ganeti.node$ apt-get install \
python-objpool \
snf-common \
snf-cyclades-gtools \
snf-pithos-backend
.. note::
Make sure `snf-webproject' has the same version with snf-common
2.2 Sync and migrate the database
---------------------------------
::
pithos-host$ pithos-migrate upgrade head
5. Bring all services up
========================
After the upgrade is finished, we bring up all services:
.. code-block:: console
astakos.host # service gunicorn start
cyclades.host # service gunicorn start
pithos.host # service gunicorn start
cyclades.host # service snf-dispatcher start
......@@ -46,6 +46,8 @@ from django.conf import settings
from astakos.im import settings as astakos_settings
from astakos.im import messages as astakos_messages
from synnefo_branding import utils as branding_utils
import logging
logger = logging.getLogger(__name__)
......@@ -99,7 +101,10 @@ class AuthProvider(object):
('add_prompt', 'Allows you to login using {title}'),
('login_extra', ''),
('username', '{username}'),
('disabled_for_create', '{title} is not available for signup.'),
('disabled_for_create', 'It seems this is the first time you\'re '
'trying to access {service_name}. '
'Unfortunately, we are not accepting new '
'users at this point.'),
('switch_success', 'Account changed successfully.'),
('cannot_login', '{title} is not available for login. '
'Please use one of your other available methods '
......@@ -294,6 +299,10 @@ class AuthProvider(object):
else:
params['username'] = self.identifier
branding_params = dict(map(lambda k: (k[0].lower(), k[1]),
branding_utils.get_branding_dict().iteritems()))
params.update(branding_params)
if not self.message_tpls_compiled:
for key, message_tpl in self.message_tpls.iteritems():
msg = self.messages.get(key, self.message_tpls.get(key))
......@@ -587,7 +596,7 @@ class LocalAuthProvider(AuthProvider):
class ShibbolethAuthProvider(AuthProvider):
module = 'shibboleth'
login_view = 'astakos.im.views.target.shibboleth.login'
username_key = 'identifier'
username_key = 'provider_info_eppn'
policies = {
'switch': False
......@@ -597,9 +606,13 @@ class ShibbolethAuthProvider(AuthProvider):
'title': _('Academic'),
'login_description': _('If you are a student, professor or researcher'
' you can login using your academic account.'),
'add_prompt': _('Allows you to login using your Academic '
'account'),
'method_details': 'Account: {username}',
'logout_extra': _('Please close all browser windows to complete '
'logout from your Academic account, too.')
'logout_success_extra': _('You may still be logged in at your Academic'
' account though. Consider logging out '
'from there too by closing all browser '
'windows')
}
......
......@@ -1097,7 +1097,8 @@ class ExtendedProfileForm(ProfileForm):
password, email = True, True
profile = super(ExtendedProfileForm, self).is_valid()
if profile and self.cleaned_data.get('change_password', None):
self.password_change_form.fields['new_password1'].required = True
self.password_change_form.fields['new_password2'].required = True
password = self.password_change_form.is_valid()
self.save_extra_forms.append('password')
if profile and self.cleaned_data.get('change_email'):
......
......@@ -768,7 +768,7 @@ def check_expiration(execute=False):
expired = objects.expired_projects()
if execute:
for project in expired:
terminate(project.id)
terminate(project.pk)
return [project.expiration_info() for project in expired]
......
......@@ -287,7 +287,7 @@ PENDING_APPLICATION_LIMIT_MODIFY = \
# Auth providers messages
AUTH_PROVIDER_LOGIN_SUCCESS = "Logged in successfully, using {method_prompt}."
AUTH_PROVIDER_LOGOUT_SUCCESS = "Logged out successfully."
AUTH_PROVIDER_LOGOUT_SUCCESS = "Logged out successfully from {service_name}."
AUTH_PROVIDER_LOGOUT_SUCCESS_EXTRA = (
"You may still be logged in at {title} though. "
"Consider logging out from there too.")
......
......@@ -131,13 +131,14 @@ class ShibbolethTests(TestCase):
# provider info stored
provider = AstakosUserAuthProvider.objects.get(module="shibboleth")
self.assertEqual(provider.affiliation, 'Test Affiliation')
self.assertEqual(provider.info, {u'email': u'kpap@synnefo.org',
u'eppn': u'kpapeppn',
u'name': u'Kostas Papadimitriou'})
self.assertEqual(provider.info['email'], u'kpap@synnefo.org')
self.assertEqual(provider.info['eppn'], u'kpapeppn')
self.assertEqual(provider.info['name'], u'Kostas Papadimitriou')
self.assertTrue('headers' in provider.info)
# login (not activated yet)
client.set_tokens(mail="kpap@synnefo.org", eppn="kpapeppn",
cn="Kostas Papadimitriou", )
cn="Kostas Papadimitriou")
r = client.get(ui_url("login/shibboleth?"), follow=True)
self.assertContains(r, 'is pending moderation')
......@@ -320,6 +321,17 @@ class ShibbolethTests(TestCase):
self.assertTrue(user.has_auth_provider('shibboleth'))
self.assertTrue(user.check_password('111'))
self.assertTrue(user.has_usable_password())
# change password via profile form
r = client.post(ui_url("profile"), {
'old_password': '111',
'new_password': '',
'new_password2': '',
'change_password': 'on',
}, follow=False)
self.assertEqual(r.status_code, 200)
self.assertFalse(r.context['profile_form'].is_valid())
self.client.logout()
# now we can login
......
......@@ -160,6 +160,7 @@ def handle_third_party_login(request, provider_module, identifier,
next_redirect = request.GET.get(
'next', request.session.get('next_url', None))
if 'next_url' in request.session:
del request.session['next_url']
......@@ -170,6 +171,7 @@ def handle_third_party_login(request, provider_module, identifier,
'affiliation': affiliation,
'info': provider_info
}
provider = auth.get_provider(provider_module, request.user, identifier,
**provider_data)
......
......@@ -61,6 +61,7 @@ class Tokens:
SHIB_EP_AFFILIATION = "HTTP_SHIB_EP_AFFILIATION"
SHIB_SESSION_ID = "HTTP_SHIB_SESSION_ID"
SHIB_MAIL = "HTTP_SHIB_MAIL"
SHIB_REMOTE_USER = "HTTP_REMOTE_USER"
@requires_auth_provider('shibboleth')
......@@ -79,7 +80,9 @@ def login(request,
shibboleth_headers = {}
for token in dir(Tokens):
if token == token.upper():
shibboleth_headers[token] = request.META.get(token, 'NOT_SET')
shibboleth_headers[token] = request.META.get(getattr(Tokens,
token),
'NOT_SET')
# log shibboleth headers
# TODO: info -> debug
......@@ -87,6 +90,7 @@ def login(request,
try:
eppn = tokens.get(Tokens.SHIB_EPPN)
if global_settings.DEBUG and not eppn:
eppn = getattr(global_settings, 'SHIBBOLETH_TEST_EPPN', None)
realname = getattr(global_settings, 'SHIBBOLETH_TEST_REALNAME',
......@@ -117,7 +121,9 @@ def login(request,
affiliation = tokens.get(Tokens.SHIB_EP_AFFILIATION, 'Shibboleth')
email = tokens.get(Tokens.SHIB_MAIL, '')
provider_info = {'eppn': eppn, 'email': email, 'name': realname}
eppn_info = tokens.get(Tokens.SHIB_EPPN)
provider_info = {'eppn': eppn_info, 'email': email, 'name': realname,
'headers': shibboleth_headers}
userid = eppn
try:
......
......@@ -6,6 +6,7 @@
## Backend settings
#BACKEND_DB_CONNECTION = 'sqlite:////usr/share/synnefo/pithos/backend.db'
#BACKEND_BLOCK_PATH = '/usr/share/synnefo/pithos/data/'
#PITHOS_BACKEND_POOL_SIZE = 8
#
## The Pithos container where images will be stored by default
#DEFAULT_PLANKTON_CONTAINER = 'images'
......
......@@ -14,9 +14,9 @@
#DEFAULT_KEYWORDS = ["OS", "Role", "Location", "Owner"]
#
## A list of allowed icons for OS Images
#IMAGE_ICONS = ["redhat", "ubuntu", "debian", "windows", "gentoo", "archlinux",
#IMAGE_ICONS = ["rhel", "ubuntu", "debian", "windows", "gentoo", "archlinux",
# "centos", "fedora", "freebsd", "netbsd", "openbsd", "slackware",
# "suse", "kubuntu"]
# "sles", "opensuse", "kubuntu"]
#
## How often should the UI request changes from the API
#UI_UPDATE_INTERVAL = 5000
......
......@@ -65,4 +65,5 @@ class Command(BaseCommand):
for bnet in network.backend_networks.exclude(operstate="DELETED"):
delete_network(network, bnet.backend)
self.stdout.write('Successfully removed network.\n')
self.stdout.write("Successfully submitted Ganeti jobs to"
" remove network %s" % network.backend_id)
......@@ -6,6 +6,7 @@
# Backend settings
BACKEND_DB_CONNECTION = 'sqlite:////usr/share/synnefo/pithos/backend.db'
BACKEND_BLOCK_PATH = '/usr/share/synnefo/pithos/data/'
PITHOS_BACKEND_POOL_SIZE = 8
# The Pithos container where images will be stored by default
DEFAULT_PLANKTON_CONTAINER = 'images'
......
......@@ -17,9 +17,9 @@ TIMEOUT = 10 * 1000
DEFAULT_KEYWORDS = ["OS", "Role", "Location", "Owner"]
# A list of allowed icons for OS Images
IMAGE_ICONS = ["redhat", "ubuntu", "debian", "windows", "gentoo", "archlinux",
IMAGE_ICONS = ["rhel", "ubuntu", "debian", "windows", "gentoo", "archlinux",
"centos", "fedora", "freebsd", "netbsd", "openbsd", "slackware",
"suse", "kubuntu"]
"sles", "opensuse", "kubuntu"]
# How often should the UI request changes from the API
UI_UPDATE_INTERVAL = 5000
......
......@@ -5,7 +5,8 @@
"fields": {
"cpu": 1,
"ram": 1024,
"disk": 20
"disk": 20,
"disk_template": "drbd"
}
},
......@@ -15,7 +16,8 @@
"fields": {
"cpu": 1,
"ram": 1024,
"disk": 30
"disk": 30,
"disk_template": "drbd"
}
},
......@@ -25,7 +27,8 @@
"fields": {
"cpu": 1,
"ram": 1024,
"disk": 40
"disk": 40,
"disk_template": "drbd"
}
},
......@@ -35,7 +38,8 @@
"fields": {
"cpu": 1,
"ram": 2048,
"disk": 20
"disk": 20,
"disk_template": "drbd"
}
},
......@@ -45,7 +49,8 @@
"fields": {
"cpu": 1,
"ram": 2048,
"disk": 30
"disk": 30,
"disk_template": "drbd"
}
},
......@@ -55,7 +60,8 @@
"fields": {
"cpu": 1,
"ram": 2048,
"disk": 40
"disk": 40,
"disk_template": "drbd"
}
},
......@@ -65,7 +71,8 @@
"fields": {
"cpu": 1,
"ram": 4096,
"disk": 20
"disk": 20,
"disk_template": "drbd"
}
},
......@@ -75,7 +82,8 @@
"fields": {
"cpu": 1,
"ram": 4096,
"disk": 30
"disk": 30,
"disk_template": "drbd"
}
},
......@@ -85,7 +93,8 @@
"fields": {
"cpu": 1,
"ram": 4096,
"disk": 40
"disk": 40,
"disk_template": "drbd"
}
},
......@@ -95,7 +104,8 @@
"fields": {
"cpu": 2,
"ram": 1024,
"disk": 20
"disk": 20,
"disk_template": "drbd"
}