Skip to content

GitLab

S
synnefo
Commit 6a71c1a2 authored by Giorgos Korfiatis's avatar Giorgos Korfiatis
Browse files
Options

Merge branch 'hotfix-0.18.1'

parents 28b8a1d5 f38c1729
Show whitespace changes
Inline Side-by-side
Showing with 297 additions and 257 deletions
Changelog
View file @ 6a71c1a2
......@@ -6,12 +6,10 @@ Unified Changelog file for Synnefo versions >= 0.13
Since v0.13 most of the Synnefo components have been merged into a single
repository and have aligned versions.
.. _Changelog-0.18:
.. _Changelog-0.18.1:
v0.18
========
Released: Wed 7 Sep 16:50:30 EEST 2016
v0.18.1
=======
Astakos
-------
......@@ -23,7 +21,7 @@ Astakos
* Support suspending and unsuspending project memberships.
* User deactivation now automatically suspends user's system project, owned
projects, and project memberships. Reactivation unsuspends them.
* Add command `user-check`. It supports suspending projects for previously
* Add script `fix_deactivated_users` to suspend projects for previously
deactivated users.
* Send an informative email to the user's current email address when they
request to change their email.
......
NEWS
View file @ 6a71c1a2
......@@ -5,14 +5,14 @@ Unified NEWS file for Synnefo versions >= 0.13
Since v0.13 all Synnefo components have been merged into a single repository.
.. _NEWS-0.18:
.. _NEWS-0.18.1:
v0.18
=====
v0.18.1
=======
Released: Wed 7 Sep 16:50:30 EEST 2016
Released: UNRELEASED
The Synnefo 0.18 release brings significant bug fixes across Synnefo.
The Synnefo 0.18.1 release brings significant improvements across Synnefo.
The most notable changes are:
......
docs/admin-guide.rst
View file @ 6a71c1a2
......@@ -3077,7 +3077,7 @@ Upgrade Notes
v0.15 -> v0.16 <upgrade/upgrade-0.16>
v0.16.1 -> v0.16.2 <upgrade/upgrade-0.16.2>
v0.16.2 -> v0.17 <upgrade/upgrade-0.17>
v0.17 -> v0.18 <upgrade/upgrade-0.18>
v0.17 -> v0.18.1 <upgrade/upgrade-0.18.1>
.. _changelog-news:
......@@ -3086,7 +3086,7 @@ Changelog, NEWS
===============
* v0.18 :ref:`Changelog <Changelog-0.18>`, :ref:`NEWS <NEWS-0.18>`
* v0.18.1 :ref:`Changelog <Changelog-0.18.1>`, :ref:`NEWS <NEWS-0.18.1>`
* v0.17 :ref:`Changelog <Changelog-0.17>`, :ref:`NEWS <NEWS-0.17>`
* v0.16.2 :ref:`Changelog <Changelog-0.16.2>`, :ref:`NEWS <NEWS-0.16.2>`
* v0.16.1 :ref:`Changelog <Changelog-0.16.1>`, :ref:`NEWS <NEWS-0.16.1>`
......
docs/upgrade/upgrade-0.18.1.rst 0 → 100644
View file @ 6a71c1a2
Upgrade to Synnefo v0.18.1
^^^^^^^^^^^^^^^^^^^^^^^^^^
Upgrade Steps
=============
The upgrade from v0.17 to v0.18.1 consists of the following steps:
#. Bring down services::
$ service gunicorn stop
$ service snf-dispatcher stop
$ service snf-ganeti-eventd stop
#. Upgrade Synnefo on all nodes to the latest version::
astakos.host$ apt-get install \
snf-common \
python-astakosclient \
snf-django-lib \
snf-webproject \
snf-branding \
snf-astakos-app
cyclades.host$ apt-get install \
snf-common \
python-astakosclient \
snf-django-lib \
snf-webproject \
snf-branding \
snf-pithos-backend \
snf-cyclades-app
pithos.host$ apt-get install \
snf-common \
python-astakosclient \
snf-django-lib \
snf-webproject \
snf-branding \
snf-pithos-backend \
snf-pithos-app \
snf-ui-app
ganeti.node$ apt-get install \
snf-common \
snf-cyclades-gtools \
snf-pithos-backend
#. Run migrations on Astakos.
.. code-block:: console
astakos.host$ snf-manage migrate
From this version on, user deactivation triggers suspension of all projects
and project memberships related to the user. To apply this new policy to
users that have already been deactivated, run:
.. code-block:: console
astakos.host$ /usr/lib/astakos/tools/fix_deactivated_users --all-users --noemail --fix
#. Restart services
.. code-block:: console
$ service gunicorn start
$ service snf-dispatcher start
$ service snf-ganeti-eventd start
New configuration options
=========================
On the admin app, there is a new access control option regarding the new modify
email action. The action setting is named 'modify_email'. The list of user
groups defined in this have access on the modify email action.
The following line (modified accordingly) should be added on 'ADMIN_RBAC'
setting under the 'user' dictionary:
.. code-block:: console
'modify_email': [ADMIN_HELPDESK_GROUP, ADMIN_GROUP],
docs/upgrade/upgrade-0.18.rst
View file @ 6a71c1a2
Upgrade to Synnefo v0.18
^^^^^^^^^^^^^^^^^^^^^^^^
Upgrade Steps
=============
The upgrade to v0.18 consists of the following steps:
#. Stop gunicorn in all nodes
.. code-block:: console
# service gunicorn stop
#. Upgrade Synnefo on all nodes to the latest version (0.18)
.. code-block:: console
# apt-get update
# apt-get upgrade
#. Run migrations on Astakos.
.. code-block:: console
astakos.host$ snf-manage migrate
From this version on, user deactivation triggers suspension of all projects
and project memberships related to the user. To apply this new policy to
users that have already been deactivated, run:
.. code-block:: console
astakos.host$ snf-manage user-check --all-users --suspend-deactivated --noemail --fix
#. Start gunicorn
.. code-block:: console
# service gunicorn start
New configuration options
=========================
On the admin app, there is a new access control option regarding the new modify
email action. The action setting is named 'modify_email'. The list of user
groups defined in this have access on the modify email action.
The following line (modified accordingly) should be added on 'ADMIN_RBAC'
setting under the 'user' dictionary:
.. code-block:: console
'modify_email': [ADMIN_HELPDESK_GROUP, ADMIN_GROUP],
Version 0.18 is not recommended; upgrade directly to 0.18.1.
snf-admin-app/synnefo_admin/admin/static/sass/_details.scss
View file @ 6a71c1a2
......@@ -245,6 +245,7 @@
&.info-data {
padding: 0 20px;
white-space: pre-wrap;
}
}
......
snf-admin-app/synnefo_admin/admin/templatetags/admin_tags.py
View file @ 6a71c1a2
......@@ -500,7 +500,7 @@ def flatten_dict_to_dl(d, default_if_empty='-'):
if isinstance(v, dict):
stack.extend(v.iteritems())
else:
a = '<dt>{0}</dt><dd>{1}</dd>'.format(k, v or default_if_empty)
a = u'<dt>{0}</dt><dd>{1}</dd>'.format(k, v or default_if_empty)
l.append(a)
return mark_safe(''.join(reversed(l)))
......
snf-astakos-app/astakos/im/activation_backends.py
View file @ 6a71c1a2
......@@ -357,7 +357,7 @@ class ActivationBackend(object):
if not ok:
return ActivationResult(self.Result.ERROR, msg)
was_deactivated = bool(user.deactivated_at)
was_deactivated = not user.is_active
user.is_active = True
user.deactivated_reason = None
user.deactivated_at = None
......
snf-astakos-app/astakos/im/forms.py
View file @ 6a71c1a2
# Copyright (C) 2010-2014 GRNET S.A.
# Copyright (C) 2010-2016 GRNET S.A.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
......@@ -1192,7 +1192,10 @@ class AddProjectMembersForm(forms.Form):
q = self.cleaned_data.get('q') or ''
users = re.split("\r\n|\n|,", q)
users = list(u.strip() for u in users if u)
db_entries = AstakosUser.objects.accepted().filter(email__in=users)
# Notice that deactivated users are reported as 'unknown' here, too.
db_entries = AstakosUser.objects.accepted().filter(
email__in=users, is_active=True)
unknown = list(set(users) - set(u.email for u in db_entries))
if unknown:
raise forms.ValidationError(
......
snf-astakos-app/astakos/im/functions.py
View file @ 6a71c1a2
......@@ -218,13 +218,17 @@ def app_check_allowed(application, request_user,
return _failure(silent)
def checkAlive(project, silent=False):
def checkAlive(project, silent=False, check_initialized=False):
def fail(msg):
if silent:
return False, msg
else:
raise ProjectConflict(msg)
if check_initialized:
if not project.is_initialized():
m = _(astakos_messages.NOT_INITIALIZED_PROJECT) % project.uuid
return fail(m)
else:
if not project.is_alive:
m = _(astakos_messages.NOT_ALIVE_PROJECT) % project.uuid
return fail(m)
......@@ -249,7 +253,9 @@ def accept_membership_checks(membership, request_user):
if not membership.check_action("accept"):
m = _(astakos_messages.NOT_MEMBERSHIP_REQUEST)
raise ProjectConflict(m)
if not membership.person.is_active:
m = _(astakos_messages.ACCOUNT_NOT_ACTIVE)
raise ProjectConflict(m)
project = membership.project
accept_membership_project_checks(project, request_user)
......@@ -350,7 +356,7 @@ def suspend_membership_checks(membership, request_user=None):
project = membership.project
project_check_allowed(project, request_user, level=ADMIN_LEVEL)
checkAlive(project)
checkAlive(project, check_initialized=True)
def suspend_membership(memb_id, request_user=None, reason=None):
......@@ -376,7 +382,7 @@ def unsuspend_membership_checks(membership, request_user=None):
project = membership.project
project_check_allowed(project, request_user, level=ADMIN_LEVEL)
checkAlive(project)
checkAlive(project, check_initialized=True)
def unsuspend_membership(memb_id, request_user=None, reason=None):
......@@ -403,12 +409,19 @@ def enroll_member_by_email(project_id, email, request_user=None, reason=None):
raise ProjectConflict(astakos_messages.UNKNOWN_USERS % email)
def enroll_member_checks(project, user, request_user):
if not user.is_active:
m = _(astakos_messages.ACCOUNT_NOT_ACTIVE)
raise ProjectConflict(m)
accept_membership_project_checks(project, request_user)
def enroll_member(project_id, user, request_user=None, reason=None):
try:
project = get_project_for_update(project_id)
except ProjectNotFound as e:
raise ProjectConflict(e.message)
accept_membership_project_checks(project, request_user)
enroll_member_checks(project, user, request_user)
try:
membership = get_membership(project.id, user.id)
......@@ -1248,19 +1261,21 @@ def _get_owned_projects(user_ids):
return _partition_by(lambda p: p.owner_id, ps)
def suspend_users_projects(users, request_user=None, reason=None, fix=True):
affected_users = []
def get_projects_and_memberships_of_users(users):
user_ids = [user.id for user in users]
if not user_ids:
return affected_users
return {}, {}, {}
base_projects_d = _get_base_projects(user_ids)
memberships_d = _get_memberships(user_ids)
owned_projects_d = _get_owned_projects(user_ids)
return base_projects_d, memberships_d, owned_projects_d
for user in users:
def suspend_user_projects_and_memberships(
user, base_project, memberships, owned_projects,
request_user=None, reason=None, fix=True):
is_affected = False
base_project = base_projects_d[user.base_project_id]
if base_project.state == Project.NORMAL:
try:
if fix:
......@@ -1269,7 +1284,6 @@ def suspend_users_projects(users, request_user=None, reason=None, fix=True):
is_affected = True
except ProjectError:
pass
memberships = memberships_d.get(user.id, [])
for membership in memberships:
try:
if fix:
......@@ -1280,7 +1294,6 @@ def suspend_users_projects(users, request_user=None, reason=None, fix=True):
except ProjectError:
pass
owned_projects = owned_projects_d.get(user.id, [])
for project in owned_projects:
try:
if fix:
......@@ -1289,14 +1302,18 @@ def suspend_users_projects(users, request_user=None, reason=None, fix=True):
is_affected = True
except ProjectError:
pass
if is_affected:
affected_users.append(user)
return affected_users
return is_affected
def suspend_user_projects(user, request_user=None, reason=None):
return suspend_users_projects(
[user], request_user=request_user, reason=reason)
bpd, md, opd = get_projects_and_memberships_of_users([user])
base_project = bpd[user.base_project_id]
memberships = md.get(user.id, [])
owned_projects = opd.get(user.id, [])
return suspend_user_projects_and_memberships(
user, base_project, memberships, owned_projects,
request_user=request_user, reason=reason)
def unsuspend_project_on_condition(project, request_user=None, reason=None,
......
snf-astakos-app/astakos/im/messages.py
View file @ 6a71c1a2
# Copyright (C) 2010-2014 GRNET S.A.
# Copyright (C) 2010-2016 GRNET S.A.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
......@@ -210,6 +210,7 @@ UNIQUE_PROJECT_NAME_CONSTRAIN_ERR = (
'The project name (as specified in its application\'s definition) must '
'be unique among all active projects.')
NOT_ALIVE_PROJECT = 'Project %s is not alive.'
NOT_INITIALIZED_PROJECT = 'Project %s is not initialized.'
SUSPENDED_PROJECT = 'Project %s is suspended.'
NOT_SUSPENDED_PROJECT = 'Project %s is not suspended.'
NOT_TERMINATED_PROJECT = 'Project %s is not terminated.'
......
snf-astakos-app/astakos/im/quotas.py
View file @ 6a71c1a2
......@@ -13,13 +13,12 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import operator
from astakos.im.models import (
Resource, AstakosUser, Service,
Project, ProjectMembership, ProjectResourceQuota)
import astakos.quotaholder_app.callpoint as qh
from astakos.quotaholder_app.exception import NoCapacityError
from django.db.models import Q, F
from django.db.models import Q
from collections import defaultdict
......@@ -332,102 +331,3 @@ def qh_sync_new_resource(resource):
member_capacity=limit))
ProjectResourceQuota.objects.bulk_create(entries)
qh_sync_projects(projects, resource=resource.name)
def _mk_quota_per_project_verbose(quotas):
"""
This function constructs a dictionary which contains the usage and limit of
of resources of users for every project.
Args:
quotas: Quotas dictionary.
Returns:
Dictionary keyed by project uuid and it contains detailed info about
user's resources limits and usages.
"""
overquotas_users = defaultdict(lambda: defaultdict(dict))
for (u_uuid, p_uuid, resource), usage in quotas.iteritems():
project = p_uuid.split(PROJECT_TAG)[1]
user = u_uuid.split(USER_TAG)[1]
value = from_holding(usage)
overquotas_users[project][user][resource] = value
return overquotas_users
def _mk_quota_per_project(quotas):
"""
This functions constructs a dictionary keyed by project and it contains
a list of uuids of users who own resources
Args:
quotas: Quotas dictionary.
Returns:
Dictionary keyed by project uuid and it contains a list of uuids of
its overquota users.
"""
overquotas_users = defaultdict(lambda: [])
for u_uuid, p_uuid, _ in quotas.keys():
project = p_uuid.split(PROJECT_TAG)[1]
user = u_uuid.split(USER_TAG)[1]
overquotas_users[project].append(user)
return overquotas_users
def get_overquota_users(projects, services, resource=None, verbose=False):
"""
Get a dictionary of overquota users per project.
This function gets users who are using resources of specific services and
surpass their current limit.
Args:
projects: Projects to get their overquota users.
services: List of services whose resources are associated
with, e.g. cyclades.
resource: Resources to check if user is overquota.
verbose: True if returned value should contain detailed info about
resources usage.
Returns:
1) Dictionary keyed by project uuid and it contains a list of uuids of
its overquota users if verbose is `False`
2) Dictionary keyed by project uuid and it contains detailed info about
user's resources limits and usages if verbose is `True`.
"""
users = []
for project in projects:
users += [get_user_ref(user) for user in project.members.all()]
projects = [get_project_ref(project) for project in projects]
flt = Q()
flt &= Q(usage_max__gt=F("limit"))
flt &= reduce(operator.or_, [Q(resource__startswith=service)
for service in services])
quotas = qh.get_quota(holders=users, sources=projects,
resources=resource, flt=flt)
return _mk_quota_per_project_verbose(quotas) if verbose\
else _mk_quota_per_project(quotas)
def is_membership_overquota(membership, services, resource=None):
"""
This function checks if a specific membership is overquota.
Args:
membership: Membership to check if it's overquota.
services: List of services which resources are associated
with, e.g. cyclades.
resource: Resources to check if membership is overquota.
Returns:
True if membership is overquota; False otherwise.
"""
user = get_user_ref(membership.person)
project = get_project_ref(membership.project)
flt = Q()
flt &= Q(usage_max__gt=F("limit"))
flt &= reduce(operator.or_, [Q(resource__startswith=service)
for service in services])
return len(qh.get_quota(holders=[user], sources=[project],
resources=resource, flt=flt)) != 0
snf-astakos-app/astakos/im/tests/api.py
View file @ 6a71c1a2
# -*- coding: utf-8 -*-
# Copyright (C) 2010-2014 GRNET S.A.
# Copyright (C) 2010-2016 GRNET S.A.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
......@@ -418,16 +418,10 @@ class TokensApiTest(TestCase):
def setUp(self):
backend = activation_backends.get_backend()
self.user1 = get_local_user(
'test1@example.org', email_verified=True, moderated=True,
is_rejected=False)
backend.activate_user(self.user1)
self.user1 = get_local_user('test1@example.org')
assert self.user1.is_active is True
self.user2 = get_local_user(
'test2@example.org', email_verified=True, moderated=True,
is_rejected=False)
backend.activate_user(self.user2)
self.user2 = get_local_user('test2@example.org')
assert self.user2.is_active is True
c1 = Component(name='component1', url='http://localhost/component1')
......@@ -661,8 +655,7 @@ class TokensApiTest(TestCase):
class UserCatalogsTest(TestCase):
def test_get_uuid_displayname_catalogs(self):
self.user = get_local_user(
'test1@example.org', email_verified=True, moderated=True,
is_rejected=False, is_active=False)
'test1@example.org', email_verified=True, is_active=False)
client = Client()
url = reverse('astakos.api.user.get_uuid_displayname_catalogs')
......@@ -689,7 +682,8 @@ class UserCatalogsTest(TestCase):
self.assertEqual(r.status_code, 401)
backend = activation_backends.get_backend()
backend.activate_user(self.user)
backend.verify_user(self.user, self.user.verification_code)
backend.accept_user(self.user)
assert self.user.is_active is True
r = client.post(url,
......
snf-astakos-app/astakos/im/tests/projects.py
View file @ 6a71c1a2
# -*- coding: utf-8 -*-
# Copyright (C) 2010-2014 GRNET S.A.
# Copyright (C) 2010-2016 GRNET S.A.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
......@@ -60,6 +60,7 @@ class ProjectAPITest(TestCase):
self.user2.uuid = "uuid2"
self.user2.save()
self.user3 = get_local_user("test3@grnet.gr")
self.user4 = get_local_user("test4@grnet.gr", is_active=False)
astakos = Component.objects.create(name="astakos")
register.add_service(astakos, "astakos_account", "account", [])
......@@ -375,6 +376,10 @@ class ProjectAPITest(TestCase):
status, body = self.enroll(project_id, self.user1, h_owner)
self.assertEqual(status, 409)
# Enroll fails, user is not active
status, body = self.enroll(project_id, self.user4, h_owner)
self.assertEqual(status, 409)
# Enroll fails, project does not exist
status, body = self.enroll(-1, self.user1, h_owner)
self.assertEqual(status, 409)
......@@ -731,6 +736,84 @@ class ProjectAPITest(TestCase):
self.assertEqual(admin_pa3, admin_pa2)
self.assertEqual(owner_pa3, owner_pa2)
@im_settings(PROJECT_ADMINS=["uuid2"])
def test_suspend_deactivated(self):
client = self.client
h_owner = {"HTTP_X_AUTH_TOKEN": self.user1.auth_token}
h_admin = {"HTTP_X_AUTH_TOKEN": self.user2.auth_token}
h_plain = {"HTTP_X_AUTH_TOKEN": self.user3.auth_token}
app1 = {"name": "test.pr",
"description": u"δεσκρίπτιον",
"end_date": "2113-5-5T20:20:20Z",
"join_policy": "auto",
"max_members": 5,
"resources": {u"σέρβις1.ρίσορς11": {
"project_capacity": 1024,
"member_capacity": 512}}
}
# Create
status, body = self.create(app1, h_owner)
self.assertEqual(status, 201)
project_id = body["id"]
app_id = body["application"]
# Approve
status = self.project_action(project_id, "approve", app_id,
headers=h_admin)
self.assertEqual(status, 200)
# Enroll
status, body = self.enroll(project_id, self.user1, h_owner)
self.assertEqual(status, 200)
m_plain_id = body["id"]
# Check user memberships
memberships = self.user1.projectmembership_set.all()
self.assertEqual(len(memberships), 2)
for membership in memberships:
self.assertEqual(membership.state, membership.ACCEPTED)
# Deactivate user
backend = activation_backends.get_backend()