Commit 6a71c1a2 authored by Giorgos Korfiatis's avatar Giorgos Korfiatis

Merge branch 'hotfix-0.18.1'

parents 28b8a1d5 f38c1729
...@@ -6,12 +6,10 @@ Unified Changelog file for Synnefo versions >= 0.13 ...@@ -6,12 +6,10 @@ Unified Changelog file for Synnefo versions >= 0.13
Since v0.13 most of the Synnefo components have been merged into a single Since v0.13 most of the Synnefo components have been merged into a single
repository and have aligned versions. repository and have aligned versions.
.. _Changelog-0.18: .. _Changelog-0.18.1:
v0.18 v0.18.1
======== =======
Released: Wed 7 Sep 16:50:30 EEST 2016
Astakos Astakos
------- -------
...@@ -23,7 +21,7 @@ Astakos ...@@ -23,7 +21,7 @@ Astakos
* Support suspending and unsuspending project memberships. * Support suspending and unsuspending project memberships.
* User deactivation now automatically suspends user's system project, owned * User deactivation now automatically suspends user's system project, owned
projects, and project memberships. Reactivation unsuspends them. projects, and project memberships. Reactivation unsuspends them.
* Add command `user-check`. It supports suspending projects for previously * Add script `fix_deactivated_users` to suspend projects for previously
deactivated users. deactivated users.
* Send an informative email to the user's current email address when they * Send an informative email to the user's current email address when they
request to change their email. request to change their email.
......
...@@ -5,14 +5,14 @@ Unified NEWS file for Synnefo versions >= 0.13 ...@@ -5,14 +5,14 @@ Unified NEWS file for Synnefo versions >= 0.13
Since v0.13 all Synnefo components have been merged into a single repository. Since v0.13 all Synnefo components have been merged into a single repository.
.. _NEWS-0.18: .. _NEWS-0.18.1:
v0.18 v0.18.1
===== =======
Released: Wed 7 Sep 16:50:30 EEST 2016 Released: UNRELEASED
The Synnefo 0.18 release brings significant bug fixes across Synnefo. The Synnefo 0.18.1 release brings significant improvements across Synnefo.
The most notable changes are: The most notable changes are:
......
...@@ -3077,7 +3077,7 @@ Upgrade Notes ...@@ -3077,7 +3077,7 @@ Upgrade Notes
v0.15 -> v0.16 <upgrade/upgrade-0.16> v0.15 -> v0.16 <upgrade/upgrade-0.16>
v0.16.1 -> v0.16.2 <upgrade/upgrade-0.16.2> v0.16.1 -> v0.16.2 <upgrade/upgrade-0.16.2>
v0.16.2 -> v0.17 <upgrade/upgrade-0.17> v0.16.2 -> v0.17 <upgrade/upgrade-0.17>
v0.17 -> v0.18 <upgrade/upgrade-0.18> v0.17 -> v0.18.1 <upgrade/upgrade-0.18.1>
.. _changelog-news: .. _changelog-news:
...@@ -3086,7 +3086,7 @@ Changelog, NEWS ...@@ -3086,7 +3086,7 @@ Changelog, NEWS
=============== ===============
* v0.18 :ref:`Changelog <Changelog-0.18>`, :ref:`NEWS <NEWS-0.18>` * v0.18.1 :ref:`Changelog <Changelog-0.18.1>`, :ref:`NEWS <NEWS-0.18.1>`
* v0.17 :ref:`Changelog <Changelog-0.17>`, :ref:`NEWS <NEWS-0.17>` * v0.17 :ref:`Changelog <Changelog-0.17>`, :ref:`NEWS <NEWS-0.17>`
* v0.16.2 :ref:`Changelog <Changelog-0.16.2>`, :ref:`NEWS <NEWS-0.16.2>` * v0.16.2 :ref:`Changelog <Changelog-0.16.2>`, :ref:`NEWS <NEWS-0.16.2>`
* v0.16.1 :ref:`Changelog <Changelog-0.16.1>`, :ref:`NEWS <NEWS-0.16.1>` * v0.16.1 :ref:`Changelog <Changelog-0.16.1>`, :ref:`NEWS <NEWS-0.16.1>`
......
Upgrade to Synnefo v0.18.1
^^^^^^^^^^^^^^^^^^^^^^^^^^
Upgrade Steps
=============
The upgrade from v0.17 to v0.18.1 consists of the following steps:
#. Bring down services::
$ service gunicorn stop
$ service snf-dispatcher stop
$ service snf-ganeti-eventd stop
#. Upgrade Synnefo on all nodes to the latest version::
astakos.host$ apt-get install \
snf-common \
python-astakosclient \
snf-django-lib \
snf-webproject \
snf-branding \
snf-astakos-app
cyclades.host$ apt-get install \
snf-common \
python-astakosclient \
snf-django-lib \
snf-webproject \
snf-branding \
snf-pithos-backend \
snf-cyclades-app
pithos.host$ apt-get install \
snf-common \
python-astakosclient \
snf-django-lib \
snf-webproject \
snf-branding \
snf-pithos-backend \
snf-pithos-app \
snf-ui-app
ganeti.node$ apt-get install \
snf-common \
snf-cyclades-gtools \
snf-pithos-backend
#. Run migrations on Astakos.
.. code-block:: console
astakos.host$ snf-manage migrate
From this version on, user deactivation triggers suspension of all projects
and project memberships related to the user. To apply this new policy to
users that have already been deactivated, run:
.. code-block:: console
astakos.host$ /usr/lib/astakos/tools/fix_deactivated_users --all-users --noemail --fix
#. Restart services
.. code-block:: console
$ service gunicorn start
$ service snf-dispatcher start
$ service snf-ganeti-eventd start
New configuration options
=========================
On the admin app, there is a new access control option regarding the new modify
email action. The action setting is named 'modify_email'. The list of user
groups defined in this have access on the modify email action.
The following line (modified accordingly) should be added on 'ADMIN_RBAC'
setting under the 'user' dictionary:
.. code-block:: console
'modify_email': [ADMIN_HELPDESK_GROUP, ADMIN_GROUP],
Upgrade to Synnefo v0.18 Upgrade to Synnefo v0.18
^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^
Upgrade Steps Version 0.18 is not recommended; upgrade directly to 0.18.1.
=============
The upgrade to v0.18 consists of the following steps:
#. Stop gunicorn in all nodes
.. code-block:: console
# service gunicorn stop
#. Upgrade Synnefo on all nodes to the latest version (0.18)
.. code-block:: console
# apt-get update
# apt-get upgrade
#. Run migrations on Astakos.
.. code-block:: console
astakos.host$ snf-manage migrate
From this version on, user deactivation triggers suspension of all projects
and project memberships related to the user. To apply this new policy to
users that have already been deactivated, run:
.. code-block:: console
astakos.host$ snf-manage user-check --all-users --suspend-deactivated --noemail --fix
#. Start gunicorn
.. code-block:: console
# service gunicorn start
New configuration options
=========================
On the admin app, there is a new access control option regarding the new modify
email action. The action setting is named 'modify_email'. The list of user
groups defined in this have access on the modify email action.
The following line (modified accordingly) should be added on 'ADMIN_RBAC'
setting under the 'user' dictionary:
.. code-block:: console
'modify_email': [ADMIN_HELPDESK_GROUP, ADMIN_GROUP],
...@@ -245,6 +245,7 @@ ...@@ -245,6 +245,7 @@
&.info-data { &.info-data {
padding: 0 20px; padding: 0 20px;
white-space: pre-wrap;
} }
} }
......
...@@ -500,7 +500,7 @@ def flatten_dict_to_dl(d, default_if_empty='-'): ...@@ -500,7 +500,7 @@ def flatten_dict_to_dl(d, default_if_empty='-'):
if isinstance(v, dict): if isinstance(v, dict):
stack.extend(v.iteritems()) stack.extend(v.iteritems())
else: else:
a = '<dt>{0}</dt><dd>{1}</dd>'.format(k, v or default_if_empty) a = u'<dt>{0}</dt><dd>{1}</dd>'.format(k, v or default_if_empty)
l.append(a) l.append(a)
return mark_safe(''.join(reversed(l))) return mark_safe(''.join(reversed(l)))
......
...@@ -357,7 +357,7 @@ class ActivationBackend(object): ...@@ -357,7 +357,7 @@ class ActivationBackend(object):
if not ok: if not ok:
return ActivationResult(self.Result.ERROR, msg) return ActivationResult(self.Result.ERROR, msg)
was_deactivated = bool(user.deactivated_at) was_deactivated = not user.is_active
user.is_active = True user.is_active = True
user.deactivated_reason = None user.deactivated_reason = None
user.deactivated_at = None user.deactivated_at = None
......
# Copyright (C) 2010-2014 GRNET S.A. # Copyright (C) 2010-2016 GRNET S.A.
# #
# This program is free software: you can redistribute it and/or modify # This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by # it under the terms of the GNU General Public License as published by
...@@ -1192,7 +1192,10 @@ class AddProjectMembersForm(forms.Form): ...@@ -1192,7 +1192,10 @@ class AddProjectMembersForm(forms.Form):
q = self.cleaned_data.get('q') or '' q = self.cleaned_data.get('q') or ''
users = re.split("\r\n|\n|,", q) users = re.split("\r\n|\n|,", q)
users = list(u.strip() for u in users if u) users = list(u.strip() for u in users if u)
db_entries = AstakosUser.objects.accepted().filter(email__in=users)
# Notice that deactivated users are reported as 'unknown' here, too.
db_entries = AstakosUser.objects.accepted().filter(
email__in=users, is_active=True)
unknown = list(set(users) - set(u.email for u in db_entries)) unknown = list(set(users) - set(u.email for u in db_entries))
if unknown: if unknown:
raise forms.ValidationError( raise forms.ValidationError(
......
...@@ -218,16 +218,20 @@ def app_check_allowed(application, request_user, ...@@ -218,16 +218,20 @@ def app_check_allowed(application, request_user,
return _failure(silent) return _failure(silent)
def checkAlive(project, silent=False): def checkAlive(project, silent=False, check_initialized=False):
def fail(msg): def fail(msg):
if silent: if silent:
return False, msg return False, msg
else: else:
raise ProjectConflict(msg) raise ProjectConflict(msg)
if check_initialized:
if not project.is_alive: if not project.is_initialized():
m = _(astakos_messages.NOT_ALIVE_PROJECT) % project.uuid m = _(astakos_messages.NOT_INITIALIZED_PROJECT) % project.uuid
return fail(m) return fail(m)
else:
if not project.is_alive:
m = _(astakos_messages.NOT_ALIVE_PROJECT) % project.uuid
return fail(m)
return True, None return True, None
...@@ -249,7 +253,9 @@ def accept_membership_checks(membership, request_user): ...@@ -249,7 +253,9 @@ def accept_membership_checks(membership, request_user):
if not membership.check_action("accept"): if not membership.check_action("accept"):
m = _(astakos_messages.NOT_MEMBERSHIP_REQUEST) m = _(astakos_messages.NOT_MEMBERSHIP_REQUEST)
raise ProjectConflict(m) raise ProjectConflict(m)
if not membership.person.is_active:
m = _(astakos_messages.ACCOUNT_NOT_ACTIVE)
raise ProjectConflict(m)
project = membership.project project = membership.project
accept_membership_project_checks(project, request_user) accept_membership_project_checks(project, request_user)
...@@ -350,7 +356,7 @@ def suspend_membership_checks(membership, request_user=None): ...@@ -350,7 +356,7 @@ def suspend_membership_checks(membership, request_user=None):
project = membership.project project = membership.project
project_check_allowed(project, request_user, level=ADMIN_LEVEL) project_check_allowed(project, request_user, level=ADMIN_LEVEL)
checkAlive(project) checkAlive(project, check_initialized=True)
def suspend_membership(memb_id, request_user=None, reason=None): def suspend_membership(memb_id, request_user=None, reason=None):
...@@ -376,7 +382,7 @@ def unsuspend_membership_checks(membership, request_user=None): ...@@ -376,7 +382,7 @@ def unsuspend_membership_checks(membership, request_user=None):
project = membership.project project = membership.project
project_check_allowed(project, request_user, level=ADMIN_LEVEL) project_check_allowed(project, request_user, level=ADMIN_LEVEL)
checkAlive(project) checkAlive(project, check_initialized=True)
def unsuspend_membership(memb_id, request_user=None, reason=None): def unsuspend_membership(memb_id, request_user=None, reason=None):
...@@ -403,12 +409,19 @@ def enroll_member_by_email(project_id, email, request_user=None, reason=None): ...@@ -403,12 +409,19 @@ def enroll_member_by_email(project_id, email, request_user=None, reason=None):
raise ProjectConflict(astakos_messages.UNKNOWN_USERS % email) raise ProjectConflict(astakos_messages.UNKNOWN_USERS % email)
def enroll_member_checks(project, user, request_user):
if not user.is_active:
m = _(astakos_messages.ACCOUNT_NOT_ACTIVE)
raise ProjectConflict(m)
accept_membership_project_checks(project, request_user)
def enroll_member(project_id, user, request_user=None, reason=None): def enroll_member(project_id, user, request_user=None, reason=None):
try: try:
project = get_project_for_update(project_id) project = get_project_for_update(project_id)
except ProjectNotFound as e: except ProjectNotFound as e:
raise ProjectConflict(e.message) raise ProjectConflict(e.message)
accept_membership_project_checks(project, request_user) enroll_member_checks(project, user, request_user)
try: try:
membership = get_membership(project.id, user.id) membership = get_membership(project.id, user.id)
...@@ -1248,55 +1261,59 @@ def _get_owned_projects(user_ids): ...@@ -1248,55 +1261,59 @@ def _get_owned_projects(user_ids):
return _partition_by(lambda p: p.owner_id, ps) return _partition_by(lambda p: p.owner_id, ps)
def suspend_users_projects(users, request_user=None, reason=None, fix=True): def get_projects_and_memberships_of_users(users):
affected_users = []
user_ids = [user.id for user in users] user_ids = [user.id for user in users]
if not user_ids: if not user_ids:
return affected_users return {}, {}, {}
base_projects_d = _get_base_projects(user_ids) base_projects_d = _get_base_projects(user_ids)
memberships_d = _get_memberships(user_ids) memberships_d = _get_memberships(user_ids)
owned_projects_d = _get_owned_projects(user_ids) owned_projects_d = _get_owned_projects(user_ids)
return base_projects_d, memberships_d, owned_projects_d
for user in users:
is_affected = False def suspend_user_projects_and_memberships(
base_project = base_projects_d[user.base_project_id] user, base_project, memberships, owned_projects,
if base_project.state == Project.NORMAL: request_user=None, reason=None, fix=True):
try: is_affected = False
if fix: if base_project.state == Project.NORMAL:
suspend(user.base_project_id, try:
request_user=request_user, reason=reason) if fix:
is_affected = True suspend(user.base_project_id,
except ProjectError: request_user=request_user, reason=reason)
pass is_affected = True
memberships = memberships_d.get(user.id, []) except ProjectError:
for membership in memberships: pass
try: for membership in memberships:
if fix: try:
suspend_membership( if fix:
membership.id, request_user=request_user, suspend_membership(
reason=reason) membership.id, request_user=request_user,
is_affected = True reason=reason)
except ProjectError: is_affected = True
pass except ProjectError:
pass
owned_projects = owned_projects_d.get(user.id, [])
for project in owned_projects: for project in owned_projects:
try: try:
if fix: if fix:
suspend( suspend(
project.id, request_user=request_user, reason=reason) project.id, request_user=request_user, reason=reason)
is_affected = True is_affected = True
except ProjectError: except ProjectError:
pass pass
if is_affected: return is_affected
affected_users.append(user)
return affected_users
def suspend_user_projects(user, request_user=None, reason=None): def suspend_user_projects(user, request_user=None, reason=None):
return suspend_users_projects( bpd, md, opd = get_projects_and_memberships_of_users([user])
[user], request_user=request_user, reason=reason) base_project = bpd[user.base_project_id]
memberships = md.get(user.id, [])
owned_projects = opd.get(user.id, [])
return suspend_user_projects_and_memberships(
user, base_project, memberships, owned_projects,
request_user=request_user, reason=reason)
def unsuspend_project_on_condition(project, request_user=None, reason=None, def unsuspend_project_on_condition(project, request_user=None, reason=None,
......
# Copyright (C) 2010-2014 GRNET S.A. # Copyright (C) 2010-2016 GRNET S.A.
# #
# This program is free software: you can redistribute it and/or modify # This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by # it under the terms of the GNU General Public License as published by
...@@ -210,6 +210,7 @@ UNIQUE_PROJECT_NAME_CONSTRAIN_ERR = ( ...@@ -210,6 +210,7 @@ UNIQUE_PROJECT_NAME_CONSTRAIN_ERR = (
'The project name (as specified in its application\'s definition) must ' 'The project name (as specified in its application\'s definition) must '
'be unique among all active projects.') 'be unique among all active projects.')
NOT_ALIVE_PROJECT = 'Project %s is not alive.' NOT_ALIVE_PROJECT = 'Project %s is not alive.'
NOT_INITIALIZED_PROJECT = 'Project %s is not initialized.'
SUSPENDED_PROJECT = 'Project %s is suspended.' SUSPENDED_PROJECT = 'Project %s is suspended.'
NOT_SUSPENDED_PROJECT = 'Project %s is not suspended.' NOT_SUSPENDED_PROJECT = 'Project %s is not suspended.'
NOT_TERMINATED_PROJECT = 'Project %s is not terminated.' NOT_TERMINATED_PROJECT = 'Project %s is not terminated.'
......
...@@ -13,13 +13,12 @@ ...@@ -13,13 +13,12 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
import operator
from astakos.im.models import ( from astakos.im.models import (
Resource, AstakosUser, Service, Resource, AstakosUser, Service,
Project, ProjectMembership, ProjectResourceQuota) Project, ProjectMembership, ProjectResourceQuota)
import astakos.quotaholder_app.callpoint as qh import astakos.quotaholder_app.callpoint as qh
from astakos.quotaholder_app.exception import NoCapacityError from astakos.quotaholder_app.exception import NoCapacityError
from django.db.models import Q, F from django.db.models import Q
from collections import defaultdict from collections import defaultdict
...@@ -332,102 +331,3 @@ def qh_sync_new_resource(resource): ...@@ -332,102 +331,3 @@ def qh_sync_new_resource(resource):
member_capacity=limit)) member_capacity=limit))
ProjectResourceQuota.objects.bulk_create(entries) ProjectResourceQuota.objects.bulk_create(entries)
qh_sync_projects(projects, resource=resource.name) qh_sync_projects(projects, resource=resource.name)
def _mk_quota_per_project_verbose(quotas):
"""
This function constructs a dictionary which contains the usage and limit of
of resources of users for every project.
Args:
quotas: Quotas dictionary.
Returns:
Dictionary keyed by project uuid and it contains detailed info about
user's resources limits and usages.
"""
overquotas_users = defaultdict(lambda: defaultdict(dict))
for (u_uuid, p_uuid, resource), usage in quotas.iteritems():
project = p_uuid.split(PROJECT_TAG)[1]
user = u_uuid.split(USER_TAG)[1]
value = from_holding(usage)
overquotas_users[project][user][resource] = value
return overquotas_users
def _mk_quota_per_project(quotas):
"""
This functions constructs a dictionary keyed by project and it contains
a list of uuids of users who own resources
Args:
quotas: Quotas dictionary.
Returns:
Dictionary keyed by project uuid and it contains a list of uuids of
its overquota users.
"""
overquotas_users = defaultdict(lambda: [])
for u_uuid, p_uuid, _ in quotas.keys():
project = p_uuid.split(PROJECT_TAG)[1]
user = u_uuid.split(USER_TAG)[1]
overquotas_users[project].append(user)
return overquotas_users
def get_overquota_users(projects, services, resource=None, verbose=False):
"""
Get a dictionary of overquota users per project.
This function gets users who are using resources of specific services and
surpass their current limit.
Args:
projects: Projects to get their overquota users.
services: List of services whose resources are associated
with, e.g. cyclades.
resource: Resources to check if user is overquota.
verbose: True if returned value should contain detailed info about
resources usage.
Returns:
1) Dictionary keyed by project uuid and it contains a list of uuids of
its overquota users if verbose is `False`
2) Dictionary keyed by project uuid and it contains detailed info about
user's resources limits and usages if verbose is `True`.
"""
users = []
for project in projects:
users += [get_user_ref(user) for user in project.members.all()]
projects = [get_project_ref(project) for project in projects]
flt = Q()
flt &= Q(usage_max__gt=F("limit"))
flt &= reduce(operator.or_, [Q(resource__startswith=service)
for service in services])
quotas = qh.get_quota(holders=users, sources=projects,