Commit e3e8c90f authored by Stratos Psomadakis's avatar Stratos Psomadakis

Merge branch 'hotfix-0.16.2'

parents 3c49605b c696fef1
Dimitris Aragiorgis <dimitris.aragiorgis@gmail.com>
......@@ -6,6 +6,101 @@ Unified Changelog file for Synnefo versions >= 0.13
Since v0.13 most of the Synnefo components have been merged into a single
repository and have aligned versions.
.. _Changelog-0.16.2:
v0.16.2
=======
Released: Fri Jul 24 14:13:24 EEST 2015
Changelog
---------
* Switch to a more verbose Changelog style, which includes more details
regarding the commits of each relase. A summary of the changes is provided
with the NEWS file.
Admin
-----
* Handle unicode chars in VM search field.
* Check usage to decide if resource is useful.
* Display last login per auth provider if more than one exists.
* Display `comments for review' in project applications.
* Display effective limit in user's quotas.
* Fix IP updated time since timestamp.
* Fetch image info using UUID and version.
Astakos
-------
* Always accept terms for users created via admin API.
* Allow admin to update the 'signed terms' date for a user via the 'signTerms'
admin API action.
* Improve admin API user creation process, in case that user moderation is
disabled.
* Provide South migration to fix initialized state of enrolled memberships.
* Handle empty project limit in application submission form.
* Include terminated projects in project list views.
* Display UUID in expired projects list.
* Fix checkbox behaviour in project members list.
* Fix style error in project creation form.
* Add project_creation_date column to list.
* Fix UI project tests.
* Route oa2 models to Astakos database in Synnefo DB router.
* Catch a couple of potential 500 errors.
* Fix auth_provider limit enforcement.
Bunrin
------
* Fix a race in public network detach.
* Handle exceptions and retry during SSH.
* Create private networks without a gateway.
CI
--
* Pin Debian Wheezy image during ci runs.
* Fix CI image flavor regexp.
Cyclades
--------
* Round up memory sizes in flavor create.
* Change volume ownership on server-modify.
Cyclades UI
-----------
* Add UI_IMAGE_LISTING_USERS setting to enable custom defined image sections
in image selection view of the VM creation wizard. The sections are defined
using the LISTING_SECTION image property.
* Change default private network gateway to 'None'.
* Include machine icon images for bitnami.
Deploy
------
* Substitute IPs with FQDNs.
* Fix CA creation.
* Use domain in ServerName.
* Update qa-init.sh for Ganeti.
* Disable v6 on loopback interface.
Documentation
-------------
* Update install guide to use Debian Wheezy.
Pithos
------
* Fix off-by-one error in Pithos API tests.
.. _Changelog-0.16.1:
v0.16.1
......
......@@ -5,6 +5,29 @@ Unified NEWS file for Synnefo versions >= 0.13
Since v0.13 all Synnefo components have been merged into a single repository.
.. _NEWS-0.16.2:
v0.16.2
=======
Released: Fri Jul 24 14:13:49 EEST 2015
The Synnefo 0.16.2 release is mostly a bug fix version. The most notable
changes are:
* Added custom image sections in the Cyclades UI.
* Round up memory sizes in flavor create.
* Change volume ownership, when reassigning VMs.
* Improved handling of projects (both API- and UI-wise).
* Provide South migration to fix initialized state of enrolled project
memberships.
* Add a `signTerms` admin API action.
* Include machine icon images for bitnami.
* Usability fixes for the admin app.
* Updates and fixes for snf-deploy.
* Various bug fixes and improvements across Synnefo. Please see the Synnefo
:ref:`Changelog <Changelog-0.16.1>` for a complete list.
.. _NEWS-0.16.1:
v0.16.1
......
......@@ -47,7 +47,7 @@ for more information on the Synnefo users and developers lists.
Copyright and license
=====================
Copyright (C) 2010-2014 GRNET S.A.
Copyright (C) 2010-2015 GRNET S.A. and individual contributors
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
......
......@@ -2,7 +2,7 @@
# Timeouts in seconds
build_timeout = 240
# Apt repository to use
apt_repo =
apt_repo = # Synnefo and X2Go repositories
deb http://apt.dev.grnet.gr wheezy/
deb http://packages.x2go.org/debian wheezy main
......@@ -57,7 +57,7 @@ flavors = name:C2R4...D20ext_.*, name:C2R4...D20drbd, id:1
# A list of images (comma seperated) to choose from
# The user can specify an image name (reg expression)
# with "name:" or an image id with "id:".
images = name:SynnefoCIWheezy.*, name:^Debian Base$, id:72d9844f-1024-4a07-a3c3-60d650b8f5cd
images = name:SynnefoCIWheezy.*, name:^Debian Base \(OldStable\)$, id:a1a2ff80-3371-473e-87e9-3389103bb80e
# File containing the ssh keys to upload/install to server
# If not set, no ssh keys will be installed
ssh_keys = ~/.ssh/id_rsa.pub
......
This diff is collapsed.
......@@ -36,6 +36,12 @@ which serves Synnefo by default, needs read access to the configuration
files and we don't want it to run as root, it must run with group
``synnefo``.
.. warning:: If you want to add your own configuration file, do not forget to
declare the appropriate encoding by adding the line
``## -*- coding: utf-8 -*-`` at the beggining of the file.
Cyclades and Pithos talk to Archipelago over some named pipes under
``/dev/shm/posixfd``. This directory is created by Archipelago, owned by
the user/group that Archipelago runs as, and at the same time it must be
......@@ -175,10 +181,12 @@ Click Create an application.
Fill the necessary information and for callback URL give::
https://node1.example.com/ui/login/twitter/authenticated
https://node1.example.com/astakos/ui/login/twitter/authenticated
Finally, add 'twitter' in ``ASTAKOS_IM_MODULES`` list. The variable resides
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf``
Edit ``/etc/synnefo/20-snf-astakos-app-settings.conf`` and set the
corresponding variables ``ASTAKOS_TWITTER_TOKEN`` and
``ASTAKOS_TWITTER_SECRET`` to reflect your newly created pair.
Finally, add 'twitter' in ``ASTAKOS_IM_MODULES`` list.
Google Authentication
~~~~~~~~~~~~~~~~~~~~~
......@@ -188,15 +196,14 @@ visit https://code.google.com/apis/console/.
Under API Access select Create another client ID, select Web application,
expand more options in Your site or hostname section and in Authorized
Redirect URIs add:
Fill the necessary information and for callback URL give::
Redirect URIs add::
https://node1.example.com/ui/login/google/authenticated
https://node1.example.com/astakos/ui/login/google/authenticated
Finally, add 'google' in ``ASTAKOS_IM_MODULES`` list. The variable resides
inside the file ``/etc/synnefo/20-snf-astakos-app-settings.conf``
Edit ``/etc/synnefo/20-snf-astakos-app-settings.conf`` and set the
corresponding variables ``ASTAKOS_GOOGLE_CLIENT_ID`` and
``ASTAKOS_GOOGLE_SECRET`` to reflect your newly created pair.
Finally, add 'google' in ``ASTAKOS_IM_MODULES`` list.
Working with Astakos
......@@ -311,7 +318,11 @@ Currently astakos supports the following identity providers:
(module name ``linkedin``)
To enable any of the above modules (by default only ``local`` accounts are
allowed), retrieve and set the required provider settings and append the
allowed) you have to install oauth2 package. To do so run::
apt-get install python-oauth2
Then retrieve and set the required provider settings and append the
module name in ``ASTAKOS_IM_MODULES``.
.. code-block:: python
......@@ -323,7 +334,6 @@ module name in ``ASTAKOS_IM_MODULES``.
# let users signup and login using their google account
ASTAKOS_IM_MODULES = ['local', 'google']
.. _auth_methods_policies:
Authentication method policies
......@@ -889,11 +899,22 @@ image to Cyclades:
Deletion of an image is done via `kamaki image unregister` command, which will
delete the Cyclades Images but will leave the Pithos file as is (unregister).
Apart from using `kamaki` to see and hangle the available images, the
Apart from using `kamaki` to see and handle the available images, the
administrator can use `snf-manage image-list` and `snf-manage image-show`
commands to list and inspect the available public images. Also, the `--user`
option can be used the see the images of a specific user.
Custom image listing sections
`````````````````````````````
Since Synnefo 0.16.2, the installation wizard supports custom image listing
sections. Images with the ``LISTING_SECTION`` image property set, and whose
owner uuid is listed in the ``UI_IMAGE_LISTING_USERS`` Cyclades setting (in
``/etc/synnefo/20-snf-cyclades-app-ui.conf``) will be displayed in a separate
section in the installation wizard. The name of the new section will be the
value of the ``LISTING_SECTION`` image property.
Virtual Servers
~~~~~~~~~~~~~~~
......@@ -2958,6 +2979,7 @@ Upgrade Notes
v0.14 -> v0.15 <upgrade/upgrade-0.15>
v0.15 -> v0.15.1 <upgrade/upgrade-0.15.1>
v0.15 -> v0.16 <upgrade/upgrade-0.16>
v0.16.1 -> v0.16.2 <upgrade/upgrade-0.16.2>
.. _changelog-news:
......@@ -2966,6 +2988,7 @@ Changelog, NEWS
===============
* v0.16.2 :ref:`Changelog <Changelog-0.16.2>`, :ref:`NEWS <NEWS-0.16.2>`
* v0.16.1 :ref:`Changelog <Changelog-0.16.1>`, :ref:`NEWS <NEWS-0.16.1>`
* v0.16 :ref:`Changelog <Changelog-0.16>`, :ref:`NEWS <NEWS-0.16>`
* v0.15.2 :ref:`Changelog <Changelog-0.15.1>`, :ref:`NEWS <NEWS-0.15.2>`
......
......@@ -8,7 +8,7 @@ reload(synnefo.versions)
from synnefo.versions.app import __version__
project = u'synnefo'
copyright = u'2012-2014, GRNET'
copyright = u'2012-2015, GRNET'
version = __version__
release = __version__
html_title = 'synnefo ' + version
......
This diff is collapsed.
Upgrade to Synnefo v0.16.2
^^^^^^^^^^^^^^^^^^^^^^^^^^
Since Synnefo v0.16.2 is mostly a bug fix version, the downtime required for
the upgrade should be minimal. Following the usual process, bring the services
down and upgrade the packages. Before bringing the services back up, you should
run `snf-manage migrate` on the astakos node, in order to fix the quotas of
users enrolled but not accepted in projects, which might be out-of-sync.
......@@ -20,6 +20,7 @@ from operator import or_, and_
from django.db.models import Q
from django.core.exceptions import FieldError
from django.conf import settings
from django.utils.encoding import smart_unicode
from synnefo_admin.admin.utils import model_dict
from synnefo_admin import admin_settings
......@@ -38,7 +39,7 @@ def prefix_strip(query):
the lookup type should be "startswith".
"""
query = str(query)
query = smart_unicode(query)
lookup_type = 'contains'
prefix = settings.BACKEND_PREFIX_ID
......
......@@ -68,13 +68,20 @@ def get_contact_id(inst):
return owner.uuid
def get_policies(inst):
def get_policies(inst, quota_dict = None):
policies = inst.projectresourcequota_set.all().prefetch_related('resource')
policy_list = []
for p in policies:
r = p.resource
if not is_resource_useful(r, p.project_capacity):
if not quota_dict:
quota_dict = get_project_quota(inst)
if not quota_dict:
usage = 0
else:
usage = quota_dict[r.name]['project_usage']
if not is_resource_useful(r, p.project_capacity, usage):
continue
policy_list.append(p)
......@@ -93,7 +100,7 @@ def get_project_usage(inst):
if not quota_dict:
return []
policies = get_policies(inst)
policies = get_policies(inst, quota_dict)
for p in policies:
r = p.resource
value = units.show(quota_dict[r.name]['project_usage'], r.unit)
......
......@@ -93,12 +93,22 @@ def get_quotas(user):
for resource_name, resource in resource_dict.iteritems():
# Chech if the resource is useful to display
project_limit = resource['project_limit']
usage = resource['usage']
r = get_resource(resource_name)
if not is_resource_useful(r, project_limit):
if not is_resource_useful(r, project_limit, usage):
continue
usage = units.show(resource['usage'], r.unit)
usage = units.show(usage, r.unit)
limit = units.show(resource['limit'], r.unit)
taken_by_others = resource['project_usage'] - resource['usage']
effective_limit = min(resource['limit'], project_limit - taken_by_others)
if effective_limit < 0:
effective_limit = 0
effective_limit = units.show(effective_limit, r.unit)
if limit != effective_limit:
limit += " (Effective Limit: " + effective_limit + ")"
q_res.append((r.report_desc, usage, limit,))
quotas.append(source)
......
......@@ -27,7 +27,7 @@
<dt>Address</dt><dd>{{ ip.address }}</dd>
<dt>Floating</dt><dd>{{ ip.floating_ip}}</dd>
<dt>Created</dt><dd>{{ ip.created }} ({{ ip.created|timesince }} ago)</dd>
<dt>Updated</dt><dd>{{ ip.updated }} ({{ ip.created|timesince }} ago)</dd>
<dt>Updated</dt><dd>{{ ip.updated }} ({{ ip.updated|timesince }} ago)</dd>
<dt>Floating</dt><dd>{{ ip.floating_ip}}</dd>
<dt>Deleted</dt><dd>{{ ip.deleted}}</dd>
......
......@@ -47,6 +47,9 @@
<dt>Project Status</dt><dd>{{ project|get_status_from_instance }}</dd>
{% if project.last_application %}
<dt>Application Status</dt><dd>{{ project.last_application|get_status_from_instance }}</dd>
{% if project.last_application.comments %}
<dt>Comments for Review</dt><dd>{{ project.last_application.comments }}</dd>
{% endif %}
{% endif %}
</dl>
</div>
......
......@@ -53,7 +53,10 @@
<dt>Deactivation reason</dt><dd>{{ user.deactivated_reason }}</dd>
{% endif %}
<dt>Last profile update at</dt><dd>{{ user.updated }}</dd>
<dt>Last logged-in at</dt><dd>{{ user.last_login }}</dd>
<dt>Last logged-in</dt>
{% for auth in user.auth_providers.all %}
<dd>{{ auth.module }} : {{ auth.last_login_at|default:"-"}}</dd>
{% endfor %}
</dl>
</div>
......
......@@ -156,7 +156,8 @@ def image_info(vm):
# Check if Cyclades DB has any info about this Image.
try:
image_info = Image.objects.get(uuid=vm.imageid)
image_info = Image.objects.get(uuid=vm.imageid,
version=vm.image_version)
except ObjectDoesNotExist:
# If all else fails, gather whatever info are available from the
# VirtualMachine instance.
......
......@@ -19,3 +19,4 @@ from synnefo_admin.admin.tests.views import *
from synnefo_admin.admin.tests.utils import *
from synnefo_admin.admin.tests.users import *
from synnefo_admin.admin.tests.projects import *
from synnefo_admin.admin.tests.vms import *
......@@ -24,6 +24,7 @@ from synnefo.db import models_factory as mf
from astakos.im import settings as astakos_settings
from snf_django.lib.api import faults
from snf_django.utils.testing import override_settings
from synnefo.util.units import PRACTICALLY_INFINITE
from synnefo_admin import admin_settings
from synnefo_admin.admin import views
......@@ -55,6 +56,12 @@ class MockRequest(object):
self.POST.update(content)
class MockResource(object):
def __init__(self, unit):
self.unit = unit
def reload_settings():
"""Reload admin settings after a Django setting has changed."""
reload(sys.modules['synnefo_admin.admin_settings'])
......@@ -169,6 +176,25 @@ class TestAdminUtilsUnit(unittest.TestCase):
reload_settings()
def test_is_resource_useful(self):
"""Test if is_resource_useful function works as expected."""
# Check if `is_resource_useful` produces the expected results,
# regardless of the resource's unit.
for unit in (None, "bytes"):
r = MockResource(unit=unit)
# Test if resource is useful, when its usage is zero
self.assertTrue(utils.is_resource_useful(r, 2048))
self.assertFalse(utils.is_resource_useful(r, 0))
self.assertFalse(utils.is_resource_useful(r, PRACTICALLY_INFINITE))
# Test if resource is useful, when its usage is not zero
self.assertTrue(utils.is_resource_useful(r, 2048, 1024))
self.assertTrue(utils.is_resource_useful(r, 0, 1024))
self.assertFalse(utils.is_resource_useful(r, PRACTICALLY_INFINITE,
1024))
class TestAdminUtilsIntegration(AdminTestCase):
......
# -*- coding: utf-8 -*-
# Copyright (C) 2015 GRNET S.A.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from .common import AdminTestCase
from django.conf import settings
from synnefo_admin.admin.queries_common import prefix_strip
class TestAdminFilterVMs(AdminTestCase):
def test_prefix_strip(self):
postfix = u'α'
stripped, lookup_type = prefix_strip(postfix)
self.assertEqual(stripped, None)
self.assertEqual(lookup_type, None)
postfix = "1234"
query = settings.BACKEND_PREFIX_ID + postfix
stripped, lookup_type = prefix_strip(query)
self.assertEqual(stripped, int(postfix))
self.assertEqual(lookup_type, "startswith")
stripped, lookup_type = prefix_strip(postfix)
self.assertEqual(stripped, int(postfix))
self.assertEqual(lookup_type, "contains")
......@@ -110,13 +110,14 @@ def get_resource(name):
cached_resources = {}
def is_resource_useful(resource, limit):
def is_resource_useful(resource, limit, usage = 0):
"""Simple function to check if the resource is useful to show.
Values that have infinite or zero limits are discarded.
Values that have infinite limits or zero limits and zero usage
are discarded.
"""
displayed_limit = units.show(limit, resource.unit)
if limit == 0 or displayed_limit == 'inf':
if (limit == 0 and usage == 0) or displayed_limit == 'inf':
return False
return True
......
## -*- coding: utf-8 -*-
##
## Boolean option to enable or disable admin
#ADMIN_ENABLED = True
......
# Copyright (C) 2010-2014 GRNET S.A.
# Copyright (C) 2010-2015 GRNET S.A.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
......@@ -600,6 +600,8 @@ def _get_memberships(query, request_user=None):
def join_project(data, request_user):
project_id = data.get("project")
if not isinstance(project_id, (basestring, int)):
raise faults.BadRequest("Invalid project ID: %s" % project_id)
with ExceptionHandler():
membership = functions.join_project(project_id, request_user)
response = {"id": membership.id}
......@@ -608,6 +610,8 @@ def join_project(data, request_user):
def enroll_user(data, request_user):
project_id = data.get("project")
if not isinstance(project_id, (basestring, int)):
raise faults.BadRequest("Invalid project ID: %s" % project_id)
email = data.get("user")
with ExceptionHandler():
m = functions.enroll_member_by_email(
......@@ -628,8 +632,7 @@ MEMBERSHIPS_ACTION = {
@transaction.commit_on_success
def post_memberships(request):
user = request.user
data = request.body
input_data = json.loads(data)
input_data = utils.get_json_body(request)
func, action_data = get_action(MEMBERSHIPS_ACTION, input_data)
return func(action_data, user)
......
......@@ -50,7 +50,7 @@ def quotas(request):
visible_resources = get_visible_resources()
resource_names = [r.name for r in visible_resources]
memberships = request.user.projectmembership_set.actually_accepted()
memberships = memberships.exclude(project__state__in=Project.SKIP_STATES)
memberships = memberships.exclude(project__state__in=Project.HIDDEN_STATES)
sources = [project_ref(m.project.uuid) for m in memberships]
result = get_user_quotas(request.user, resources=resource_names,
......
......@@ -16,6 +16,7 @@
import logging
from functools import wraps, partial
from datetime import datetime
from django.views.decorators.csrf import csrf_exempt
from django import http
......@@ -186,7 +187,7 @@ def users_create(request):
if not last_name:
raise faults.BadRequest("Invalid last_name")
has_signed_terms = not(get_latest_terms())
has_signed_terms = True
try:
user = make_local_user(email, first_name=first_name,
......@@ -204,9 +205,13 @@ def users_create(request):
ver_res = activation_backend.handle_verification(user, code)
if ver_res.is_error():
raise Exception(ver_res.message)
mod_res = activation_backend.handle_moderation(user, accept=True)
if mod_res.is_error():
raise Exception(ver_res.message)
# in case of auto moderation user moderation is handled within the
# verification process, no need to reapply moderation process
if not user.moderated:
mod_res = activation_backend.handle_moderation(user, accept=True)
if mod_res.is_error():
raise Exception(ver_res.message)
except Exception, e:
raise faults.BadRequest(e.message)
......@@ -275,6 +280,22 @@ def user_action(request, user_id):
return http.HttpResponse(data, status=200,
content_type='application/json')