Commit 4dbaf138 authored by Christos Stavrakakis's avatar Christos Stavrakakis
Browse files

Merge branch 'release-0.16'

parents d4fffe49 7ffc81f1

Too many changes to show.

To preserve performance only 1000 of 1000+ files are displayed.
......@@ -8,6 +8,9 @@ bin/
share/
build/
include/
!snf-cyclades-app/synnefo/ui/static/snf/extra/noVNC/include/
_sass/
.sass-cache/
*.pt.py
*.installed.cfg
*.sqlite
......@@ -20,7 +23,7 @@ selenium-server-standalone-2.0b2.jar
.settings/
settings.d/*-local.conf
*.egg-info
dist
#dist
_build
.coverage
......@@ -31,6 +34,7 @@ pithos/data
*/synnefo/versions/*.py
*/pithos/*/version.py
!*/synnefo/versions/__init__.py
snf-admin-app/synnefo_admin/version.py
snf-astakos-app/astakos/version.py
snf-tools/synnefo_tools/version.py
snf-quotaholder-app/quotaholder_django/version.py
......@@ -41,5 +45,8 @@ astakosclient/astakosclient/version.py
snf-django-lib/snf_django/version.py
snf-branding/synnefo_branding/version.py
snf-deploy/snfdeploy/version.py
# Temp dirs generated by snf-deploy keygen
snf-deploy/files/root/.ssh
snf-deploy/files/root/ddns
*.egg
*.tar.gz
This diff is collapsed.
......@@ -6,6 +6,257 @@ Unified Changelog file for Synnefo versions >= 0.13
Since v0.13 most of the Synnefo components have been merged into a single
repository and have aligned versions.
.. _Changelog-0.16:
v0.16
=====
Released: Tue Nov 11 10:44:57 EET 2014
Synnefo-wide
------------
* Replace accumulative projects with pool projects:
* Projects are now viewed as a source of finite resources. A member can
reserve a part of these resources up to a specified limit.
* Base quota are now offered through a special purpose user-specific system
project, identified with the same UUID as the user.
* Each actual resource (Cyclades VM, network, floating IP and Pithos
container) is now also associated with a project besides the owner.
* In resource creation, project defaults to the user-specific system
project, if not specified otherwise. It is also possible to change the
project assignment of an existing resource.
* All existing resources have been assigned to the respective
user-specific system projects.
* Logging mechanism for Synnefo management commands
* Log all stdout and stderr output of every invocation of snf-manage,
on unique filenames under a given directory.
* Add a new setting in the snf-common package, namely 'LOG_DIR', which
specifies the directory to be used by Synnefo components to write
their log files.
* Rename argument names of Synnefo management commands
* All Synnefo management commands now use 'user/flavor/image/network'
as argument names, instead of 'userid/user-id/owner/flavor-id/image
-id/network-id', for consistency across commands.
Astakos
-------
* Decouple projects from applications:
* Support project creation (by the system) and modification (by a
privileged user) without the need to submit/approve an application.
* View applications as modifications. When a project is uninitialized
(e.g. an application for a new project is pending), no further
modification is allowed.
* Applications are removed from the API. A project's last application is
only accessible as part of the project details.
* Decouple project state from application state; they can be combined by
an API client, if needed.
* Changes concerning quota and pool projects:
* A project must provide limits for all registered resources. On project
activation, resources missing are automatically completed using a
skeleton.
* Field `uplimit' of registed resources is exposed as `system_default' and
provide the skeleton for user-specific system projects. A new field
`project_default' is introduce to act as a skeleton for conventional
projects.
* The quotaholder now also records project quota besides user quota. The
two types of holders are distinguished with a prefix: `user:' and
`project:'.
* The quota API is extended to make project quota available.
* Removed setting `ASTAKOS_PROJECTS_VISIBLE'; we now always display
projects in Astakos menu.
* Projects can be set `private', making it accessible only to its owner and
members.
* Admin users API
* Extend astakos identity API to support user management functionality.
* Admin API related settings
* ``ADMIN_API_ENABLED``
Whether or not the admin api endpoints will be enabled.
* ``ADMIN_API_PERMITTED_GROUPS``
A list of group names for which users which belong to any of
them be granted full access to the admin API endpoints.
* Updated projects integration in UI
* Display both max per member/total in group quota in project views.
* Enhanced usage view to display individual resource usage details for
each of the projects the user is associated with.
* Display project usage statistics in project details view.
* Display system projects in admin project list view.
* Handle display of infinite resource quota values.
* Explicitly specify the lists in which every notification will be sent.
Cyclades
--------
* Introduce Volumes and Snapshots:
* Implement 'cyclades_volumes' service, containing the /volumes, /types,
and /snapshots API endpoints under '/volumes/v2.0'.
* Implement `snf-manage volume-{create, list, show, modify, inspect, remove}'
management commands for handling of volumes.
* Implement `snf-manage snapshot-{create, list, show, modify}` management
commands for handling of snapshots.
* Implement 'volume-type-{create, list, modify, show, remove}` management
commands for handling of volume types.
* Implement reconciliation for stale snapshots.
* Add new settings:
* GANETI_MAX_DISKS_PER_INSTANCE: Maximum number of disks that each Ganeti
instance can have.
* CYCLADES_VOLUME_MAX_SIZE: The maximum allowed size (in GB) for Cyclades
volumes.
* CYCLADES_SNAPSHOTS_ENABLED: Enable/Disable the snapshots feature
altogether at the API level.
* Integrate Cyclades resources with Astakos projects.
* Extend API calls that create resources with the 'project' attribute
in order to assign resources to the specified project.
* Implement API calls for reassigning resources to a new project.
* Export the project that a resource belongs in the 'tenant_id' API
attribute.
* Add support for snf-vncauthproxy 1.6 (configurable VNC console types).
* Change the ``CYCLADES_VNCAUTHPROXY_OPTS`` setting to a list of dictionaries
and support configurable vncauthproxy proxy addresses (added in
snf-vncauthproxy-1.6).
* Add support for more types of VM console:
* Modify the current 'console' action to support VNC WebSockets (requires
snf-vncauthproxy=1.6).
* Add support for the three OpenStack Compute console actions
(VNC, RDP, Spice). RDP and Spice are currently not implemented.
* Update Cyclades to work with Pithos with integrated Archipelago v2
mapfiles.
* Include functionality for checking the status of the Cyclades update path,
which includes Ganeti, AMQP, snf-ganeti-eventd and snf-dispatcher. This
functionality is implemented as part of the snf-dispatcher and can be used
by passing the '--status-check' option.
* Add setting `CYCLADES_VM_MAX_METADATA` to limit the maximum number of
metadata per vm.
* Add setting `CYCLADES_VOLUME_MAX_METADATA` to limit the maximum number of
metadata per volume.
* Use common -u/--user option to specify the UUID or email of the user, for
all management commands.
* Store basic information about images that have been used to create servers,
in order to preserve this information even if images are deleted from
Pithos.
* Make 'snf-ganeti-eventd' tolerate failures when processing Ganeti jobs. The
daemon will not crash but continue to run in order to process jobs that can
be processed.
* Update 'backend-list' command to not count the free IPs from networks that
are drained.
* Fix the'network-inspect' command to not contain externally reserved IPs
in th number of available IPs.
* Add `GANETI_DISKS_WAIT_FOR_SYNC` setting to decide whether Ganeti will
wait for the disk mirror to sync (`--no-wait-for-sync` Ganeti option).
* Fix mishandling of `MAX_CIDR_BLOCK` setting. Allowed CIDR sizes
changed from (MAX_CIDR_BLOCK, 29) to [MAX_CIDR_BLOCK, 29].
* Fix various minor bugs.
* Remove stale '--public' and '--user' options from 'image-show' and
'snapshot-show' management commands.
Cyclades UI
-----------
* In sync with the updated astakos projects API
* Include a project select widget within all resource create views, to let user
decide the project that the created resources will be assigned to.
* Display resource assigned project in resource list views.
* Let user reassign resource project.
* Volumes API integration.
* Introduce the Disks list/create views.
* Display machine attached disks in icon/single machine views.
* Use ``CYCLADES_VOLUME_MAX_SIZE`` setting to determine the
maximum allowed disk size.
* Integrate volume snapshots
* Include available snapshots in disk/vm create wizards.
* Disk snaphot create actions.
* The ui snapshoting functionality (snapshot actions and listing)
can be enabled/disabled using the introduced ``UI_SNAPSHOTS_ENABLED``
setting.
* Replace TigerVNC Java client with an HTML5 Websocket-based client (noVNC)
* Other UI fixes
* Disabled suspended vm actions
* Custom error message for 413 api error response codes
* Enable resize actions from both info/actions subviews for active
machines
* Update images collection each time machine create wizard opens
* Fixed a couple of title truncate in several views
* Handle display of infinite resource quota values.
* Improve network create wizard. Support for custom gateway.
* Common font styling for all resource titles
Pithos
------
* Backend modifications to enable/support snapshot creation.
* Backend and API modifications to reflect the modifications in the resource
allocation mechanism via the accumulative projects with pool projects.
* Change default disposition type: If no disposition-type is
specifically requested or an invalid value is passed, the disposition type
is set to 'inline'.
Admin
-----
* Introduce Administrator Dashboard, which provides the following:
* Graphic access to the details of various Synnefo entities (users, VMs,
Projects).
* Intuitive filtering.
* Multiple actions and notifications.
* Charts and statistics generation.
* Define the Admin node URL and path with the ``ADMIN_BASE_URL`` setting
* The ``ADMIN_BASE_URL`` setting should be adjusted in every node that Admin
is installed.
Tools
-----
* Extend snf-burnin to include testing of snapshots.
.. _Changelog-0.15.2:
v0.15.2
......@@ -29,6 +280,7 @@ Cyclades UI
.. _Changelog-0.15.1:
v0.15.1
=======
......@@ -49,7 +301,6 @@ Astakos
* Fix Authenticate Identity API call with trailing slash, which used to fail
with 405 (Method not allowed)
Cyclades
--------
......
......@@ -5,6 +5,88 @@ Unified NEWS file for Synnefo versions >= 0.13
Since v0.13 all Synnefo components have been merged into a single repository.
.. _NEWS-0.16:
v0.16
=====
Released: Tue Nov 11 10:44:57 EET 2014
The Synnefo 0.16 release brings major new features and significant bug fixes
across Synnefo. All users are strongly encouraged to upgrade. Please see
the Synnefo documentation for upgrade instructions to v0.16.
The most notables changes are:
Synnefo-wide
-------------
* Administrator Dashboard: Synnefo 0.16 features a new, integrated Web-based UI
for managing Synnefo users and resources. It allows the administrator to
view, filter, modify, and produce stats for all Synnefo resources. This new
interface will eventually replace the old read-only helpdesk interface, which
has been obsoleted and will be removed in Synnefo 0.17.
* Pool projects: Projects have been upgraded to a new unified `pool project`
type in Synnefo 0.16. Resources get assigned to projects, and project members
may reserve resources from individual projects for their own use. Users may
choose the project where a new resource be charged (e.g., a new Cyclades VM or
a new IP address), and re-assign them freely. Pre-0.16 projects are a subset
of pool projects, with per-user constraints on resource consumption.
* Management command logging: All Synnefo management commands and their outputs
are individually logged, which may prove very useful for debugging and
auditing purposes.
* Archipelago has been adopted as the single backend for Pithos. It is a
unified access layer supporting NFS or Ceph/RADOS-based storage. Current
Pithos installations over NFS can be migrated seamlessly to Pithos over
Archipelago over NFS, please see the Synnefo upgrade notes (FIXME)
and the Archipelago Administrator's Guide for more details.
Cyclades VMs continue to run over all Ganeti-supported disk templates,
including DRBD, LVM, Ceph/RBD, and Archipelago.
* Various bug fixes and performance improvements. Please see the Synnefo
`Changelog` for a complete list.
Cyclades
--------
* Storage: Major improvements in VM storage handling.
This release introduces a complete implementation of the OpenStack Block Storage
(Cinder) API v2.0, with distinct /volumes, /snapshots/, /types API endpoints.
* Storage: You can now add and remove virtual disks to and from VMs, even when
they are powered on, with hot-(un)plugging.
* Storage: You can now snapshot running VMs to files on Pithos, provided
their storage is backed by Archipelago. New snapshots appear in Pithos and
are manage-able as Pithos files.
* Storage: You can now spawn new VMs from pre-existing snapshots, regardless
of their disk template, e.g., you can use an existing snapshot to spawn a
DRBD VM. If the new VM is to be based on Archipelago, its creation is a thin
clone and completes in seconds.
PLEASE NOTE: Snapshot support is a work-in-progress, and Archipelago does not
yet perform garbage collection on snapshot deletion. This will be fixed in
Synnefo 0.17.
* Console: Support noVNC-based console for an HTML5-based UI to server consoles
over VNC, with HTTPS encryption. This removes the need for a working Java Runtime
Environment in the client's browser.
* Console: Support multiple console types, including VNC over raw TCP sockets
and VNC over Websockets, with optional encryption.
* Projects: Supports arbitrary (re-)assignment of Cyclades resources to
individual projects.
* Admin: Support end-to-end checking of the Ganeti-to-Synnefo update path,
including all intermediate daemons and the Message Queue.
* Numerous bug fixes, performance improvements and improved usability in
management commands.
Pithos
------
* Pithos has been refactored to use Archipelago as its single unified storage
backend. Individual storage types (e.g., NFS, or RADOS) are handled by
Archipelago itself.
* Snapshots: Snapshots created from Cyclades VMs on Archipelago are now
presented as files on Pithos and may be downloaded via the Pithos UI.
.. _NEWS-0.15.2:
v0.15.2
......@@ -145,7 +227,7 @@ Synnefo-wide
.. _NEWS-0.14.10:
v0.14.10
=======
========
Released: Tue Nov 26 11:03:37 EET 2013
......@@ -247,7 +329,7 @@ Synnefo-wide
* Branding customization support across synnefo frontend components:
* ability to adapt the Astakos, Pithos and Cyclades Web UI to a companys
* ability to adapt the Astakos, Pithos and Cyclades Web UI to a company's
visual identity. This is possible using the snf-branding component, which
is automatically installed on the nodes running the API servers for
Astakos, Pithos and Cyclades.
......
......@@ -47,36 +47,18 @@ for more information on the Synnefo users and developers lists.
Copyright and license
=====================
Copyright (C) 2010-2014 GRNET S.A. All rights reserved.
Redistribution and use in source and binary forms, with or
without modification, are permitted provided that the following
conditions are met:
1. Redistributions of source code must retain the above
copyright notice, this list of conditions and the following
disclaimer.
2. Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following
disclaimer in the documentation and/or other materials
provided with the distribution.
THIS SOFTWARE IS PROVIDED BY GRNET S.A. ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL GRNET S.A. OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
The views and conclusions contained in the software and
documentation are those of the authors and should not be
interpreted as representing official policies, either expressed
or implied, of GRNET S.A.
Copyright (C) 2010-2014 GRNET S.A.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
include distribute_setup.py
include distribute_setup.py README.md
astakosclient
=============
Overview
--------
This is Synnefo's astakosclient component. Please see the [official Synnefo
site](http://www.synnefo.org) for more information.
Copyright and license
=====================
Copyright (C) 2010-2014 GRNET S.A.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
This diff is collapsed.
# Copyright (C) 2012, 2013 GRNET S.A. All rights reserved.
# Copyright (C) 2010-2014 GRNET S.A.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
# conditions are met:
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# 1. Redistributions of source code must retain the above
# copyright notice, this list of conditions and the following
# disclaimer.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials
# provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY GRNET S.A. ``AS IS'' AND ANY EXPRESS
# OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL GRNET S.A OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
# AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
# The views and conclusions contained in the software and
# documentation are those of the authors and should not be
# interpreted as representing official policies, either expressed
# or implied, of GRNET S.A.
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""
Astakos Client Exceptions
......@@ -38,15 +20,25 @@ Astakos Client Exceptions
class AstakosClientException(Exception):
"""Base AstakosClientException Class"""
def __init__(self, message='', details='', status=500):
def __init__(self, message='', details='', status=500,
response=None, errobject=None):
self.message = message
self.details = details
self.errobject = errobject
self.response = response
if not hasattr(self, 'status'):
self.status = status
super(AstakosClientException,
self).__init__(self.message, self.details, self.status)
class ConnectionError(AstakosClientException):
"""Connection failed"""
def __init__(self, errobject):
super(ConnectionError, self).__init__(
message=str(errobject), errobject=errobject)
class BadValue(AstakosClientException):
"""Re-define ValueError Exception under AstakosClientException"""
def __init__(self, details):
......@@ -56,8 +48,6 @@ class BadValue(AstakosClientException):
class InvalidResponse(AstakosClientException):
"""Return simplejson parse Exception as AstakosClient one"""
def __init__(self, message, details):
super(InvalidResponse, self).__init__(message, details)
class BadRequest(AstakosClientException):
......
#!/usr/bin/env python
#
# Copyright (C) 2012, 2013 GRNET S.A. All rights reserved.
# Copyright (C) 2010-2014 GRNET S.A.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
# conditions are met:
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# 1. Redistributions of source code must retain the above
# copyright notice, this list of conditions and the following
# disclaimer.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# 2. Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials
# provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY GRNET S.A. ``AS IS'' AND ANY EXPRESS
# OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL GRNET S.A OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
# AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
# The views and conclusions contained in the software and
# documentation are those of the authors and should not be
# interpreted as representing official policies, either expressed
# or implied, of GRNET S.A.
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""Unit Tests for the astakos-client module
......@@ -42,7 +24,11 @@ the astakos client library
import re
import sys
import simplejson
try:
import simplejson as json
except ImportError:
import json
import astakosclient
from astakosclient import AstakosClient
......@@ -289,10 +275,10 @@ def _req_tokens(conn, method, url, **kwargs):
if 'body' in kwargs:
# Return endpoints with authenticate info
return ("", simplejson.dumps(endpoints_with_info), 200)
return ("", json.dumps(endpoints_with_info), 200)
else:
# Return endpoints without authenticate info