Commit cb30577e authored by Stavros Sachtouris's avatar Stavros Sachtouris
Browse files

Merge branch 'develop' into feature-api-docs

parents ce7c631d ee6f0a62
......@@ -34,4 +34,5 @@ snf-quotaholder-app/quotaholder_django/version.py
*.iml
*.graffle
snf-stats-app/synnefo_stats/version.py
snf-astakos-client/astakosclient/version.py
astakosclient/astakosclient/version.py
snf-django-lib/snf_django/version.py
Copyright (C) 2010, 2011, 2012 GRNET S.A. All rights reserved.
Copyright (C) 2010, 2011, 2012, 2013 GRNET S.A. All rights reserved.
Redistribution and use in source and binary forms, with or
without modification, are permitted provided that the following
......@@ -16,7 +16,7 @@ conditions are met:
THIS SOFTWARE IS PROVIDED BY GRNET S.A. ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL GRNET S.A OR
PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL GRNET S.A. OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
......
.. _Changelog:
Unified Changelog file for Synnefo versions >= 0.13
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Since v0.13 most of the Synnefo components have been merged into a single
repository and have aligned versions.
.. _Changelog-0.13next:
v0.13next
=========
Released: UNRELEASED
Synnefo-wide
------------
* Create 'snf_django' Python package to hold common code for all Synnefo
components.
Astakos
-------
Cyclades
--------
Pithos
------
* Remove PITHOS_AUTHENTICATION_USERS setting, which was used to override
astakos users.
Tools
-----
.. _Changelog-0.13:
v0.13
=====
Released: Wed Apr 10 18:52:50 EEST 2013
In v0.13 the code was very heavily refactored for increased uniformity since
most of the Synnefo components have been merged into a single repository. Thus,
**just for this version** we will not document a complete Changelog (features,
fixes, improvements, issues, setting changes), but rather just copy from the
`NEWS` file with minor additions wherever needed.
Synnefo-wide
------------
* Support for pooling throughout Synnefo
* Pooled Django DB connections, Pithos backend connections, HTTP
connections using single `objpool` package
* Improved management commands
* Unified codebase for output of tables in JSON, CSV
* Bring most of Synnefo code inside a single, unified repository
* support automatic Python and Debian package builds for individual commits
* with automatic version generation
* Overhauling of Synnefo settings: renames and refactoring, for increased
uniformity (in progress)
* Deployment: Standardize on gunicorn, with gevent-based workers
and use of Green threads throughout Synnefo
* Documentation: New scale-out guide, with distinct node roles,
for mass Synnefo deployments
Astakos
-------
* Support multiple authentication methods
* Classic (username/password), Shibboleth, LDAP/Active Directory,
Google, Twitter, LinkedIn
* Users can enable/disable auth methods, and switch between them
* Introduce a UUID as a global identifier for users, throughout Synnefo
* The UUID remains constant as the user enables/disables login methods
* Allow users to modify their email address freely
* Per-user, per-resource accounting mechanism (quotaholder)
* Full quota support, with per-user, per-resource quotas, based on quotaholder
* Projects: Users can create and join Projects
* Projects grant extra resources to their members
* UI Enhancements for quotas and projects
* distinct Usage tab, showing usage of individual resources
* Project management UI
* New Overview page
Cyclades
--------
* Commission resources on quotaholder/Astakos
* Support mass creation of flavors
* Support for the ExtStorage disk template in Ganeti
* Query and report quotas in the UI
* Pass VM configuration parameters over a VM-side API (`vmapi`)
* Do not pass sensitive data as Ganeti OS parameters
* Keep sensitive data in memory caches (memcached) and
never allow them to hit the disk
* Display additional backend information in helpdesk machines list
* Allow helpdesk users to search for an account using a known machine id
* Helpdesk actions are now logged using the synnefo's common login
infrastructure
Pithos
------
* Support storage of blocks on a RADOS backend, for Archipelago
* new settings:
PITHOS_RADOS_STORAGE, PITHOS_RADOS_POOL_BLOCKS, PITHOS_RADOS_POOL_MAPS
* Rewritten support for public URLs, with admin-selectable length
* new settings:
PITHOS_PUBLIC_URL_SECURITY, PITHOS_PUBLIC_URL_ALPHABET
* Enable pithos backend to use external quotaholder component
* new settings:
PITHOS_USE_QUOTAHOLDER, PITHOS_QUOTAHOLDER_URL, PITHOS_QUOTAHOLDER_TOKEN,
PITHOS_QUOTAHOLDER_POOLSIZE
* Moderated version debiting mechanism
* new setting:
PITHOS_BACKEND_FREE_VERSIONING
* Proxy Astakos user-visible services
* new settings:
PITHOS_PROXY_USER_SERVICES, PITHOS_USER_CATALOG_URL,
PITHOS_USER_FEEDBACK_URL, PITHOS_USER_LOGIN_URL
Tools
-----
* Extend snf-burnin to include testing of Pithos functionality
.. _NEWS:
Unified NEWS file for Synnefo versions >= 0.13
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Since v0.13 all Synnefo components have been merged into a single repository.
.. _NEWS-0.13next:
v0.13next
=========
Released: UNRELEASED
Synnefo-wide
------------
Astakos
-------
Cyclades
--------
Pithos
------
Tools
-----
.. _NEWS-0.13:
v0.13
=====
Released: Wed Apr 10 18:52:50 EEST 2013
Synnefo-wide
------------
* Support for pooling throughout Synnefo
* Pooled Django DB connections, Pithos backend connections, HTTP
connections using single `objpool` package
* Improved management commands
* Unified codebase for output of tables in JSON, CSV
* Bring most of Synnefo code inside a single, unified repository
* support automatic Python and Debian package builds for individual commits
* with automatic version generation
* Overhauling of Synnefo settings: renames and refactoring, for increased
uniformity (in progress)
* Deployment: Standardize on gunicorn, with gevent-based workers
and use of Green threads throughout Synnefo
* Documentation: New scale-out guide, with distinct node roles,
for mass Synnefo deployments
Astakos
-------
* Support multiple authentication methods
* Classic (username/password), Shibboleth, LDAP/Active Directory,
Google, Twitter, LinkedIn
* Users can enable/disable auth methods, and switch between them
* Introduce a UUID as a global identifier for users, throughout Synnefo
* The UUID remains constant as the user enables/disables login methods
* Allow users to modify their email address freely
* Per-user, per-resource accounting mechanism (quotaholder)
* Full quota support, with per-user, per-resource quotas, based on quotaholder
* Projects: Users can create and join Projects
* Projects grant extra resources to their members
* UI Enhancements for quotas and projects
* distinct Usage tab, showing usage of individual resources
* Project management UI
* New Overview page
Cyclades
--------
* Commission resources on quotaholder/Astakos
* Support mass creation of flavors
* Support for the ExtStorage disk template in Ganeti
* Query and report quotas in the UI
* Pass VM configuration parameters over a VM-side API (`vmapi`)
* Do not pass sensitive data as Ganeti OS parameters
* Keep sensitive data in memory caches (memcached) and
never allow them to hit the disk
* Display additional backend information in helpdesk machines list
* Allow helpdesk users to search for an account using a known machine id
* Helpdesk actions are now logged using the synnefo's common login
infrastructure
Pithos
------
* Support storage of blocks on a RADOS backend, for Archipelago
* Rewritten support for public URLs, with admin-selectable length
Tools
-----
* Extend snf-burnin to include testing of Pithos functionality
......@@ -141,30 +141,28 @@ class AstakosClient():
kwargs['headers'].setdefault('content-length',
len(body) if body else 0)
# Get the connection object
conn = self.conn_class(self.netloc)
# Send request
try:
(data, status) = _do_request(conn, method, request_path, **kwargs)
# Get the connection object
with self.conn_class(self.netloc) as conn:
# Send request
(message, data, status) = \
_do_request(conn, method, request_path, **kwargs)
except Exception as err:
self.logger.error("Failed to send request: %s" % repr(err))
raise AstakosClientException(str(err))
finally:
conn.close()
# Return
self.logger.debug("Request returned with status %s" % status)
if status == 400:
raise BadRequest(data)
raise BadRequest(message, data)
elif status == 401:
raise Unauthorized(data)
raise Unauthorized(message, data)
elif status == 403:
raise Forbidden(data)
raise Forbidden(message, data)
elif status == 404:
raise NotFound(data)
raise NotFound(message, data)
elif status < 200 or status >= 300:
raise AstakosClientException(data, status)
raise AstakosClientException(message, data, status)
return simplejson.loads(unicode(data))
# ------------------------
......@@ -316,4 +314,5 @@ def _do_request(conn, method, url, **kwargs):
length = response.getheader('content-length', None)
data = response.read(length)
status = int(response.status)
return (data, status)
message = response.reason
return (message, data, status)
......@@ -33,36 +33,29 @@
class AstakosClientException(Exception):
def __init__(self, message, status=0):
def __init__(self, message='', details='', status=None):
self.message = message
self.status = status
def __str__(self):
return repr(self.message)
self.details = details
if not hasattr(self, 'status'):
self.status = status
super(AstakosClientException,
self).__init__(self.message, self.details, self.status)
class BadRequest(AstakosClientException):
def __init__(self, message):
"""400 Bad Request"""
super(BadRequest, self).__init__(message, 400)
status = 400
class Unauthorized(AstakosClientException):
def __init__(self, message):
"""401 Invalid X-Auth-Token"""
super(Unauthorized, self).__init__(message, 401)
status = 401
class Forbidden(AstakosClientException):
def __init__(self, message):
"""403 Forbidden"""
super(Forbidden, self).__init__(message, 403)
status = 403
class NotFound(AstakosClientException):
def __init__(self, message):
"""404 Not Found"""
super(NotFound, self).__init__(message, 404)
status = 404
class NoUserName(AstakosClientException):
......
......@@ -71,35 +71,39 @@ def _request_offline(conn, method, url, **kwargs):
def _request_status_302(conn, method, url, **kwargs):
"""This request returns 302"""
message = "FOUND"
status = 302
data = '<html>\r\n<head><title>302 Found</title></head>\r\n' \
'<body bgcolor="white">\r\n<center><h1>302 Found</h1></center>\r\n' \
'<hr><center>nginx/0.7.67</center>\r\n</body>\r\n</html>\r\n'
return (data, status)
return (message, data, status)
def _request_status_404(conn, method, url, **kwargs):
"""This request returns 404"""
message = "Not Found"
status = 404
data = '<html><head><title>404 Not Found</title></head>' \
'<body><h1>Not Found</h1><p>The requested URL /foo was ' \
'not found on this server.</p><hr><address>Apache Server ' \
'at example.com Port 80</address></body></html>'
return (data, status)
return (message, data, status)
def _request_status_401(conn, method, url, **kwargs):
"""This request returns 401"""
message = "UNAUTHORIZED"
status = 401
data = "Invalid X-Auth-Token\n"
return (data, status)
return (message, data, status)
def _request_status_400(conn, method, url, **kwargs):
"""This request returns 400"""
message = "BAD REQUEST"
status = 400
data = "Method not allowed.\n"
return (data, status)
return (message, data, status)
def _request_ok(conn, method, url, **kwargs):
......@@ -136,7 +140,7 @@ def _req_authenticate(conn, method, url, **kwargs):
if "usage=1" not in url:
# Strip `usage' key from `user'
del user['usage']
return (simplejson.dumps(user), 200)
return ("", simplejson.dumps(user), 200)
def _req_catalogs(conn, method, url, **kwargs):
......@@ -176,7 +180,7 @@ def _req_catalogs(conn, method, url, **kwargs):
return_catalog = {"displayname_catalog": catalogs, "uuid_catalog": {}}
else:
return_catalog = {"displayname_catalog": {}, "uuid_catalog": {}}
return (simplejson.dumps(return_catalog), 200)
return ("", simplejson.dumps(return_catalog), 200)
# ----------------------------
......
......@@ -31,9 +31,10 @@
# interpreted as representing official policies, either expressed
# or implied, of GRNET S.A.
import httplib
from httplib import HTTPConnection, HTTPSConnection
from contextlib import closing
import objpool.http
from objpool.http import PooledHTTPConnection
from astakosclient.errors import AstakosClientException
......@@ -59,18 +60,24 @@ def retry(func):
def scheme_to_class(scheme, use_pool, pool_size):
"""Return the appropriate conn class for given scheme"""
def _objpool(netloc):
return objpool.http.get_http_connection(
netloc=netloc, scheme=scheme, pool_size=pool_size)
return PooledHTTPConnection(
netloc=netloc, scheme=scheme, size=pool_size)
def _http_connection(netloc):
return closing(HTTPConnection(netloc))
def _https_connection(netloc):
return closing(HTTPSConnection(netloc))
if scheme == "http":
if use_pool:
return _objpool
else:
return httplib.HTTPConnection
return _http_connection
elif scheme == "https":
if use_pool:
return _objpool
else:
return httplib.HTTPSConnection
return _https_connection
else:
return None
......@@ -59,7 +59,7 @@ CLASSIFIERS = []
# Package requirements
INSTALL_REQUIRES = [
"objpool",
"objpool >= 0.2",
"simplejson"
]
......
PROJECTS="\
snf-common\
snf-webproject\
astakosclient\
snf-django-lib\
snf-astakos-app\
snf-astakos-client\
snf-pithos-backend\
snf-cyclades-gtools\
snf-cyclades-app\
......
......@@ -41,7 +41,8 @@ from stat import S_IFDIR, S_IFREG
from sys import argv
from time import time
from synnefo.lib.parsedate import parse_http_date
from snf_django.lib.api.parsedate import parse_http_date
from pithos.tools.lib.client import OOS_Client, Fault
from pithos.tools.lib.fuse import FUSE, FuseOSError, Operations
......
......@@ -96,7 +96,7 @@ class Command(object):
if email_re.match(self.user):
try:
from synnefo.lib.astakos import get_user_uuid
from snf_django.lib.astakos import get_user_uuid
from pithos.api.settings import SERVICE_TOKEN
self.user = get_user_uuid(SERVICE_TOKEN, self.user)
except ImportError:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment