.. _Changelog: Unified Changelog file for Synnefo versions >= 0.13 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Since v0.13 most of the Synnefo components have been merged into a single repository and have aligned versions. .. _Changelog-0.16: v0.16 ===== Released: Tue Nov 11 10:44:57 EET 2014 Synnefo-wide ------------ * Replace accumulative projects with pool projects: * Projects are now viewed as a source of finite resources. A member can reserve a part of these resources up to a specified limit. * Base quota are now offered through a special purpose user-specific system project, identified with the same UUID as the user. * Each actual resource (Cyclades VM, network, floating IP and Pithos container) is now also associated with a project besides the owner. * In resource creation, project defaults to the user-specific system project, if not specified otherwise. It is also possible to change the project assignment of an existing resource. * All existing resources have been assigned to the respective user-specific system projects. * Logging mechanism for Synnefo management commands * Log all stdout and stderr output of every invocation of snf-manage, on unique filenames under a given directory. * Add a new setting in the snf-common package, namely 'LOG_DIR', which specifies the directory to be used by Synnefo components to write their log files. * Rename argument names of Synnefo management commands * All Synnefo management commands now use 'user/flavor/image/network' as argument names, instead of 'userid/user-id/owner/flavor-id/image -id/network-id', for consistency across commands. Astakos ------- * Decouple projects from applications: * Support project creation (by the system) and modification (by a privileged user) without the need to submit/approve an application. * View applications as modifications. When a project is uninitialized (e.g. an application for a new project is pending), no further modification is allowed. * Applications are removed from the API. A project's last application is only accessible as part of the project details. * Decouple project state from application state; they can be combined by an API client, if needed. * Changes concerning quota and pool projects: * A project must provide limits for all registered resources. On project activation, resources missing are automatically completed using a skeleton. * Field `uplimit' of registed resources is exposed as `system_default' and provide the skeleton for user-specific system projects. A new field `project_default' is introduce to act as a skeleton for conventional projects. * The quotaholder now also records project quota besides user quota. The two types of holders are distinguished with a prefix: `user:' and `project:'. * The quota API is extended to make project quota available. * Removed setting `ASTAKOS_PROJECTS_VISIBLE'; we now always display projects in Astakos menu. * Projects can be set `private', making it accessible only to its owner and members. * Admin users API * Extend astakos identity API to support user management functionality. * Admin API related settings * ``ADMIN_API_ENABLED`` Whether or not the admin api endpoints will be enabled. * ``ADMIN_API_PERMITTED_GROUPS`` A list of group names for which users which belong to any of them be granted full access to the admin API endpoints. * Updated projects integration in UI * Display both max per member/total in group quota in project views. * Enhanced usage view to display individual resource usage details for each of the projects the user is associated with. * Display project usage statistics in project details view. * Display system projects in admin project list view. * Handle display of infinite resource quota values. * Explicitly specify the lists in which every notification will be sent. Cyclades -------- * Introduce Volumes and Snapshots: * Implement 'cyclades_volumes' service, containing the /volumes, /types, and /snapshots API endpoints under '/volumes/v2.0'. * Implement `snf-manage volume-{create, list, show, modify, inspect, remove}' management commands for handling of volumes. * Implement `snf-manage snapshot-{create, list, show, modify}` management commands for handling of snapshots. * Implement 'volume-type-{create, list, modify, show, remove}` management commands for handling of volume types. * Implement reconciliation for stale snapshots. * Add new settings: * GANETI_MAX_DISKS_PER_INSTANCE: Maximum number of disks that each Ganeti instance can have. * CYCLADES_VOLUME_MAX_SIZE: The maximum allowed size (in GB) for Cyclades volumes. * CYCLADES_SNAPSHOTS_ENABLED: Enable/Disable the snapshots feature altogether at the API level. * Integrate Cyclades resources with Astakos projects. * Extend API calls that create resources with the 'project' attribute in order to assign resources to the specified project. * Implement API calls for reassigning resources to a new project. * Export the project that a resource belongs in the 'tenant_id' API attribute. * Add support for snf-vncauthproxy 1.6 (configurable VNC console types). * Change the ``CYCLADES_VNCAUTHPROXY_OPTS`` setting to a list of dictionaries and support configurable vncauthproxy proxy addresses (added in snf-vncauthproxy-1.6). * Add support for more types of VM console: * Modify the current 'console' action to support VNC WebSockets (requires snf-vncauthproxy=1.6). * Add support for the three OpenStack Compute console actions (VNC, RDP, Spice). RDP and Spice are currently not implemented. * Update Cyclades to work with Pithos with integrated Archipelago v2 mapfiles. * Include functionality for checking the status of the Cyclades update path, which includes Ganeti, AMQP, snf-ganeti-eventd and snf-dispatcher. This functionality is implemented as part of the snf-dispatcher and can be used by passing the '--status-check' option. * Add setting `CYCLADES_VM_MAX_METADATA` to limit the maximum number of metadata per vm. * Add setting `CYCLADES_VOLUME_MAX_METADATA` to limit the maximum number of metadata per volume. * Use common -u/--user option to specify the UUID or email of the user, for all management commands. * Store basic information about images that have been used to create servers, in order to preserve this information even if images are deleted from Pithos. * Make 'snf-ganeti-eventd' tolerate failures when processing Ganeti jobs. The daemon will not crash but continue to run in order to process jobs that can be processed. * Update 'backend-list' command to not count the free IPs from networks that are drained. * Fix the'network-inspect' command to not contain externally reserved IPs in th number of available IPs. * Add `GANETI_DISKS_WAIT_FOR_SYNC` setting to decide whether Ganeti will wait for the disk mirror to sync (`--no-wait-for-sync` Ganeti option). * Fix mishandling of `MAX_CIDR_BLOCK` setting. Allowed CIDR sizes changed from (MAX_CIDR_BLOCK, 29) to [MAX_CIDR_BLOCK, 29]. * Fix various minor bugs. * Remove stale '--public' and '--user' options from 'image-show' and 'snapshot-show' management commands. Cyclades UI ----------- * In sync with the updated astakos projects API * Include a project select widget within all resource create views, to let user decide the project that the created resources will be assigned to. * Display resource assigned project in resource list views. * Let user reassign resource project. * Volumes API integration. * Introduce the Disks list/create views. * Display machine attached disks in icon/single machine views. * Use ``CYCLADES_VOLUME_MAX_SIZE`` setting to determine the maximum allowed disk size. * Integrate volume snapshots * Include available snapshots in disk/vm create wizards. * Disk snaphot create actions. * The ui snapshoting functionality (snapshot actions and listing) can be enabled/disabled using the introduced ``UI_SNAPSHOTS_ENABLED`` setting. * Replace TigerVNC Java client with an HTML5 Websocket-based client (noVNC) * Other UI fixes * Disabled suspended vm actions * Custom error message for 413 api error response codes * Enable resize actions from both info/actions subviews for active machines * Update images collection each time machine create wizard opens * Fixed a couple of title truncate in several views * Handle display of infinite resource quota values. * Improve network create wizard. Support for custom gateway. * Common font styling for all resource titles Pithos ------ * Backend modifications to enable/support snapshot creation. * Backend and API modifications to reflect the modifications in the resource allocation mechanism via the accumulative projects with pool projects. * Change default disposition type: If no disposition-type is specifically requested or an invalid value is passed, the disposition type is set to 'inline'. Admin ----- * Introduce Administrator Dashboard, which provides the following: * Graphic access to the details of various Synnefo entities (users, VMs, Projects). * Intuitive filtering. * Multiple actions and notifications. * Charts and statistics generation. * Define the Admin node URL and path with the ``ADMIN_BASE_URL`` setting * The ``ADMIN_BASE_URL`` setting should be adjusted in every node that Admin is installed. Tools ----- * Extend snf-burnin to include testing of snapshots. .. _Changelog-0.15.2: v0.15.2 ======= Released: Tue May 27 13:50:31 EEST 2014 Cyclades -------- * Fix minor problem in network stats plugin of collectd * Fix small issue in the 'stats-cyclades' command arising when Ganeti cannot communicate with a node. * Check max size of server metadata * Add support for Ganeti 2.10 Cyclades UI ----------- * Fix oraclelinux icon name .. _Changelog-0.15.1: v0.15.1 ======= Released: Thu May 15 13:31:37 EEST 2014 Synnefo-wide ------------ * Documentation: New CentOS installation guide. * Handle Unicode issue in proxy requests. Astakos ------- * Change statistics API to expose statistics for all users and for users per authentication provider (users that are exclusively using a provider). * Fix minor problem in weblogin, which could lead to unexpected exception * Fix Authenticate Identity API call with trailing slash, which used to fail with 405 (Method not allowed) Cyclades -------- * Fix bug in snf-dispatcher that could result in unsynced NICs between Cyclades and Ganeti. * Support queries by IPv6 address in helpdesk app. * Extend statistics API with detailed information about Ganeti clusters, virtual servers and networks, public IPv4 pools and images. * Allow creation of ports with existing floating IPs from networks that are marked as drained. Cyclades UI ----------- * Include icons for Oracle Linux OS (oraclelinux) * Include a null gateway IP when creating subnets for private networks to override the default gateway IP of Neutron API. Pithos ------ * Support selectable storage backend * new settings: - PITHOS_BACKEND_STORAGE: Possible values are 'nfs' and 'rados'. - PITHOS_RADOS_CEPH_CONF: The Ceph configuration file. * Remove unused variable PITHOS_RADOS_STORAGE * Add 'pithos-sync-rados.sh' shell script in Pithos tools, a Pithos object synchronization tool from NFS to Rados. * Fix missing Content-Disposition header in object response, which caused publicly shared objects to be saved with a meaningless name (without extension) .. _Changelog-0.15: v0.15 ===== Released: Mon Mar 10 14:01:32 EET 2014 Synnefo-wide ------------ * Integrate Pithos tests in continuous integration. * Change astakosclient to accept AUTH_URL instead of BASE_URL ASTAKOS_BASE_URL settings has been removed from Pithos and Cyclades and has been replaced with ASTAKOS_AUTH_URL. Both Pithos and Cyclades proxy the Astakos services under ASTAKOS_PROXY_PREFIX path. ASTAKOS_PROXY_PREFIX by default has a value of '_astakos'. More specifically, Astakos' identity service is proxied under '_astakos/identity', Astakos' account service is under '_astakos/account' and Astakos' ui service is under '_astakos/ui'. * Add 'mail_admins' handler to 'django.request' logger in order to send email notifications to users listed in 'ADMINS' setting about unhandled exceptions in the code. * Extend astakosclient to request and validate OAuth 2.0 access tokens * Change response status code from 400 (Bad Request) to 405 (Not allowed method) in case of an unexpected request method. Astakos ------- * Changes in project schema: * A Project entry is created when submitting an application for a new project, rather than on approval. Its state is dependent on the state of its `reference' application (current definition). Lock Project rather than Chain (the latter is semantically obsolete). * Project states "Active - Pending" and "Suspended - Pending" have been removed. In management command `project-list', the existence of a pending modification is indicated by a non-blank `Pending AppID'. * Improve recording of project, application, and membership actions. * Implement API calls for projects. * Store the base URL of a component. Deployer should provide it when adding a new component. Service endpoints originating from a component are expected to match its base URL; otherwise, a warning is issued. Re-registration with `snf-component-register' affects both the base and the ui URL. * Changes in resource and quota handling: * New resources are registered with unlimited default base quota, represented by 2**63-1. * Each newly accepted user copies the default value for all resources as their own base quota. A base quota is considered 'custom' if its value differs from the default. * Changing resource's default quota affects the base quota *only* of future users. * Resource definition got flags 'api_visible' and 'ui_visible', replacing flag 'allow_in_projects'. They control whether a user can access these resources. The system internally always accounts for all resources, and a user can get off quota even for a resource that is not visible. * Remove API call GET /account/v1.0/authenticate in favor of POST /identity/v2.0/tokens. * Export basic statistics about Astakos service at the '/admin/stats/detail' API endpoint. Access to this endpoint is only allowed to users that belong to the Astakos groups that are defined in the 'ASTAKOS_ADMIN_STATS_PERMITTED_GROUPS' setting. Statistics are also available through the 'snf-manage stats-astakos' management command. * Implement OAuth 2.0 Authorization Code Grant Add API calls for authorization code and access token generation * Add API call for validating OAuth 2.0 access tokens * **Shibboleth module** Extract unique identifier from the ``REMOTE_USER`` header. * Automatically fill third-party signup form fields when available by the the third-party provider. * Management commands: * Introduced new commands: * component-show * quota-list (replacing quota, supports various filters) * quota-verify (replacing quota) * oauth2-client-add (register OAuth 2.0 client) * oauth2-client-list (list registered oauth 2.0 clients) * oauth2-client-remove (remove OAuth 2.0 client) * Changed commands: * component-add got options --base-url and --ui-url * resource-modify --limit became --default-quota * user-modify can operate on multiple users with --all and --exclude * user-modify --set-base-quota became --base-quota * Removed commands: * quota * resource-import (subsumed by service-import) * resource-export-astakos (subsumed by service-export-astakos) * Fix request authorization code failures due to Unicode issue: https://code.grnet.gr/issues/4971 * Omit validation issues for non-required metadata values received from the third-party authentication provider. Cyclades -------- * Major changes to Cyclades networks: * Implement 'cyclades_network' service, containing the /networks, /ports, /subnets and /floatingips API endpoints under '/network/v2.0'. The old /networks API of 'cyclades_compute' (under /compute/v2.0) is removed. * Implement `snf-manage subnet-{create, list, modify, inspect}' management commands for handling of subnets. * Implement `snf-manage port-{create, list, remove, inspect}' management commands for handling of ports. * Add two new settings, 'CYCLADES_FORCED_SERVER_NETWORKS' and 'CYCLADES_DEFAULT_SERVER_NETWORKS' to control the networks that newly created servers will be connected. * Implement Floating IP addresses, which are IPv4 addresses that can be dynamically added and removed to a running server. * Add new 'cyclades.floating_ip' resource. * Implement 'snf-manage floating-ip-{create,list,remove,attach,detach}' management commands to handle floating IPs. * Add 'floating_ip_pool' attribute to networks to mark networks that can be used as floating IP pools. * Implement 'resize' server action. * Implement the 'resize' server action, to change the flavor of a server. Only 'cpu' and 'memory' resizing is supported. * Compute quotas for CPU and memory of running VMs. * Change 'cyclades.cpu' and 'cyclades.ram' resources to represent the CPU and RAM for running VMs. Total CPU and RAM usage is represented by new 'cyclades.total_cpu' and 'cyclades.total_ram' resources. * Refer to Ganeti NICs by their name instead of their index. * Make cyclades give a unique name to each Ganeti NIC. NICs are refered by their unique name and not by their index inside the VM that are connected to. * Support firewall profile for all NICs of an instance. Change firewall settings to be filled with the unique name of the NIC. The affected settings are the GANETI_FIREWALL_{ENABLED, DISABLED, PROTECTED}_TAG settings. * Add accounting for public IP addresses that is accessible via `snf-manage ip-list` management command and via the helpdesk app. * Implement IPv6 only networks. * Extend servers info API response with 'SNF:fqdn' attribute, and introduce CYCLADES_SERVERS_FQDN to set the template for servers FDQN. Remove 'UI_VM_HOSTNAME_FORMAT' setting. * Extend servers info API response with 'SNF:port_forwarding' attribute, describing port forwarding rules (DNAT) that are applied to vms. The description of such rules is done via the new CYCLADES_PORT_FORWARDING setting. * Speed up server reconciliation, by performing parallel reconciliation for each backend. * Change --dhcp option of network management commands from a flag to a boolean value, e.g. --dhcp=True * Remove 'ARCHIPELAGO_BACKENDS' setting used to distinguish between backends that hosted only archipelago backends. Instead allocation is based on which disk-templates are enabled in each backend. * Implement 'snf-manage server-remove' management command. * Move reconciliation of IP pools from 'reconcile-networks' to 'reconcile-pools'. The IP pool reconciliation does not reconcile the IP pools with Ganeti. Instead it checks if the pool is consistent with the IPs that are used by instances. * Do not automatically release externally reserved IPs if they are released from a Ganeti backend. Management of externally reserved IPs must be performed from Cyclades with 'network-modify' command. * Export basic statistics about Cyclades Service at the '/admin/stats/detail' API endpoint. Access to this endpoint is only allowed to users that belong to the Astakos groups that are defined in the 'ADMIN_STATS_PERMITTED_GROUPS' setting. Statistics are also available through the 'snf-manage stats-cyclades' management command. * Support enforcing quota through command 'enforce-resources-cyclades'. * Remove command 'resource-export-cyclades' subsumed by 'service-export-cyclades'. * Obsolete PUBLIC_USE_POOL setting, since Cyclades manages IP pool for all type of networks. * Encrypt / decrypt the instance id / hostname in the stats URL in snf-cyclades-app and snf-stats-app, using the 'CYCLADES_STATS_SECRET_KEY' and 'STATS_SECRET_KEY' respectively. * Add support for snf-vncauthproxy-1.5 and the setting 'CYCLADES_VNCAUTHPROXY_OPTS', which configures the extra options / arguments needed by the newer version of snf-vncauthproxy. Support for older versions of snf-vncauthproxy has been dropped. See also the upgrade notes for Synnefo and snf-vncauthproxy-1.5. * Remove 'DEFAULT_ROUTING_TABLE' setting. If a link for an IP_LESS_ROUTED network is not specified, the link will be uniquely named 'snf-link-$network_id'. * Extend flavors with 'allow_create' attribute. Flavors that have this attribute unset cannot be used by users to create new servers. * Store each image property(metadata) as a separate Pithos metadata and check that the size of each property is valid. * Fix a few occurrences of HTTP 500 errors being triggered due to insufficient validation of incoming requests in the Network API * Speed up some API calls by fixing the relevant DB queries Cyclades UI ----------- - Retrieve all networks information from the introduced cyclades network service. - New IPs pane from which user can manage floating IPs. - Redesign public keys overlay as an additional pane view. - Split networking configuration into an additional step in machine create wizard. - Display forced networks and choices of the available floating IPs which will be assigned to the created machine. - Support for machine resize action. Explicit handling when machine is started by displaying an utility shutdown button within the resize overlay. - Machine IPs toggling subview in icon/single views. - Replace IPv4/IPv6 with machine's FQDN in icon/single view. When no FQDN can be resolved display a message. Message can be configured using the introduced ``UI_NO_FQDN_MESSAGE``. Setting ``UI_VM_HOSTNAME_FORMAT`` has been removed and no longer used. - Respect ``SNF:task_state`` machine attribute in order to improve machine status display. - Append software version as a url parameter in HTML static files in order to force browser cache invalidation between versions. - Configurable Google fonts base url. Fonts base url can be changed usint the ``SYNNEFO_FONTS_BASE_URL`` setting. - Regression fix: Display reboot required notification on machine firewall parameters. - Handling of ``GANETI_USE_HOTPLUG`` setting. Do not allow live network actions when setting is set to ``False``. - Double escaping fix in machine create wizard images list and machine details subview. - Fix image ordering in machine create wizard. - New setting ``UI_SSH_SUPPORT_OSFAMILY_EXCLUDE_LIST``. A list of image OS families for which ui will disable ssh key injection in machine wizard. - Setting ``UI_SUPPORT_SSH_OS_LIST`` removed and no longer used. - Group public networks by name if setting ``UI_GROUP_PUBLIC_NETWORKS`` is set to True. - Setting ``UI_GROUPED_PUBLIC_NETWORK_NAME`` has been deprecated and no longer used. - Fix UI to filter available flavors in VM wizard Cyclades Userdata ----------------- - Maximum allowed length of ssh key content. Configurable from the ``USERDATA_SSH_KEY_MAX_CONTENT_SIZE`` setting. Pithos ------ * Rewrite tests. * Performance optimizations in object listing. * Introduce backend method decorator for handling transaction management if no transaction is initiated from the frontend. * Fix Internal Server Errors https://code.grnet.gr/issues/4501 & https://code.grnet.gr/issues/4502. * Fix REQUEST ENTITY TOO LARGE request failure during move operations https://code.grnet.gr/issues/4154. * Fix FORBIDDEN request failure while listing implicitly shared objects https://code.grnet.gr/issues/4131. * Fix issue with the computed size of an updated object. * Reply with the Merkle hash in the ETag header if MD5 is deactivated. * Reply with FORBIDDEN (403) to public listing requests performed by non path owners. * Change response status to NOT FOUND (404) while trying to delete an already deleted object. * Change SQLAlchemy version to 0.7 * Change view authorization The pithos views do not use the cookie information for user authentication. They request (from Astakos) and use a short-term access token for a specific resource. * Remove PITHOS_ASTAKOS_COOKIE_NAME setting, since it is no longer useful * Add PITHOS_OAUTH2_CLIENT_CREDENTIALS setting to authenticate the views with astakos during the resource access token generation procedure * Add PITHOS_UNSAFE_DOMAIN setting to restrict file serving endpoints to a specific host * Added new 'file-show' management command * Remove command 'resource-export-pithos' subsumed by 'service-export-pithos'. * Extend API to optionally enforce a specific content disposition type in view and public requests: https://code.grnet.gr/issues/5019 * Fix bulk operation (delete/copy/move) failures in overpopulated containers/folders: https://code.grnet.gr/issues/5119 * Fix performance issue due to missing index * Fix backend open hashfile bug: https://code.grnet.gr/issues/5011 * Fix GET request for public path with If-None-Match https://code.grnet.gr/issues/5168 .. _Changelog-0.14.10: v0.14.10 ======= Released: Tue Nov 26 11:03:37 EET 2013 Cyclades ------- * This is the first release to support Ganeti 2.8. Support for older versions of Ganeti is dropped. * Use Ganeti opportunistic locking to achive parallelized instance creations in the same backend. Add setting 'GANETI_USE_OPPORTUNISTIC_LOCKING' to enable the use of this feature. * Fix warning message while getting object permissions to appear only when path is None and the object has permissions * Add name to newly created NICs and the corresponding firewall tags. .. _Changelog-0.14.9: v0.14.9 ======= Released: Mon Nov 11 12:13:31 EET 2013 * Astakos: Fix minor problems with logging in the Astakos module, which could lead to unexpected exceptions .. _Changelog-0.14.8: v0.14.8 ======= Released: Fri Nov 8 17:25:08 EET 2013 Synnefo-wide ------------ * This is the first release to support Debian Wheezy along with Squeeze. You can mix and match nodes freely. * Update Django dependency to Django>=1.2,<1.5. Django 1.4.5 is available for Squeeze through squeeze-backports. * Since this version, Synnefo ships an example Gunicorn configuration file that is automatically installed at ``/etc/gunicorn.d/synnefo.example``. Cyclades -------- * On VM creation, pass the hashmap of the image (pithosmap://) instead of the image URL (pithos://). Access to the Pithos DB by Ganeti nodes is no longer required. * Workaround race between server creation and server deletion. This will be fixed properly by updating Ganeti to support the 'depends' attribute for OP_INSTANCE_REMOVE. Astakos ------- * For Shibboleth logins, store all attributes along with the user in the DB. .. _Changelog-0.14.7: v0.14.7 ======= Released: Wed Sep 18 17:50:12 EEST 2013 Cyclades -------- * Fix bug in helpdesk view .. _Changelog-0.14.6: v0.14.6 ======= Released: Wed Sep 18 16:18:58 EEST 2013 Pithos ------ * Substitute the PITHOS_BACKEND_QUOTA setting with two distinct settings: PITHOS_BACKEND_ACCOUNT_QUOTA & PITHOS_BACKEND_CONTAINER_QUOTA * Set PITHOS_BACKEND_CONTAINER_QUOTA default value to 0 (unlimited) * Fix bug that resulted in DB deadlocks. Cyclades -------- * Fix bug in snf-dispatcher that resulted in servers to be deleted from the DB even if the corresponding Ganeti job failed. Branding -------- * Add new BRANDING_FOOTER_EXTRA_MESSAGE setting. .. _Changelog-0.14.5: v0.14.5 ======= Released: Wed Aug 7 11:19:49 EEST 2013 Pithos ------ * Fix security issue with handling Pithos versions. .. _Changelog-0.14.4: v0.14.4 ======= Released: Mon Jul 29 12:24:22 EEST 2013 Pithos ------ * Fix bug in reconcile resources management command. .. _Changelog-0.14.3: v0.14.3 ======= Released: Thu Jul 25 12:22:47 EEST 2013 Synnefo-wide ------------ * Use the SYNNEFO_TRACE environmental variable to control whether the greenlet tracing code will get loaded or not. * Split the HIDDEN_COOKIES setting in HIDDEN_HEADERS and HIDDEN_COOKIES, and add the MAIL_MAX_LEN setting, to limit the mail size for unhandled exceptions. .. _Changelog-0.14.2: Released: Fri Jul 12 13:13:32 EEST 2013 v0.14.2 ======= Cyclades -------- * Add new setting PITHOS_BACKEND_POOL_SIZE, which configures the size of the pool of Pithos backends that are used by plankton. Pithos ------ * Refactor metadata schema (table attributes) in Pithos DB to speedup current objects by domain attribute. This is used by Plankton for listing VM images. .. _Changelog-0.14: v0.14 ===== Released: Tue Jun 25 14:01:19 EEST 2013 Synnefo-wide ------------ * Create 'snf_django' Python package to hold common code for all Synnefo components. * Create a JSON-exportable definition document for each Synnefo Components (Astakos, Cyclades, Pithos, etc.) that consolidates APIs (services), resources, and other standardized properties (e.g. default URL prefixes). * Standardize URLs for Synnefo Components, impose structure and naming conventions to related settings. Make each component deployable under a user-configurable _BASE_URL. Each API (compute, image, etc.) is deployable under a developer-configurable prefix beneath BASE_URL. * Deprecate CLOUDBAR_ACTIVE_SERVICE setting from all apps. * Common synnefo 404/500 templates (located in snf-webproject) Astakos ------- * Redesign of the accounting system (quotaholder) and integration into Astakos. * Simplified the quotaholder model; removed tables Entity and Policy; now table Holding contains limit and usage for every holding. * Extended table Holding, so that we can keep track of quota for every valid combination of holder (e.g. user), resource, and source (e.g. the default system or some specific project). * Refactored code for issuing and resolving commissions for robustness; added a 'force' option to bypass the upper limit check when issuing a commission. * Simplified syncing to the quotaholder; removed fields from models Project and ProjectMembership, previously needed for syncing; removed state PROJECT_DEACTIVATED from ProjectMembership. * Removed settings ASTAKOS_QUOTAHOLDER_URL, ASTAKOS_QUOTAHOLDER_TOKEN, and ASTAKOS_QUOTAHOLDER_POOLSIZE. * API-related changes: * Implemented API calls for quota, resources, and commissions. * Moved all API calls under '/account/v1.0'. * Implemented the keystone API call POST /tokens under '/identity/v2.0'. * Service and resource specification and handling: * Specified a format for defining services along with the API endpoints and the resources they expose. Migrated internal resource name by prefixing it with service name (e.g. 'vm' becomes 'cyclades.vm'); renamed registered service 'pithos+' to 'pithos'. * Specified a procedure to register a Synnefo component, its services and their resources in astakos and set the resources' default base quota limit. Removed resource definitions from settings. * Moved service and resource presentation data out of the respective db models into a separate file of UI constants. * Converted the limit on pending applications from a setting to a quotable resource. Converted the related user setting to a user-specific base quota limit. Deprecated model UserSetting; removed setting ASTAKOS_PENDING_APPLICATION_LIMIT. * Changes in locking strategy: * Lock only project's chain for all project operations; lock user before syncing to quotaholder. * When locking multiple rows (e.g. users or holdings) include an ORDER BY clause in the query to impose ordering on locking. * Changes in views: * Replaced custom transaction context with a simple decorator for managing transactions and a context 'ExceptionHandler', which logs and suppresses exceptions * Added fine grain user auth provider's policies. * Administrator can override default auth provider policies to a specific user or group of users. * Optionally a user can be assigned to a list of groups, based on the authentication method he choosed to signup. * Removed explicit handling of SMTP errors on each email delivery. Exceptions are now propagated to base django exception handler. * Email used in html/email tempaltes which prompt user to contact for service support prompts is now defined in ``CONTACT_EMAIL`` setting introduced in snf-common settings. * Improvements in user activation flow * User moderation now takes place after the user has verified his email address. * User model enriched with additional user state fields * Split activation email from moderation process. Administrator is required to moderate user explicitly using the `user-modify --accept` or `user-modify --reject` commands. * Improved logging throught out user activation procedures. * Remove deprecated AstakosUser model fields: `provider`, `third_party_identifier` * Allow override of authentication provider messages using the following format in setting names: ``ASTAKOS___MSG`` * Cloudbar automatically tries to identify the active service based on window location. * Removing authentication provider view is now CSRF protected. * New `API access` view, containing useful information to users on how to access available Synnefo services API's. * Remove of ASTAKOS_*_EMAIL_SUBJECT settings. All email subjects are now defined in astakos.im.messages module. Overriding default values can be achieved using custom gettext files or using astakos messages settings:: #change of greeting email subject ASTAKOS_GREETING_EMAIL_SUBJECT_MESSAGE = 'Welcome to my cloud' * Remove ``ASTAKOS_ACTIVATION_REDIRECT_URL`` and ``ASTAKOS_LOGIN_SUCCESS_URL`` from astakos .conf file. Settings are dynamically computed based on ``ASTAKOS_BASE_URL``. * Management commands: * Introduced new commands: * authpolicy-{add, list, remove, set, show} * group-{add, list} * component-{add, list, modify, remove} * reconcile-resources-astakos * resource-{export-astakos, import, modify} * service-{export-astakos, import, show} * Renamed commands: * astakos-quota to quota * user-update to user-modify * full-cleanup to cleanup-full * Removed commands: * astakos-init * invitation-{details, list} * project-sync * resource-{add, remove} * service-{add, remove, token-renew, update} * user-invite * user-set-initial-quota (integrated its functionality in user-modify and quota) * Added quota and project-related information in user-show command; added membership information in project-show. Cyclades -------- * Make 'type' attribute required for network create API request. * Networks not created to all Ganeti backends upon creation, they are instead created to a backend only when a VM connects to the network. * Add 'CYCLADES_ASTAKOSCLIENT_POOLSIZE' setting which tunes the size of the http connection pool to astakos. * Remove 'CYCLADES_USER_CATALOG_URL' and 'CYCLADES_USER_FEEDBACK_URL' settings * Remove CYCLADES_USE_QUOTAHOLDER, CYCLADES_QUOTAHOLDER_TOKEN, CYCLADES_QUOTAHOLDER_URL, CYCLADES_QUOTAHOLDER_POOLSIZE settings * Rename 'cyclades-usage-verify' management command to 'reconcile-resources-cyclades'. Also, remove 'cyclades-usage-reset' command, which is equivalent to 'reconcile-resources-cyclades --fix'. * Rename 'cyclades-reconcile-commissions' management command to 'reconcile-commissions-cyclades'. * Remove obsolete 'MAX_VMS_PER_USER', 'MAX_NETWORKS_PER_USER', "VMS_USER_QUOTA" and "NETWORKS_USER_QUOTA" settings, since their usage is covered by Quotaholder. * Remove obsolete setting 'API_ROOT_URL', since it is being covered by the use of CYCLADES_BASE_URL* Remove obsolete setting 'API_ROOT_URL', since it is being covered by 'CYCLADES_BASE_URL'. * Remove obsolete settings GANETI_DISK_TEMPLATES and DEFAULT_GANETI_DISK_TEMPLATE Cyclades helpdesk ----------------- * Additional start/stop vm action * Display extend backend info in vm's view * Fixed IP lookup Pithos ------ * Remove PITHOS_AUTHENTICATION_USERS setting, which was used to override astakos users. * Remove 'PITHOS_USER_CATALOG_URL', 'PITHOS_USER_FEEDBACK_URL' and 'PITHOS_USER_LOGIN_URL' settings. * Remove PITHOS_USE_QUOTAHOLDER, PITHOS_QUOTAHOLDER_URL, PITHOS_QUOTAHOLDER_TOKEN and PITHOS_ASTAKOSCLIENT_POOLSIZE * Enforce container-level atomicity in (most) Pithos API calls. Tools ----- .. _Changelog-0.13: v0.13 ===== Released: Wed Apr 10 18:52:50 EEST 2013 In v0.13 the code was very heavily refactored for increased uniformity since most of the Synnefo components have been merged into a single repository. Thus, **just for this version** we will not document a complete Changelog (features, fixes, improvements, issues, setting changes), but rather just copy from the `NEWS` file with minor additions wherever needed. Synnefo-wide ------------ * Support for pooling throughout Synnefo * Pooled Django DB connections, Pithos backend connections, HTTP connections using single `objpool` package * Improved management commands * Unified codebase for output of tables in JSON, CSV * Bring most of Synnefo code inside a single, unified repository * support automatic Python and Debian package builds for individual commits * with automatic version generation * Overhauling of Synnefo settings: renames and refactoring, for increased uniformity (in progress) * Deployment: Standardize on gunicorn, with gevent-based workers and use of Green threads throughout Synnefo * Documentation: New scale-out guide, with distinct node roles, for mass Synnefo deployments Astakos ------- * Support multiple authentication methods * Classic (username/password), Shibboleth, LDAP/Active Directory, Google, Twitter, LinkedIn * Users can enable/disable auth methods, and switch between them * Introduce a UUID as a global identifier for users, throughout Synnefo * The UUID remains constant as the user enables/disables login methods * Allow users to modify their email address freely * Per-user, per-resource accounting mechanism (quotaholder) * Full quota support, with per-user, per-resource quotas, based on quotaholder * Projects: Users can create and join Projects * Projects grant extra resources to their members * UI Enhancements for quotas and projects * distinct Usage tab, showing usage of individual resources * Project management UI * New Overview page * refactored/improved /login endpoint used by desktop/mobile clients. * endpoint url is now exposed by `weblogin` service * clients should use unauthenticated identity/tokens api to resolve the endpoint url * view only allows redirects to `pithos://` scheme urls * removed uuid from redirect parameters. Client should use authenticated request to identity/tokens to retrieve user uuid. Cyclades -------- * Commission resources on quotaholder/Astakos * Support mass creation of flavors * Support for the ExtStorage disk template in Ganeti * Query and report quotas in the UI * Pass VM configuration parameters over a VM-side API (`vmapi`) * Do not pass sensitive data as Ganeti OS parameters * Keep sensitive data in memory caches (memcached) and never allow them to hit the disk * Display additional backend information in helpdesk machines list * Allow helpdesk users to search for an account using a known machine id * Helpdesk actions are now logged using the synnefo's common login infrastructure UI ^^ * Removed feedback endpoint. Feedback requests delegate to astakos feedback service. ``FEEDBACK_CONTACTS``, ``FEEDBACK_EMAIL_FROM`` settings removed, and no longer used. * ``UI_LOGIN_URL``, ``UI_GLANCE_URL``, ``COMPUTE_URL`` settings no longer required to be set and are dynamically computed based on ``ASTAKOS_BASE_URL`` and ``CYCLADES_BASE_URL`` settings. * File group is no longer included in ssh keys personality metadata sent in create vm calls. Pithos ------ * Support storage of blocks on a RADOS backend, for Archipelago * new settings: PITHOS_RADOS_STORAGE, PITHOS_RADOS_POOL_BLOCKS, PITHOS_RADOS_POOL_MAPS * X-Object-Public now contains full url (domain + proper component prefix + file path) * Rewritten support for public URLs, with admin-selectable length * new settings: PITHOS_PUBLIC_URL_SECURITY, PITHOS_PUBLIC_URL_ALPHABET * Enable pithos backend to use external quotaholder component * new settings: PITHOS_USE_QUOTAHOLDER, PITHOS_QUOTAHOLDER_URL, PITHOS_QUOTAHOLDER_TOKEN, PITHOS_QUOTAHOLDER_POOLSIZE * Moderated version debiting mechanism * new setting: PITHOS_BACKEND_FREE_VERSIONING * Proxy Astakos user-visible services * new settings: PITHOS_PROXY_USER_SERVICES, PITHOS_USER_CATALOG_URL, PITHOS_USER_FEEDBACK_URL, PITHOS_USER_LOGIN_URL Tools ----- * Extend snf-burnin to include testing of Pithos functionality