Commit fb3e4750 authored by Giorgos Korfiatis's avatar Giorgos Korfiatis

astakos: Rename base to system projects

Rename base projects to system projects in management commands and API.
Update docs and changelog.
parent 3937e5f4
...@@ -19,18 +19,18 @@ Synnefo-wide ...@@ -19,18 +19,18 @@ Synnefo-wide
* Projects are now viewed as a source of finite resources. A member can * Projects are now viewed as a source of finite resources. A member can
reserve a part of these resources up to a specified limit. reserve a part of these resources up to a specified limit.
* Base quota are now offered through a special purpose user-specific base * Base quota are now offered through a special purpose user-specific system
project, identified with the same UUID as the user. project, identified with the same UUID as the user.
* Each actual resource (Cyclades VM, network, floating IP and Pithos * Each actual resource (Cyclades VM, network, floating IP and Pithos
container) is now also associated with a project besides the owner. container) is now also associated with a project besides the owner.
* In resource creation, project defaults to the user-specific base * In resource creation, project defaults to the user-specific system
project, if not specified otherwise. It is also possible to change the project, if not specified otherwise. It is also possible to change the
project assignment of an existing resource. project assignment of an existing resource.
* All existing resources have been assigned to the respective * All existing resources have been assigned to the respective
user-specific base projects. user-specific system projects.
* Logging mechanism for Synnefo management commands * Logging mechanism for Synnefo management commands
...@@ -65,8 +65,8 @@ Astakos ...@@ -65,8 +65,8 @@ Astakos
activation, resources missing are automatically completed using a activation, resources missing are automatically completed using a
skeleton. skeleton.
* Field `uplimit' of registed resources is exposed as `base_default' and * Field `uplimit' of registed resources is exposed as `system_default' and
provide the skeleton for user-specific base projects. A new field provide the skeleton for user-specific system projects. A new field
`project_default' is introduce to act as a skeleton for conventional `project_default' is introduce to act as a skeleton for conventional
projects. projects.
......
...@@ -326,17 +326,17 @@ Default quota ...@@ -326,17 +326,17 @@ Default quota
````````````` `````````````
Upon user creation, a special purpose user-specific project is automatically Upon user creation, a special purpose user-specific project is automatically
created in order to hold the base quota provided by the system. These *base* created in order to hold the quota provided by the system. These *system*
projects are identified with the same UUID as the user. projects are identified with the same UUID as the user.
To inspect the quota that future users will receive by default through their To inspect the quota that future users will receive by default through their
base projects, check column ``base_default`` in:: base projects, check column ``system_default`` in::
# snf-manage resource-list # snf-manage resource-list
You can modify the default base quota limit for all future users with:: You can modify the default system quota limit for all future users with::
# snf-manage resource-modify <resource_name> --base-default <value> # snf-manage resource-modify <resource_name> --system-default <value>
Grant extra quota through projects Grant extra quota through projects
`````````````````````````````````` ``````````````````````````````````
...@@ -392,10 +392,10 @@ One can change the quota limits of an initialized project with:: ...@@ -392,10 +392,10 @@ One can change the quota limits of an initialized project with::
# snf-manage project-modify <project-uuid> --limit <resource_name> <member_limit> <project_limit> # snf-manage project-modify <project-uuid> --limit <resource_name> <member_limit> <project_limit>
One can set base quota for all accepted users (that is, set limits for base One can set system quota for all accepted users (that is, set limits for system
project), with possible exceptions, with:: projects), with possible exceptions, with::
# snf-manage project-modify --all-base-projects --exclude <uuid1>,<uuid2> --limit ... # snf-manage project-modify --all-system-projects --exclude <uuid1>,<uuid2> --limit ...
Quota for a given resource are reported for all projects that the user is Quota for a given resource are reported for all projects that the user is
member in with:: member in with::
......
...@@ -528,7 +528,7 @@ project Project assignment ✔ **✘** ...@@ -528,7 +528,7 @@ project Project assignment ✔ **✘**
should rather be defined should rather be defined
* **project** (optional) is the project where the VM is to be assigned. If not * **project** (optional) is the project where the VM is to be assigned. If not
given, user's base project is assumed (identified with the same uuid as the given, user's system project is assumed (identified with the same uuid as the
user). user).
* **personality** (optional) is a list of personality injections. A personality * **personality** (optional) is a list of personality injections. A personality
......
...@@ -40,7 +40,7 @@ cannot exceed the former. A limit on the number of members allowed is still ...@@ -40,7 +40,7 @@ cannot exceed the former. A limit on the number of members allowed is still
enforced. enforced.
Projects will be the sole source of resources. Current base quota offered to Projects will be the sole source of resources. Current base quota offered to
users by the system will be expressed in terms of special-purpose *base* users by the system will be expressed in terms of special-purpose *system*
projects. Due to the central role that projects now acquire, we will alter projects. Due to the central role that projects now acquire, we will alter
the project schema to facilitate project creation and modification without the project schema to facilitate project creation and modification without
the extra overhead of submitting and approving applications. the extra overhead of submitting and approving applications.
...@@ -90,29 +90,29 @@ Deallocation is always allowed as long as usage does not fall below zero. ...@@ -90,29 +90,29 @@ Deallocation is always allowed as long as usage does not fall below zero.
Counters with zero usage and limit could by garbage collected by Astakos, if Counters with zero usage and limit could by garbage collected by Astakos, if
needed. needed.
Base projects System projects
------------- ---------------
For reasons of uniformity, we replace the base quota mechanism with projects. For reasons of uniformity, we replace the base quota mechanism with projects.
In a similar vein to OpenStack tenants, we define new user-specific *base* In a similar vein to OpenStack tenants, we define new user-specific *system*
projects to account for the base quota for each user. These projects should projects to account for the base quota for each user. These projects should
be clearly associated with a single user, restrict join/leave actions and be clearly associated with a single user, restrict join/leave actions and
specify the quota granted by the system. When a new user is created, specify the quota granted by the system. When a new user is created,
their base project will be automatically created and linked back to the user. their system project will be automatically created and linked back to the user.
User activation will trigger project activation, granting the default resource User activation will trigger project activation, granting the default resource
quota. Base projects will have no owner, marked thusly as `system' projects. quota. These projects will have no owner, marked thusly as `system' projects.
The administrator can, following the usual project logic, alter quota by The administrator can, following the usual project logic, alter quota by
modifying the project. Users cannot apply for modification of their base modifying the project. Users cannot apply for modification of their system
projects. projects.
Projects will, from now on, be identified by a UUID. Base projects will Projects will, from now on, be identified by a UUID. System projects will
receive the same UUID as the user itself. ProjectID, which appears above in receive the same UUID as the user itself. ProjectID, which appears above in
the Quotaholder entries, refers to the project UUID. the Quotaholder entries, refers to the project UUID.
Base quota will be expressed both in terms of a project-level and a Base quota will be expressed both in terms of a project-level and a
member-level limit. This will result in two operationally equivalent member-level limit. This will result in two operationally equivalent
Quotaholder counters, as in the following example. In the future, we could Quotaholder counters, as in the following example. In the future, we could
admit third-party users to a user's base project; in that case, those admit third-party users to a user's system project; in that case, those
counters would differ. counters would differ.
:: ::
...@@ -125,23 +125,23 @@ counters would differ. ...@@ -125,23 +125,23 @@ counters would differ.
Private projects Private projects
---------------- ----------------
Since the introduction of base projects will explode the number of total Since the introduction of system projects will explode the number of total
projects, we will need to control their visibility. We add a new flag projects, we will need to control their visibility. We add a new flag
*private* in project definitions. A private project can only be accessed by *private* in project definitions. A private project can only be accessed by
its owner and members and not be advertized in the UI. Base projects are its owner and members and not be advertized in the UI. System projects are
marked as private. marked as private.
Decouple projects from applications Decouple projects from applications
----------------------------------- -----------------------------------
Base projects do not fit well in the current project/application scheme, System projects do not fit well in the current project/application scheme,
because no user has applied for them. Moveover, we would like to easily because no user has applied for them. Moveover, we would like to easily
modify project properties, particularly quota limits, without the need to modify project properties, particularly quota limits, without the need to
apply for an application for each project and then approve it. apply for an application for each project and then approve it.
We will decouple projects from applications by incorporating the project We will decouple projects from applications by incorporating the project
definition into the project object rather than relying on an application. definition into the project object rather than relying on an application.
The system will directly make a new (base) project upon user creation and a The system will directly make a new (system) project upon user creation and a
privileged user will be able to modify an existing project by directly privileged user will be able to modify an existing project by directly
modifying it. An unprivileged user will still need to make an application. modifying it. An unprivileged user will still need to make an application.
...@@ -169,7 +169,7 @@ System default quota and resource registration ...@@ -169,7 +169,7 @@ System default quota and resource registration
Each resource registered in the system is assigned a default quota limit. Each resource registered in the system is assigned a default quota limit.
A newly-activated user is given these limits as their base quota. This is A newly-activated user is given these limits as their base quota. This is
till now done by copying the default limits as user's entries in till now done by copying the default limits as user's entries in
AstakosUserQuota. Default limits will from now on be copied into the base AstakosUserQuota. Default limits will from now on be copied into the system
project's resource definitions. project's resource definitions.
Conventional projects are created through a project application, which Conventional projects are created through a project application, which
...@@ -182,7 +182,7 @@ one specifying the default base quota, in order to fill in missing limits ...@@ -182,7 +182,7 @@ one specifying the default base quota, in order to fill in missing limits
for conventional projects. It will be controled by a new option for conventional projects. It will be controled by a new option
``--project-default`` of command ``resource-modify``. ``--project-default`` of command ``resource-modify``.
When a project is activated, either directly in the case of base projects When a project is activated, either directly in the case of system projects
or through the approval of a project application, limits for resources not or through the approval of a project application, limits for resources not
specified are automatically completed by consulting the appropriate specified are automatically completed by consulting the appropriate
skeleton. skeleton.
...@@ -237,7 +237,7 @@ In Cyclades, each VM, floating IP, or other distinct resource should be ...@@ -237,7 +237,7 @@ In Cyclades, each VM, floating IP, or other distinct resource should be
linked to a project. Pithos should link containers to projects. linked to a project. Pithos should link containers to projects.
Astakos will handle its own resource ``astakos.pending_app`` in a special Astakos will handle its own resource ``astakos.pending_app`` in a special
way: it will always be charged at the user's base project. way: it will always be charged at the user's system project.
Resource reassignment Resource reassignment
--------------------- ---------------------
...@@ -356,7 +356,7 @@ All service API calls that create resources can specify the project where ...@@ -356,7 +356,7 @@ All service API calls that create resources can specify the project where
they will be attributed. they will be attributed.
In cyclades, ``POST /servers`` (likewise for networks and floating IPs) will In cyclades, ``POST /servers`` (likewise for networks and floating IPs) will
receive an extra argument ``project``. If it is missing, the user's base receive an extra argument ``project``. If it is missing, the user's system
project will be assumed. In calls detailing a resource (e.g., ``GET project will be assumed. In calls detailing a resource (e.g., ``GET
/servers/<server_id>``), the field ``tenant_id`` will contain the /servers/<server_id>``), the field ``tenant_id`` will contain the
project id. project id.
...@@ -399,7 +399,7 @@ User interface ...@@ -399,7 +399,7 @@ User interface
User quota will be presented per project, including the aggregate activity User quota will be presented per project, including the aggregate activity
of other project members: the Resource Usage page will include a drop-down of other project members: the Resource Usage page will include a drop-down
menu with all relevant projects. By default, user's base project will menu with all relevant projects. By default, user's system project will
be assumed. When choosing a project, usage for all resources will be be assumed. When choosing a project, usage for all resources will be
presented for the given project in the following style:: presented for the given project in the following style::
...@@ -462,7 +462,7 @@ Currently, the administrator can change the user base quota with: ...@@ -462,7 +462,7 @@ Currently, the administrator can change the user base quota with:
``snf-manage user-modify <id> --base-quota <resource> <capacity>``. ``snf-manage user-modify <id> --base-quota <resource> <capacity>``.
This will be removed in favor of the ``project-modify`` command, so that all This will be removed in favor of the ``project-modify`` command, so that all
quota are handled in a uniform way. Similar to ``user-modify --all``, quota are handled in a uniform way. Similar to ``user-modify --all``,
``project-modify`` will get options ``--all-base-projects`` to ``project-modify`` will get options ``--all-system-projects`` to
allow updating base quota in bulk. allow updating base quota in bulk.
Migration steps Migration steps
...@@ -475,24 +475,24 @@ Existing projects need to be converted to resource-pool ones. The following ...@@ -475,24 +475,24 @@ Existing projects need to be converted to resource-pool ones. The following
steps must be taken in Astakos: steps must be taken in Astakos:
* compute project-level limits for each resource as * compute project-level limits for each resource as
max_members * member-level limit max_members * member-level limit
* create base projects based on base quota for each user * create system projects based on base quota for each user
* make Quotaholder entries for projects and user/project pairs * make Quotaholder entries for projects and user/project pairs
* assign all current usage to the base projects (both project * assign all current usage to the system projects (both project
and user/project entries) and user/project entries)
* set usage for all other entries to zero * set usage for all other entries to zero
Cyclades and Pithos should initialize their project attribute on each resource Cyclades and Pithos should initialize their project attribute on each resource
with the user's base project, that is, the same UUID as the resource owner. with the user's system project, that is, the same UUID as the resource owner.
Initial resource reassignment Initial resource reassignment
----------------------------- -----------------------------
Once migration has finished, users will be off-quota on their base project, Once migration has finished, users will be off-quota on their system project,
if they had used additional quota from projects. To alleviate this if they had used additional quota from projects. To alleviate this
situation, each service can attempt to reassign resources to other projects, situation, each service can attempt to reassign resources to other projects,
following this strategy: following this strategy:
* consult Astakos for projects and quota for a given user * consult Astakos for projects and quota for a given user
* select resources that can fit in another project * select resources that can fit in another project
* issue a commission to decrease usage of the base project and likewise * issue a commission to decrease usage of the system project and likewise
increase usage of the available project increase usage of the available project
* record the new ProjectUUID for the reassigned resources * record the new ProjectUUID for the reassigned resources
...@@ -912,7 +912,7 @@ resource. ...@@ -912,7 +912,7 @@ resource.
.. code-block:: console .. code-block:: console
# snf-manage resource-modify cyclades.vm --base-default 2 # snf-manage resource-modify cyclades.vm --system-default 2
Setting Resource Visibility Setting Resource Visibility
--------------------------- ---------------------------
......
...@@ -91,7 +91,7 @@ Status Description ...@@ -91,7 +91,7 @@ Status Description
"leave_policy": "auto" | "moderated" | "closed", "leave_policy": "auto" | "moderated" | "closed",
"max_members": int, "max_members": int,
"private": boolean, "private": boolean,
"base_project": boolean, "system_project": boolean,
"resources": {"cyclades.vm": {"project_capacity": int, "resources": {"cyclades.vm": {"project_capacity": int,
"member_capacity": int "member_capacity": int
} }
......
...@@ -49,7 +49,7 @@ Quotas ...@@ -49,7 +49,7 @@ Quotas
The system specifies user quotas for each available resource. Resources The system specifies user quotas for each available resource. Resources
can be allocated from various sources. By default, users get resources can be allocated from various sources. By default, users get resources
from their user-specific `base projects', which are identified by the same from their user-specific `system projects', which are identified by the same
uuid as the users. For each combination of user, uuid as the users. For each combination of user,
source, and resource, the quota system keeps track of the maximum allowed source, and resource, the quota system keeps track of the maximum allowed
value (limit) and the current actual usage. The former is controlled by value (limit) and the current actual usage. The former is controlled by
...@@ -96,7 +96,7 @@ Status Description ...@@ -96,7 +96,7 @@ Status Description
.. code-block:: javascript .. code-block:: javascript
{ {
"base_project_id": { "system_project_id": {
"cyclades.ram": { "cyclades.ram": {
"usage": 536870912, "usage": 536870912,
"limit": 1073741824, "limit": 1073741824,
...@@ -169,7 +169,7 @@ Status Description ...@@ -169,7 +169,7 @@ Status Description
{ {
"1a6165d0-5020-4b6d-a4ad-83476632a584": { "1a6165d0-5020-4b6d-a4ad-83476632a584": {
"base_project_id": { "system_project_id": {
"cyclades.ram": { "cyclades.ram": {
"usage": 536870912, "usage": 536870912,
"limit": 1073741824, "limit": 1073741824,
...@@ -238,7 +238,7 @@ Status Description ...@@ -238,7 +238,7 @@ Status Description
.. code-block:: javascript .. code-block:: javascript
{ {
"base_project_id": { "system_project_id": {
"cyclades.ram": { "cyclades.ram": {
"project_usage": 536870912, "project_usage": 536870912,
"project_limit": 1073741824, "project_limit": 1073741824,
...@@ -250,7 +250,7 @@ Status Description ...@@ -250,7 +250,7 @@ Status Description
"project_pending": 0 "project_pending": 0
} }
}, },
"base_project2_id": { "system_project2_id": {
"cyclades.ram": { "cyclades.ram": {
"project_usage": 0, "project_usage": 0,
"project_limit": 1073741824, "project_limit": 1073741824,
......
...@@ -116,24 +116,24 @@ are now viewed as a source of finite resources, instead of a means to ...@@ -116,24 +116,24 @@ are now viewed as a source of finite resources, instead of a means to
accumulate quota. They are the single source of resources, and quota are now accumulate quota. They are the single source of resources, and quota are now
managed at a project/member level. managed at a project/member level.
System-provided base quota are now handled through special purpose System-provided quota are now handled through special purpose
user-specific *base projects*, identified with the same UUID as the user. user-specific *system projects*, identified with the same UUID as the user.
These have been created during the database migration process. They are These have been created during the database migration process. They are
included in the project listing with:: included in the project listing with::
snf-manage project-list --base-projects snf-manage project-list --system-projects
All projects must specify quota limits for all registered resources. Default All projects must specify quota limits for all registered resources. Default
values have been set for all resources, listed with:: values have been set for all resources, listed with::
astakos-host$ snf-manage resource-list astakos-host$ snf-manage resource-list
Column `base_default` (previously known as `default_quota`) provides the Column `system_default` (previously known as `default_quota`) provides the
skeleton for the quota limits of user-specific base projects. Column skeleton for the quota limits of user-specific system projects. Column
`project_default` is new and acts as skeleton for `applied` (non-base) `project_default` is new and acts as skeleton for `applied` (non-system)
projects (i.e., for resources not specified in a project application). projects (i.e., for resources not specified in a project application).
Project defaults have been initialized during migration based on the base Project defaults have been initialized during migration based on the system
default values: they have been set to `inf` if `base_default` is also `inf`, default values: they have been set to `inf` if `system_default` is also `inf`,
otherwise set to zero. otherwise set to zero.
This default, affecting all future projects, can be modified with:: This default, affecting all future projects, can be modified with::
...@@ -154,8 +154,8 @@ order to change a project's resource limits, run:: ...@@ -154,8 +154,8 @@ order to change a project's resource limits, run::
With the new mechanism, when a new resource is allocated (e.g., a VM or a With the new mechanism, when a new resource is allocated (e.g., a VM or a
Pithos container is created), it is also associated with a project besides Pithos container is created), it is also associated with a project besides
its owner. The migration process has associated existing resources with its owner. The migration process has associated existing resources with
their owner's base project. Note that users who had made use of projects to their owner's system project. Note that users who had made use of projects to
increase their quota may end up overlimit on some resources of their base increase their quota may end up overlimit on some resources of their system
projects and will need to *reassign* some of their reserved resources to projects and will need to *reassign* some of their reserved resources to
another project in order to overcome this restriction. another project in order to overcome this restriction.
......
...@@ -140,7 +140,7 @@ def get_projects_details(projects, request_user=None): ...@@ -140,7 +140,7 @@ def get_projects_details(projects, request_user=None):
"leave_policy": leave_policy, "leave_policy": leave_policy,
"max_members": project.limit_on_members_number, "max_members": project.limit_on_members_number,
"private": project.private, "private": project.private,
"base_project": project.is_base, "system_project": project.is_base,
"resources": resources, "resources": resources,
} }
......
...@@ -689,7 +689,7 @@ def make_base_project(username): ...@@ -689,7 +689,7 @@ def make_base_project(username):
owner=None, owner=None,
realname="tmp", realname="tmp",
homepage="", homepage="",
description=("base project for user " + username), description=("system project for user " + username),
end_date=(datetime.now() + relativedelta(years=100)), end_date=(datetime.now() + relativedelta(years=100)),
member_join_policy=CLOSED_POLICY, member_join_policy=CLOSED_POLICY,
member_leave_policy=CLOSED_POLICY, member_leave_policy=CLOSED_POLICY,
...@@ -829,7 +829,7 @@ def submit_application(owner=None, ...@@ -829,7 +829,7 @@ def submit_application(owner=None,
if [x for x in [owner, name, homepage, description] if x is not None]: if [x for x in [owner, name, homepage, description] if x is not None]:
raise ProjectConflict( raise ProjectConflict(
"Cannot modify fields 'owner', 'name', 'homepage', and " "Cannot modify fields 'owner', 'name', 'homepage', and "
"'description' of a base project.") "'description' of a system project.")
application = ProjectApplication.objects.create( application = ProjectApplication.objects.create(
applicant=request_user, applicant=request_user,
......
...@@ -71,10 +71,10 @@ class Command(ListCommand): ...@@ -71,10 +71,10 @@ class Command(ListCommand):
dest='deleted', dest='deleted',
default=False, default=False,
help="Also show cancelled/terminated projects"), help="Also show cancelled/terminated projects"),
make_option('--base-projects', make_option('--system-projects',
action='store_true', action='store_true',
default=False, default=False,
help="Also show base projects"), help="Also show system projects"),
) )
def get_owner(project): def get_owner(project):
...@@ -118,7 +118,7 @@ class Command(ListCommand): ...@@ -118,7 +118,7 @@ class Command(ListCommand):
if not options['deleted']: if not options['deleted']:
self.excludes["state__in"] = Project.SKIP_STATES self.excludes["state__in"] = Project.SKIP_STATES
if not options['base_projects']: if not options['system_projects']:
self.excludes["is_base"] = True self.excludes["is_base"] = True
if options["pending"]: if options["pending"]:
......
...@@ -95,25 +95,25 @@ def make_options(): ...@@ -95,25 +95,25 @@ def make_options():
class Command(SynnefoCommand): class Command(SynnefoCommand):
args = "<project id> (or --all-base-projects)" args = "<project id> (or --all-system-projects)"
help = "Modify an already initialized project" help = "Modify an already initialized project"
option_list = SynnefoCommand.option_list + make_options() + ( option_list = SynnefoCommand.option_list + make_options() + (
make_option('--all-base-projects', make_option('--all-system-projects',
action='store_true', action='store_true',
default=False, default=False,
help="Modify in bulk all initialized base projects"), help="Modify in bulk all initialized system projects"),
make_option('--exclude', make_option('--exclude',
help=("If `--all-base-projects' is given, exclude projects" help=("If `--all-system-projects' is given, exclude projects"
" given as a list of uuids: uuid1,uuid2,uuid3")), " given as a list of uuids: uuid1,uuid2,uuid3")),
) )
def check_args(self, args, all_base, exclude): def check_args(self, args, all_base, exclude):
if all_base and args or not all_base and len(args) != 1: if all_base and args or not all_base and len(args) != 1:
m = "Please provide a project ID or --all-base-projects" m = "Please provide a project ID or --all-system-projects"
raise CommandError(m) raise CommandError(m)
if not all_base and exclude: if not all_base and exclude:
m = ("Option --exclude is meaningful only combined with " m = ("Option --exclude is meaningful only combined with "
" --all-base-projects.") " --all-system-projects.")
raise CommandError(m)