Commit fb3e4750 authored by Giorgos Korfiatis's avatar Giorgos Korfiatis
Browse files

astakos: Rename base to system projects

Rename base projects to system projects in management commands and API.
Update docs and changelog.
parent 3937e5f4
......@@ -19,18 +19,18 @@ Synnefo-wide
* Projects are now viewed as a source of finite resources. A member can
reserve a part of these resources up to a specified limit.
* Base quota are now offered through a special purpose user-specific base
* Base quota are now offered through a special purpose user-specific system
project, identified with the same UUID as the user.
* Each actual resource (Cyclades VM, network, floating IP and Pithos
container) is now also associated with a project besides the owner.
* In resource creation, project defaults to the user-specific base
* In resource creation, project defaults to the user-specific system
project, if not specified otherwise. It is also possible to change the
project assignment of an existing resource.
* All existing resources have been assigned to the respective
user-specific base projects.
user-specific system projects.
* Logging mechanism for Synnefo management commands
......@@ -65,8 +65,8 @@ Astakos
activation, resources missing are automatically completed using a
skeleton.
* Field `uplimit' of registed resources is exposed as `base_default' and
provide the skeleton for user-specific base projects. A new field
* Field `uplimit' of registed resources is exposed as `system_default' and
provide the skeleton for user-specific system projects. A new field
`project_default' is introduce to act as a skeleton for conventional
projects.
......
......@@ -326,17 +326,17 @@ Default quota
`````````````
Upon user creation, a special purpose user-specific project is automatically
created in order to hold the base quota provided by the system. These *base*
created in order to hold the quota provided by the system. These *system*
projects are identified with the same UUID as the user.
To inspect the quota that future users will receive by default through their
base projects, check column ``base_default`` in::
base projects, check column ``system_default`` in::
# snf-manage resource-list
You can modify the default base quota limit for all future users with::
You can modify the default system quota limit for all future users with::
# snf-manage resource-modify <resource_name> --base-default <value>
# snf-manage resource-modify <resource_name> --system-default <value>
Grant extra quota through projects
``````````````````````````````````
......@@ -392,10 +392,10 @@ One can change the quota limits of an initialized project with::
# snf-manage project-modify <project-uuid> --limit <resource_name> <member_limit> <project_limit>
One can set base quota for all accepted users (that is, set limits for base
project), with possible exceptions, with::
One can set system quota for all accepted users (that is, set limits for system
projects), with possible exceptions, with::
# snf-manage project-modify --all-base-projects --exclude <uuid1>,<uuid2> --limit ...
# snf-manage project-modify --all-system-projects --exclude <uuid1>,<uuid2> --limit ...
Quota for a given resource are reported for all projects that the user is
member in with::
......
......@@ -528,7 +528,7 @@ project Project assignment ✔ **✘**
should rather be defined
* **project** (optional) is the project where the VM is to be assigned. If not
given, user's base project is assumed (identified with the same uuid as the
given, user's system project is assumed (identified with the same uuid as the
user).
* **personality** (optional) is a list of personality injections. A personality
......
......@@ -40,7 +40,7 @@ cannot exceed the former. A limit on the number of members allowed is still
enforced.
Projects will be the sole source of resources. Current base quota offered to
users by the system will be expressed in terms of special-purpose *base*
users by the system will be expressed in terms of special-purpose *system*
projects. Due to the central role that projects now acquire, we will alter
the project schema to facilitate project creation and modification without
the extra overhead of submitting and approving applications.
......@@ -90,29 +90,29 @@ Deallocation is always allowed as long as usage does not fall below zero.
Counters with zero usage and limit could by garbage collected by Astakos, if
needed.
Base projects
-------------
System projects
---------------
For reasons of uniformity, we replace the base quota mechanism with projects.
In a similar vein to OpenStack tenants, we define new user-specific *base*
In a similar vein to OpenStack tenants, we define new user-specific *system*
projects to account for the base quota for each user. These projects should
be clearly associated with a single user, restrict join/leave actions and
specify the quota granted by the system. When a new user is created,
their base project will be automatically created and linked back to the user.
their system project will be automatically created and linked back to the user.
User activation will trigger project activation, granting the default resource
quota. Base projects will have no owner, marked thusly as `system' projects.
quota. These projects will have no owner, marked thusly as `system' projects.
The administrator can, following the usual project logic, alter quota by
modifying the project. Users cannot apply for modification of their base
modifying the project. Users cannot apply for modification of their system
projects.
Projects will, from now on, be identified by a UUID. Base projects will
Projects will, from now on, be identified by a UUID. System projects will
receive the same UUID as the user itself. ProjectID, which appears above in
the Quotaholder entries, refers to the project UUID.
Base quota will be expressed both in terms of a project-level and a
member-level limit. This will result in two operationally equivalent
Quotaholder counters, as in the following example. In the future, we could
admit third-party users to a user's base project; in that case, those
admit third-party users to a user's system project; in that case, those
counters would differ.
::
......@@ -125,23 +125,23 @@ counters would differ.
Private projects
----------------
Since the introduction of base projects will explode the number of total
Since the introduction of system projects will explode the number of total
projects, we will need to control their visibility. We add a new flag
*private* in project definitions. A private project can only be accessed by
its owner and members and not be advertized in the UI. Base projects are
its owner and members and not be advertized in the UI. System projects are
marked as private.
Decouple projects from applications
-----------------------------------
Base projects do not fit well in the current project/application scheme,
System projects do not fit well in the current project/application scheme,
because no user has applied for them. Moveover, we would like to easily
modify project properties, particularly quota limits, without the need to
apply for an application for each project and then approve it.
We will decouple projects from applications by incorporating the project
definition into the project object rather than relying on an application.
The system will directly make a new (base) project upon user creation and a
The system will directly make a new (system) project upon user creation and a
privileged user will be able to modify an existing project by directly
modifying it. An unprivileged user will still need to make an application.
......@@ -169,7 +169,7 @@ System default quota and resource registration
Each resource registered in the system is assigned a default quota limit.
A newly-activated user is given these limits as their base quota. This is
till now done by copying the default limits as user's entries in
AstakosUserQuota. Default limits will from now on be copied into the base
AstakosUserQuota. Default limits will from now on be copied into the system
project's resource definitions.
Conventional projects are created through a project application, which
......@@ -182,7 +182,7 @@ one specifying the default base quota, in order to fill in missing limits
for conventional projects. It will be controled by a new option
``--project-default`` of command ``resource-modify``.
When a project is activated, either directly in the case of base projects
When a project is activated, either directly in the case of system projects
or through the approval of a project application, limits for resources not
specified are automatically completed by consulting the appropriate
skeleton.
......@@ -237,7 +237,7 @@ In Cyclades, each VM, floating IP, or other distinct resource should be
linked to a project. Pithos should link containers to projects.
Astakos will handle its own resource ``astakos.pending_app`` in a special
way: it will always be charged at the user's base project.
way: it will always be charged at the user's system project.
Resource reassignment
---------------------
......@@ -356,7 +356,7 @@ All service API calls that create resources can specify the project where
they will be attributed.
In cyclades, ``POST /servers`` (likewise for networks and floating IPs) will
receive an extra argument ``project``. If it is missing, the user's base
receive an extra argument ``project``. If it is missing, the user's system
project will be assumed. In calls detailing a resource (e.g., ``GET
/servers/<server_id>``), the field ``tenant_id`` will contain the
project id.
......@@ -399,7 +399,7 @@ User interface
User quota will be presented per project, including the aggregate activity
of other project members: the Resource Usage page will include a drop-down
menu with all relevant projects. By default, user's base project will
menu with all relevant projects. By default, user's system project will
be assumed. When choosing a project, usage for all resources will be
presented for the given project in the following style::
......@@ -462,7 +462,7 @@ Currently, the administrator can change the user base quota with:
``snf-manage user-modify <id> --base-quota <resource> <capacity>``.
This will be removed in favor of the ``project-modify`` command, so that all
quota are handled in a uniform way. Similar to ``user-modify --all``,
``project-modify`` will get options ``--all-base-projects`` to
``project-modify`` will get options ``--all-system-projects`` to
allow updating base quota in bulk.
Migration steps
......@@ -475,24 +475,24 @@ Existing projects need to be converted to resource-pool ones. The following
steps must be taken in Astakos:
* compute project-level limits for each resource as
max_members * member-level limit
* create base projects based on base quota for each user
* create system projects based on base quota for each user
* make Quotaholder entries for projects and user/project pairs
* assign all current usage to the base projects (both project
* assign all current usage to the system projects (both project
and user/project entries)
* set usage for all other entries to zero
Cyclades and Pithos should initialize their project attribute on each resource
with the user's base project, that is, the same UUID as the resource owner.
with the user's system project, that is, the same UUID as the resource owner.
Initial resource reassignment
-----------------------------
Once migration has finished, users will be off-quota on their base project,
Once migration has finished, users will be off-quota on their system project,
if they had used additional quota from projects. To alleviate this
situation, each service can attempt to reassign resources to other projects,
following this strategy:
* consult Astakos for projects and quota for a given user
* select resources that can fit in another project
* issue a commission to decrease usage of the base project and likewise
* issue a commission to decrease usage of the system project and likewise
increase usage of the available project
* record the new ProjectUUID for the reassigned resources
......@@ -912,7 +912,7 @@ resource.
.. code-block:: console
# snf-manage resource-modify cyclades.vm --base-default 2
# snf-manage resource-modify cyclades.vm --system-default 2
Setting Resource Visibility
---------------------------
......
......@@ -91,7 +91,7 @@ Status Description
"leave_policy": "auto" | "moderated" | "closed",
"max_members": int,
"private": boolean,
"base_project": boolean,
"system_project": boolean,
"resources": {"cyclades.vm": {"project_capacity": int,
"member_capacity": int
}
......
......@@ -49,7 +49,7 @@ Quotas
The system specifies user quotas for each available resource. Resources
can be allocated from various sources. By default, users get resources
from their user-specific `base projects', which are identified by the same
from their user-specific `system projects', which are identified by the same
uuid as the users. For each combination of user,
source, and resource, the quota system keeps track of the maximum allowed
value (limit) and the current actual usage. The former is controlled by
......@@ -96,7 +96,7 @@ Status Description
.. code-block:: javascript
{
"base_project_id": {
"system_project_id": {
"cyclades.ram": {
"usage": 536870912,
"limit": 1073741824,
......@@ -169,7 +169,7 @@ Status Description
{
"1a6165d0-5020-4b6d-a4ad-83476632a584": {
"base_project_id": {
"system_project_id": {
"cyclades.ram": {
"usage": 536870912,
"limit": 1073741824,
......@@ -238,7 +238,7 @@ Status Description
.. code-block:: javascript
{
"base_project_id": {
"system_project_id": {
"cyclades.ram": {
"project_usage": 536870912,
"project_limit": 1073741824,
......@@ -250,7 +250,7 @@ Status Description
"project_pending": 0
}
},
"base_project2_id": {
"system_project2_id": {
"cyclades.ram": {
"project_usage": 0,
"project_limit": 1073741824,
......
......@@ -116,24 +116,24 @@ are now viewed as a source of finite resources, instead of a means to
accumulate quota. They are the single source of resources, and quota are now
managed at a project/member level.
System-provided base quota are now handled through special purpose
user-specific *base projects*, identified with the same UUID as the user.
System-provided quota are now handled through special purpose
user-specific *system projects*, identified with the same UUID as the user.
These have been created during the database migration process. They are
included in the project listing with::
snf-manage project-list --base-projects
snf-manage project-list --system-projects
All projects must specify quota limits for all registered resources. Default
values have been set for all resources, listed with::
astakos-host$ snf-manage resource-list
Column `base_default` (previously known as `default_quota`) provides the
skeleton for the quota limits of user-specific base projects. Column
`project_default` is new and acts as skeleton for `applied` (non-base)
Column `system_default` (previously known as `default_quota`) provides the
skeleton for the quota limits of user-specific system projects. Column
`project_default` is new and acts as skeleton for `applied` (non-system)
projects (i.e., for resources not specified in a project application).
Project defaults have been initialized during migration based on the base
default values: they have been set to `inf` if `base_default` is also `inf`,
Project defaults have been initialized during migration based on the system
default values: they have been set to `inf` if `system_default` is also `inf`,
otherwise set to zero.
This default, affecting all future projects, can be modified with::
......@@ -154,8 +154,8 @@ order to change a project's resource limits, run::
With the new mechanism, when a new resource is allocated (e.g., a VM or a
Pithos container is created), it is also associated with a project besides
its owner. The migration process has associated existing resources with
their owner's base project. Note that users who had made use of projects to
increase their quota may end up overlimit on some resources of their base
their owner's system project. Note that users who had made use of projects to
increase their quota may end up overlimit on some resources of their system
projects and will need to *reassign* some of their reserved resources to
another project in order to overcome this restriction.
......
......@@ -140,7 +140,7 @@ def get_projects_details(projects, request_user=None):
"leave_policy": leave_policy,
"max_members": project.limit_on_members_number,
"private": project.private,
"base_project": project.is_base,
"system_project": project.is_base,
"resources": resources,
}
......
......@@ -689,7 +689,7 @@ def make_base_project(username):
owner=None,
realname="tmp",
homepage="",
description=("base project for user " + username),
description=("system project for user " + username),
end_date=(datetime.now() + relativedelta(years=100)),
member_join_policy=CLOSED_POLICY,
member_leave_policy=CLOSED_POLICY,
......@@ -829,7 +829,7 @@ def submit_application(owner=None,
if [x for x in [owner, name, homepage, description] if x is not None]:
raise ProjectConflict(
"Cannot modify fields 'owner', 'name', 'homepage', and "
"'description' of a base project.")
"'description' of a system project.")
application = ProjectApplication.objects.create(
applicant=request_user,
......
......@@ -71,10 +71,10 @@ class Command(ListCommand):
dest='deleted',
default=False,
help="Also show cancelled/terminated projects"),
make_option('--base-projects',
make_option('--system-projects',
action='store_true',
default=False,
help="Also show base projects"),
help="Also show system projects"),
)
def get_owner(project):
......@@ -118,7 +118,7 @@ class Command(ListCommand):
if not options['deleted']:
self.excludes["state__in"] = Project.SKIP_STATES
if not options['base_projects']:
if not options['system_projects']:
self.excludes["is_base"] = True
if options["pending"]:
......
......@@ -95,25 +95,25 @@ def make_options():
class Command(SynnefoCommand):
args = "<project id> (or --all-base-projects)"
args = "<project id> (or --all-system-projects)"
help = "Modify an already initialized project"
option_list = SynnefoCommand.option_list + make_options() + (
make_option('--all-base-projects',
make_option('--all-system-projects',
action='store_true',
default=False,
help="Modify in bulk all initialized base projects"),
help="Modify in bulk all initialized system projects"),
make_option('--exclude',
help=("If `--all-base-projects' is given, exclude projects"
help=("If `--all-system-projects' is given, exclude projects"
" given as a list of uuids: uuid1,uuid2,uuid3")),
)
def check_args(self, args, all_base, exclude):
if all_base and args or not all_base and len(args) != 1:
m = "Please provide a project ID or --all-base-projects"
m = "Please provide a project ID or --all-system-projects"
raise CommandError(m)
if not all_base and exclude:
m = ("Option --exclude is meaningful only combined with "
" --all-base-projects.")
" --all-system-projects.")
raise CommandError(m)
def mk_all_base_filter(self, all_base, exclude):
......@@ -125,7 +125,7 @@ class Command(SynnefoCommand):
@transaction.commit_on_success
def handle(self, *args, **options):
all_base = options["all_base_projects"]
all_base = options["all_system_projects"]
exclude = options["exclude"]
self.check_args(args, all_base, exclude)
......
......@@ -36,7 +36,7 @@ class Command(ListCommand):
"service_type": ("service_type", "Service type"),
"service_origin": ("service_origin", "Service"),
"unit": ("unit", "Unit of measurement"),
"base_default": ("limit_with_unit", "Base project default quota"),
"system_default": ("limit_with_unit", "System project default quota"),
"project_default": ("project_limit_with_unit",
"Project default quota"),
"description": ("desc", "Description"),
......@@ -46,7 +46,7 @@ class Command(ListCommand):
"Resource accessibility through the UI"),
}
fields = ["id", "name", "base_default", "project_default",
fields = ["id", "name", "system_default", "project_default",
"api_visible", "ui_visible"]
def handle_args(self, *args, **options):
......
......@@ -27,12 +27,12 @@ class Command(SynnefoCommand):
help = "Modify a resource's quota defaults and boolean flags."
option_list = SynnefoCommand.option_list + (
make_option('--base-default',
make_option('--system-default',
metavar='<limit>',
help="Specify default quota for base projects"),
help="Specify default quota for system projects"),
make_option('--project-default',
metavar='<limit>',
help="Specify default quota for non-base projects"),
help="Specify default quota for non-system projects"),
make_option('--unit-style',
default='mb',
help=("Specify display unit for resource values "
......@@ -52,7 +52,7 @@ class Command(SynnefoCommand):
resource = self.get_resource(resource_name)
actions = {
'base_default': self.change_base_default,
'system_default': self.change_base_default,
'project_default': self.change_project_default,
'api_visible': self.set_api_visible,
'ui_visible': self.set_ui_visible,
......
......@@ -203,7 +203,7 @@ NOT_ALLOWED_NEXT_PARAM = 'Not allowed next parameter.'
MISSING_KEY_PARAMETER = 'Missing key parameter.'
INVALID_KEY_PARAMETER = 'Invalid key.'
DOMAIN_VALUE_ERR = 'Enter a valid domain.'
BASE_PROJECT_NAME_ERR = 'Enter a valid base project name.'
BASE_PROJECT_NAME_ERR = 'Enter a valid system project name.'
QH_SYNC_ERROR = 'Failed to get synchronized with quotaholder.'
UNIQUE_PROJECT_NAME_CONSTRAIN_ERR = (
'The project name (as specified in its application\'s definition) must '
......@@ -211,7 +211,7 @@ UNIQUE_PROJECT_NAME_CONSTRAIN_ERR = (
NOT_ALIVE_PROJECT = 'Project %s is not alive.'
NOT_SUSPENDED_PROJECT = 'Project %s is not suspended.'
NOT_TERMINATED_PROJECT = 'Project %s is not terminated.'
BASE_NO_TERMINATE = "Cannot terminate: %s is a base project."
BASE_NO_TERMINATE = "Cannot terminate: %s is a system project."
NOT_ALLOWED = 'You do not have the permissions to perform this action.'
MEMBER_NUMBER_LIMIT_REACHED = (
'You have reached the maximum number of members for this Project.')
......@@ -261,7 +261,7 @@ APPLICATION_DISMISSED = "Project application has been dismissed."
REACHED_PENDING_APPLICATION_LIMIT = ("You have reached the maximum number "
"of pending project applications: %s.")
UNINITIALIZED_NO_MODIFY = "Cannot modify: project %s is not initialized."
BASE_NO_MODIFY_FIELDS = "Cannot modify field(s) '%s' of base projects."
BASE_NO_MODIFY_FIELDS = "Cannot modify field(s) '%s' of system projects."
PENDING_APPLICATION_LIMIT_ADD = \
("You are not allowed to create a new project "
......
......@@ -45,7 +45,7 @@ class Migration(DataMigration):
owner=None,
realname=("base:" + user.uuid),
homepage="",
description=("base project for user " + user.username),
description=("system project for user " + user.username),
end_date=(datetime.datetime.now() + relativedelta(years=100)),
member_join_policy=CLOSED_POLICY,
member_leave_policy=CLOSED_POLICY,
......
......@@ -2823,7 +2823,7 @@
parse: function(resp) {
_.each(resp, function(project){
if (project.base_project) {
if (project.system_project) {
this.user_project_uuid = project.id;
}
if (project.id == synnefo.user.get_username()) {
......@@ -2838,7 +2838,7 @@
},
comparator: function(project) {
if (project.get('base_project')) { return -100 }
if (project.get('system_project')) { return -100 }
return project.get('name');
}
});
......
......@@ -930,7 +930,7 @@ class Astakos(base.Component):
def set_astakos_default_quota(self):
cmd = "snf-manage resource-modify"
return [
"%s --base-default 2 astakos.pending_app" % cmd,
"%s --system-default 2 astakos.pending_app" % cmd,
"%s --project-default 0 astakos.pending_app" % cmd,
]
......@@ -938,14 +938,14 @@ class Astakos(base.Component):
def set_cyclades_default_quota(self):
cmd = "snf-manage resource-modify"
return [
"%s --base-default 4 cyclades.vm" % cmd,
"%s --base-default 40G cyclades.disk" % cmd,
"%s --base-default 16G cyclades.total_ram" % cmd,
"%s --base-default 8G cyclades.ram" % cmd,
"%s --base-default 32 cyclades.total_cpu" % cmd,
"%s --base-default 16 cyclades.cpu" % cmd,
"%s --base-default 4 cyclades.network.private" % cmd,
"%s --base-default 4 cyclades.floating_ip" % cmd,
"%s --system-default 4 cyclades.vm" % cmd,
"%s --system-default 40G cyclades.disk" % cmd,
"%s --system-default 16G cyclades.total_ram" % cmd,
"%s --system-default 8G cyclades.ram" % cmd,
"%s --system-default 32 cyclades.total_cpu" % cmd,
"%s --system-default 16 cyclades.cpu" % cmd,
"%s --system-default 4 cyclades.network.private" % cmd,
"%s --system-default 4 cyclades.floating_ip" % cmd,
"%s --project-default 0 cyclades.vm" % cmd,
"%s --project-default 0 cyclades.disk" % cmd,
"%s --project-default inf cyclades.total_ram" % cmd,
......@@ -960,13 +960,13 @@ class Astakos(base.Component):
def set_pithos_default_quota(self):
cmd = "snf-manage resource-modify"
return [
"%s --base-default 40G pithos.diskspace" % cmd,
"%s --system-default 40G pithos.diskspace" % cmd,
"%s --project-default 0 pithos.diskspace" % cmd,
]
@base.run_cmds
def modify_all_quota(self):
cmd = "snf-manage project-modify --all-base-projects --limit"
cmd = "snf-manage project-modify --all-system-projects --limit"
return [
"%s pithos.diskspace 40G 40G" % cmd,
"%s astakos.pending_app 2 2" % cmd,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment