Commit 3b0bc8bc authored by Nikos Skalkotos's avatar Nikos Skalkotos

Merge branch 'release-0.14'

parents 58fef81d 31a37bd6
Copyright 2011-2014 GRNET S.A. All rights reserved.
Copyright 2011-2016 GRNET S.A. All rights reserved.
Redistribution and use in source and binary forms, with or
without modification, are permitted provided that the following
......
.. _Changelog:
Unified Changelog file for Kamaki versions >= 0.13
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Changelog file for Kamaki versions >= 0.13
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. _Changelog-0.14:
v0.14
=====
Bugfixes
--------
* Fix handling of astakosclient errors
Features
--------
* Rename "wait_*" kamaki.clients.Client methods to
"wait_until" and "wait_while"
* Implement "wait" commands in CLI, where applicable
* Implement "wait" functionality for volumes
* Print public url in "kamaki file upload --public"
* Show HTTP data (body) with a single runtime argument "-vv"
* Document config options and show descriptions in
"kamaki config list"
* Modify some help messages (-c, -o, HTTP log separators) for clarity
* Show image file hashmap in "kamaki image info", with "--hashmap"
* Implement compute detachable volumes (API lib and CLI)
* Adjust BlockStorage client API and CLI commands to comply with
Synnefo 0.16next API
.. _Changelog-0.13:
......
FROM debian
# Install Python Setuptools
RUN apt-get update && apt-get install -y python-pip ca-certificates --no-install-recommends
# Bundle app source
ADD . /src
# Install test requirements
RUN pip install mock ansicolors
# Initialize app environment
WORKDIR /src
RUN python setup.py install
......@@ -22,7 +22,7 @@ Please see the [official Synnefo site](http://www.synnefo.org) and the
Copyright and license
=====================
Copyright (C) 2011-2014 GRNET S.A. All rights reserved.
Copyright (C) 2011-2016 GRNET S.A. All rights reserved.
Redistribution and use in source and binary forms, with or
without modification, are permitted provided that the following
......
#!/usr/bin/env sh
set -e
python kamaki/cli/test.py
python kamaki/clients/test.py
......@@ -216,6 +216,10 @@ server (Compute/Cyclades)
delete Delete a virtual server
console Create a VNC console and show connection information
wait Wait for server to finish [BUILD, STOPPED, REBOOT, ACTIVE]
attachment Details on a volume attachment
attachments List of all volume attachments for a server
attach Attach a volume on a server
detach Delete an attachment/detach a volume from a server
Showcase: Create a server
^^^^^^^^^^^^^^^^^^^^^^^^^
......
# -*- coding: utf-8 -*-
#
# Copyright 2011-2013 GRNET S.A. All rights reserved.
# Copyright 2011-2016 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
......@@ -123,13 +123,13 @@ master_doc = 'index'
# General information about the project.
project = u'Kamaki'
copyright = u'2014, GRNET'
copyright = u'2016, GRNET'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = '0.13'
version = '0.14'
# The full version, including alpha/beta/rc tags.
try:
......@@ -325,6 +325,7 @@ latex_documents = [
# (source start file, name, description, authors, manual section).
man_pages = [
('man/kamaki', 'kamaki', 'Command-line tool for managing clouds', '', 1),
('man/kamaki-shell', 'kamaki-shell', 'Interactive shell for managing clouds', '', 1),
]
# If true, show URL addresses after external links.
......
:orphan:
kamaki shell manual page
========================
Synopsis
--------
**kamaki-shell** [*group*] [*command*] [...] [*arguments*]
Description
-----------
:program:`kamaki` is a simple, yet intuitive, command-line tool for managing
clouds. It can be used in three forms: as an interactive shell
(`kamaki-shell`), as a command line tool (`kamaki`) or as a clients API for
other applications (`kamaki.clients`).
Launch options
--------------
.. code-block:: console
-v Verbose output, without HTTP data
-vv Verbose output, including HTTP data
-d Use debug output.
-o KEY=VAL Override a config value (can be repeated)
--cloud CLOUD Cloud to be used for this shell session
Commands
--------
:manpage: `kamaki(1)`
Shell Management Commands
*************************
exit
Exit the interactive shell or a command namespace inside the shell
shell
Execute commands on host system shell (e.g. bash)
Kamaki and API commands
***********************
The Kamaki and API commands are the same in both the CLI and the shell. For a
complete list of the common commands, check the "COMMANDS" section at the
following manpage:
:manpage: `kamaki(1)`
Author
------
Synnefo development team <synnefo-devel@googlegroups.com>.
This diff is collapsed.
This diff is collapsed.
......@@ -209,6 +209,10 @@ command group (server) and of a command in that group (list).
start: Start an existing virtual server
shutdown: Shutdown an active virtual server
delete: Delete a virtual server
attachment: Details on a volume attachment
attachments: List of all volume attachments for a server
attach: Attach a volume on a server
detach: Delete an attachment/detach a volume from a server
.. code-block:: console
:emphasize-lines: 1,2
......
# Copyright 2012-2014 GRNET S.A. All rights reserved.
# Copyright 2012-2015 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
......@@ -33,6 +33,7 @@
import logging
from sys import argv, exit, stdout, stderr
import os
from os.path import basename, exists
from inspect import getargspec
......@@ -51,6 +52,7 @@ from kamaki.clients.utils import https, escape_ctrl_chars
_debug = False
kloger = None
DEF_CLOUD_ENV = 'KAMAKI_DEFAULT_CLOUD'
# command auxiliary methods
......@@ -165,7 +167,7 @@ cmd_spec_locations = [
# Generic init auxiliary functions
def _setup_logging(debug=False, verbose=False):
def _setup_logging(debug=False, verbose=False, _verbose_with_data=False):
"""handle logging for clients package"""
sfmt, rfmt = '> %(message)s', '< %(message)s'
......@@ -180,6 +182,9 @@ def _setup_logging(debug=False, verbose=False):
logger.add_stream_logger(__name__, DEBUGV)
# else:
# logger.add_stream_logger(__name__, logging.WARNING)
if _verbose_with_data:
from kamaki import clients
clients.Client.LOG_DATA = True
global kloger
kloger = logger.get_logger(__name__)
......@@ -221,10 +226,11 @@ def _init_session(arguments, is_non_api=False):
_help = arguments['help'].value
global _debug
_debug = arguments['debug'].value
_verbose = arguments['verbose'].value
_verbose_with_data = arguments['verbose_with_data'].value
_verbose = arguments['verbose'].value or _verbose_with_data
_cnf = arguments['config']
_setup_logging(_debug, _verbose)
_setup_logging(_debug, _verbose, _verbose_with_data)
if _help or is_non_api:
return None
......@@ -258,7 +264,7 @@ def _init_session(arguments, is_non_api=False):
remove_colors()
cloud = arguments['cloud'].value or _cnf.value.get(
'global', 'default_cloud')
'global', 'default_cloud') or os.environ.get(DEF_CLOUD_ENV)
if not cloud:
num_of_clouds = len(_cnf.value.keys('cloud'))
if num_of_clouds == 1:
......@@ -274,6 +280,7 @@ def _init_session(arguments, is_non_api=False):
' kamaki config get cloud.<cloud name>',
'To set a default cloud:',
' kamaki config set default_cloud <cloud name>',
' or set the %s enviroment variable' % DEF_CLOUD_ENV,
'To pick a cloud for the current session, use --cloud:',
' kamaki --cloud=<cloud name> ...'])
if cloud not in _cnf.value.keys('cloud'):
......@@ -541,7 +548,7 @@ def main(func):
logger.add_stream_logger(
__name__, logging.WARNING,
fmt='%(levelname)s (%(name)s): %(message)s')
_config_arg = ConfigArgument('Path to config file')
_config_arg = ConfigArgument('Path to a custom config file')
parser = ArgumentParseManager(exe, arguments=dict(
config=_config_arg,
cloud=ValueArgument(
......@@ -549,12 +556,17 @@ def main(func):
help=Argument(0, 'Show help message', ('-h', '--help')),
debug=FlagArgument('Include debug output', ('-d', '--debug')),
verbose=FlagArgument(
'More info at response', ('-v', '--verbose')),
'Show HTTP requests and responses, without HTTP body',
('-v', '--verbose')),
verbose_with_data=FlagArgument(
'Show HTTP requests and responses, including HTTP body',
('-vv', '--verbose-with-data')),
version=VersionArgument(
'Print current version', ('-V', '--version')),
options=RuntimeConfigArgument(
_config_arg,
'Override a config value', ('-o', '--options')),
'Override a config option (not persistent)',
('-o', '--options')),
ignore_ssl=FlagArgument(
'Allow connections to SSL sites without certs',
('-k', '--ignore-ssl', '--insecure')),
......
# Copyright 2012-2014 GRNET S.A. All rights reserved.
# Copyright 2012-2015 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
......@@ -480,7 +480,8 @@ class StatusArgument(ValueArgument):
First state is the default"""
def __init__(self, *args, **kwargs):
self.valid_states = kwargs.pop('valid_states', ['BUILD', ])
self.valid_states = [
s.upper() for s in kwargs.pop('valid_states', ['BUILD', ])]
super(StatusArgument, self).__init__(*args, **kwargs)
@property
......@@ -551,6 +552,63 @@ class ProgressBarArgument(FlagArgument):
mybar.finish()
class PithosLocationArgument(ValueArgument):
"""Resolve pithos URI, return in the form pithos://uuid/container[/object]
UPDATE: URLs without an object are also resolvable. Therefore, caller
methods should check if there is an object or not
"""
def __init__(
self, help=None, parsed_name=None, default=None, user_uuid=None):
super(PithosLocationArgument, self).__init__(
help=help, parsed_name=parsed_name, default=default)
self.uuid, self.container, self.object = user_uuid, None, None
def setdefault(self, term, value):
if not getattr(self, term, None):
setattr(self, term, value)
@property
def dict(self):
""":returns: location as {user_uuid: .., container: .., object: ..}"""
return dict(
user_uuid=self.uuid, container=self.container, object=self.object)
@property
def tuple(self):
"""returns: location as (user_uuid, container, object)"""
return (self.uuid, self.container, self.object)
@property
def value(self):
object_ = ('/%s' % self.object) if self.object else ''
return 'pithos://%s/%s%s' % (self.uuid, self.container, object_)
@value.setter
def value(self, location):
if location:
from kamaki.cli.cmds.pithos import _PithosContainer as pc
try:
uuid, self.container, self.object = pc.resolve_pithos_url(
location)
self.uuid = uuid or self.uuid
assert self.container, 'No container in pithos URI'
except Exception as e:
raise CLIInvalidArgument(
'Invalid Pithos+ location %s (%s)' % (location, e),
details=[
'The image location must be a valid Pithos+',
'location. There are two valid formats:',
' pithos://USER_UUID/CONTAINER[/OBJECT]',
'OR',
' /CONTAINER/OBJECT]',
'To see all containers:',
' [kamaki] container list',
'To list the contents of a container:',
' [kamaki] container list CONTAINER'])
# Initial command line interface arguments
......
# Copyright 2013-2014 GRNET S.A. All rights reserved.
# Copyright 2013-2016 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
......@@ -221,7 +221,7 @@ class FlagArgument(TestCase):
help, pname, default = 'help', 'pname', 'default'
fa = argument.FlagArgument(help, pname, default)
self.assertTrue(isinstance(fa, argument.FlagArgument))
arg.assert_called_once(0, help, pname, default)
arg.assert_called_once_with(0, help, pname, default)
class ValueArgument(TestCase):
......@@ -231,7 +231,7 @@ class ValueArgument(TestCase):
help, pname, default = 'help', 'pname', 'default'
fa = argument.ValueArgument(help, pname, default)
self.assertTrue(isinstance(fa, argument.ValueArgument))
arg.assert_called_once(1, help, pname, default)
arg.assert_called_once_with(1, help, pname, default)
class IntArgument(TestCase):
......@@ -368,7 +368,7 @@ class ProgressBarArgument(TestCase):
pba = argument.ProgressBarArgument(help, pname, default)
self.assertTrue(isinstance(pba, argument.ProgressBarArgument))
self.assertEqual(pba.suffix, '%(percent)d%%')
init.assert_called_once(help, pname, default)
init.assert_called_once_with(help, pname, default)
def test_clone(self):
pba = argument.ProgressBarArgument(parsed_name='--progress')
......@@ -392,7 +392,7 @@ class ProgressBarArgument(TestCase):
self.assertEqual(pba.bar.message, '%s%s' % (
msg, ' ' * (msg_len - len(msg))))
self.assertEqual(pba.bar.suffix, '%(percent)d%% - %(eta)ds')
start.assert_called_once()
assert start.call_count == 1
pba.get_generator(msg, msg_len, countdown=True)
self.assertTrue(
......@@ -414,7 +414,7 @@ class ProgressBarArgument(TestCase):
pba.bar = argument.KamakiProgressBar()
with patch('%s.KamakiProgressBar.finish' % arg_path) as finish:
pba.finish()
finish.assert_called_once()
assert finish.call_count == 1
class ArgumentParseManager(TestCase):
......@@ -440,7 +440,7 @@ class ArgumentParseManager(TestCase):
self.assertEqual(apm._unparsed, None)
self.assertEqual(parse.mock_calls[-1], call())
if arguments:
update_parser.assert_called_once()
assert update_parser.call_count == 2
def test_syntax(self):
apm = argument.ArgumentParseManager('exe', {})
......@@ -452,7 +452,7 @@ class ArgumentParseManager(TestCase):
@patch('%s.ArgumentParseManager.update_parser' % arg_path)
def test_arguments(self, update_parser):
apm = argument.ArgumentParseManager('exe', {})
update_parser.assert_called_once()
assert update_parser.call_count == 1
exp = {'k1': 'v1', 'k2': 'v2'}
apm.arguments = exp
assert_dicts_are_equal(self, apm.arguments, exp)
......
# Copyright 2011-2014 GRNET S.A. All rights reserved.
# Copyright 2011-2015 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
......@@ -31,7 +31,7 @@
# interpreted as representing official policies, either expressed
# or implied, of GRNET S.A.command
from sys import stdin, stdout, stderr
from sys import stdin, stdout, stderr, exit
from traceback import format_exc
from kamaki.cli.logger import get_logger
......@@ -58,6 +58,7 @@ def dont_raise(*errs):
('%s=%s' % items) for items in kwargs.items()])))
log.debug(format_exc(e))
return None
wrap.__name__ = func.__name__
return wrap
return decorator
......@@ -68,6 +69,7 @@ def client_log(func):
return func(self, *args, **kwargs)
finally:
self._set_log_params()
wrap.__name__ = func.__name__
return wrap
......@@ -80,6 +82,7 @@ def fall_back(func):
log.warning('Kamaki will use original data to go on')
finally:
return inp
wrap.__name__ = func.__name__
return wrap
......@@ -199,22 +202,22 @@ class CommandInit(object):
return self._usernames2uuids([username]).get(username, None)
def _set_log_params(self):
if not self.client:
if not getattr(self, 'client', None):
return
try:
self.client.LOG_TOKEN = (
self.client.LOG_TOKEN = self.client.LOG_TOKEN or (
self['config'].get('global', 'log_token').lower() == 'on')
except Exception as e:
log.debug('Failed to read custom log_token setting:'
'%s\n default for log_token is off' % e)
try:
self.client.LOG_DATA = (
self.client.LOG_DATA = self.client.LOG_DATA or (
self['config'].get('global', 'log_data').lower() == 'on')
except Exception as e:
log.debug('Failed to read custom log_data setting:'
'%s\n default for log_data is off' % e)
try:
self.client.LOG_PID = (
self.client.LOG_PID = self.client.LOG_PID or (
self['config'].get('global', 'log_pid').lower() == 'on')
except Exception as e:
log.debug('Failed to read custom log_pid setting:'
......@@ -388,22 +391,33 @@ class Wait(object):
)
def wait(
self, service, service_id, status_method, current_status,
countdown=True, timeout=60):
(progress_bar, wait_cb) = self._safe_progress_bar(
'%s %s: status is still %s' % (
service, service_id, current_status),
self, service, service_id, status_method, status,
countdown=True, timeout=60, msg='still', update_cb=None):
(progress_bar, wait_gen) = self._safe_progress_bar(
'%s %s status %s %s' % (service, service_id, msg, status),
countdown=countdown, timeout=timeout)
wait_step = None
if wait_gen:
wait_step = wait_gen(timeout)
wait_step.next()
def wait_cb(item_details):
if wait_step:
if update_cb:
progress_bar.bar.goto(update_cb(item_details))
else:
wait_step.next()
try:
new_mode = status_method(
service_id, current_status, max_wait=timeout, wait_cb=wait_cb)
if new_mode:
self.error('%s %s: status is now %s' % (
service, service_id, new_mode))
item_details = status_method(
service_id, status, max_wait=timeout, wait_cb=wait_cb)
if item_details:
self.error('')
self.error('%s %s status: %s' % (
service, service_id, item_details['status']))
else:
self.error('%s %s: status is still %s' % (
service, service_id, current_status))
exit("Operation timed out")
except KeyboardInterrupt:
self.error('\n- canceled')
self.error(' - canceled')
finally:
self._safe_progress_bar_finish(progress_bar)
......@@ -654,15 +654,15 @@ class endpoint_list(_AstakosInit, OptionalOutput, NameFilter):
_project_specs = """
{
"name": name,
"owner": uuid, # if omitted, request user assumed
"name": name.in.domainlike.format,
"owner": user-uuid, # if omitted, request user assumed
"homepage": homepage, # optional
"description": description, # optional
"comments": comments, # optional
"max_members": max_members, # optional
"private": true | false, # optional
"start_date": date, # optional
"end_date": date,
"start_date": date, # optional - in ISO8601 format
"end_date": date, # in ISO8601 format e.g., YYYY-MM-DDThh:mm:ssZ
"join_policy": "auto" | "moderated" | "closed", # default: "moderated"
"leave_policy": "auto" | "moderated" | "closed", # default: "auto"
"resources": {
......
# Copyright 2014 GRNET S.A. All rights reserved.
# Copyright 2014-2015 GRNET S.A. All rights reserved.
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
......@@ -32,10 +32,11 @@
# or implied, of GRNET S.A.command
from kamaki.cli import command
from kamaki.cli.errors import CLIInvalidArgument
from kamaki.cli.cmdtree import CommandTree
from kamaki.cli.cmds import (
CommandInit, errors, client_log, OptionalOutput)
from kamaki.clients.cyclades import CycladesBlockStorageClient
CommandInit, errors, client_log, OptionalOutput, Wait)
from kamaki.clients.cyclades import CycladesBlockStorageClient, ClientError
from kamaki.cli import argument
......@@ -44,6 +45,24 @@ snapshot_cmds = CommandTree('snapshot', 'Block Storage API snapshot commands')
namespaces = [volume_cmds, snapshot_cmds]
_commands = namespaces
volume_states = (
'creating', 'available', 'attaching', 'detaching', 'in_use', 'deleting',
'deleted', 'error', 'error_deleting', 'backing_up', 'restoring_backup',
'error_restoring', )
class _VolumeWait(Wait):
def wait_while(self, volume_id, current_status, timeout=60):
super(_VolumeWait, self).wait(
'Volume', volume_id, self.client.wait_volume_while, current_status,
timeout=timeout)
def wait_until(self, volume_id, target_status, timeout=60):
super(_VolumeWait, self).wait(
'Volume', volume_id, self.client.wait_volume_until, target_status,
timeout=timeout, msg='not yet')
class _BlockStorageInit(CommandInit):
@errors.Generic.all
......@@ -88,49 +107,65 @@ class volume_info(_BlockStorageInit, OptionalOutput):
@command(volume_cmds)
class volume_create(_BlockStorageInit, OptionalOutput):
class volume_create(_BlockStorageInit, OptionalOutput, _VolumeWait):
"""Create a new volume"""
arguments = dict(
size=argument.IntArgument('Volume size in GB', '--size'),
server_id=argument.ValueArgument(
'The server for the new volume', '--server-id'),
'The server to attach the volume to', '--server-id'),
name=argument.ValueArgument('Display name', '--name'),
# src_volume_id=argument.ValueArgument(
# 'Associate another volume to the new volume',
# '--source-volume-id'),
description=argument.ValueArgument(
'Volume description', '--description'),
snapshot_id=argument.ValueArgument(
'Associate a snapshot to the new volume', '--snapshot-id'),
image_id=argument.ValueArgument(
'Associate an image to the new volume', '--image-id'),
volume_type=argument.ValueArgument('Volume type', '--volume-type'),
volume_type=argument.ValueArgument(
'default: if combined with --server-id, the default is '
'automatically configured to match the server, otherwise it is '
'ext_archipelago',
'--volume-type'),
metadata=argument.KeyValueArgument(
'Metadata of key=value form (can be repeated)', '--metadata'),
project_id=argument.ValueArgument(
'Assign volume to a project', '--project-id'),
wait=argument.FlagArgument(
'Wait volume to be created and ready for use', ('-w', '--wait')),
)
required = ('size', 'server_id', 'name')