This replaces the field 'vg_names' in the RPC call of 'node info' by
'storage_units'. A storage unit is a tuple <storage_type,key>
and a generalization of a vg_name. The list of vg names is replaced by
a list of storage units. The modified RPC call will be used to report
storage space for more than just lvm volume groups. What the 'key' is
depends on the storage type. For storage type lvm-vg, the key is the
volume group name. To keep backward compatibility, all functions that
use the old vg_names, convert them to a list where every volume group
is mapped to a tuple [('lvm-vg',volume_group)] before making the call.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

* stable-2.8: (45 commits)
  Update NEWS with disk creation fixes
  Sort cmdlib-related entries in Makefile.am
  cmdlib: Cleanup public/private functions
  cmdlib: Extract instance query related functionality
  cmdlib: Extract instance operation functionality
  cmdlib: Extract migration related functionality
  cmdlib: Extract storage related functionality
  Reformat and define exports in cmdlib/__init__.py
  Extract miscellaneous logical units from cmdlib
  Extract os related logical units from cmdlib
  Extract query related logical units from cmdlib
  Extract backup related logical units from cmdlib
  Extract instance related logical units from cmdlib
  Extract node related logical units from cmdlib
  Extract group related logial units from cmdlib
  Extract cluster related logical units from cmdlib
  Extract test logical units from cmdlib
  Extract network related logical units from cmdlib
  Extract tags related logical units from cmdlib
  Extract base classes from cmdlib
  ...

Conflicts:
	devel/build_chroot
	lib/cmdlib.py

devel/build_chroot is straightforward: one side has added versions, the
other has added one library. lib/cmdlib.py has been split in many files in
stable-2.8, so I've semi-manually applied the changes from master.

This merge also fixes a problem with merge f2d87a5e, which partially
reverted changes from 912737ba by mistake.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

* stable-2.7:
  Update NEWS with disk creation fixes
  Don't fail to deactivate master IP if already down
  Add QA for recreating single instance disks
  Add QA for gnt-instance modify --disk
  Clean up when "gnt-instance modify" fails to create a disk
  recreate-disks honors the prealloc_wipe_disks flag
  Introduce wrapper for cmdlib._WipeDisks()
  Don't catch an exception that cannot be raised
  Wipe disks added through "gnt-instance modify"
  Support /var/run being a symlink in upload
  Final NEWS and configure.ac update for 2.7.0~rc1
  gnt-job list: deal with non-ascii encoding in jobs

Conflicts:
	NEWS
	lib/cmdlib.py
	qa/ganeti-qa.py
	qa/qa-sample.json

NEWS, qa/ganeti-qa.py and qa/qa-sample.py had trivial conflicts. But I've
updated QA changes to use the new interfaces. lib/cmdlib.py was renamed and
split, so I had to semi-manually apply the changes to the new files. I had
to change the names of some functions by removing or adding the initial
underscore and update the imported names.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Also document a couple more fixes.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Files in the cmdlib directory are sorted alphabetically in
Makefile.am.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All functions/classes which are used outside of their defining module
(with tests as an exception) no longer have a leading underscore.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Split instance.py further by extracting instance querying related
logical units and functions to instance_query.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Split instance.py further by extracting instance operations
(start/stop/reboot/etc.) related logical units and functions to
instance_operation.py.

The extracted operations have in common that they affect the operating
system in a running instance directly.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Split instance.py further by extracting migration related logical units
and functions to instance_migration.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Split instance.py further by extracting storage related logical units
and functions to instance_storage.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

cmdlib/__init__.py now simply defines the interface of the cmdlib module
by importing all classes which should be visible to clients.

Also don't ignore C0302 (Too many lines in module) any more.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All remaining classes in __init__.py are extracted to misc.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUOs* classes are extracted to operating_system.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUQuery* classes are extracted to query.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUBackup* classes are extracted to backup.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUInstance* classes are extracted to instance.py. Common functions
are moved to common.py if used by non-instance logical units as well.
Additionally, helper functions which are only used by LUBackup* and
LUInstance* are moved to instance_utils.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUNode* classes are extracted to node.py. Common functions are moved
to common.py if used by non-node logical units as well.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUGroup* classes are moved to group.py. Common functions are
extracted to common.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

All LUCluster* classes are extracted to cluster.py. Shared functions are
extracted to common.py, helper functions only used by LUCluster* are
extracted to cluster.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

LUTest* are moved to test.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

LUNetwork* and associated helper functions are extracted to network.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

LUTags* and their base class, TagsLU, are extracted to tags.py. An
additional shared function, _ShareAll, is extracted to common.py for
shared usage.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Base classes holding common functionality is extracted into base.py.
Utility functions used by both base classes and subclasses is moved to
common.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

The master IP setup script now checks if the master IP is actually
configured on the machine before trying to remove the IP.

This fixes issue 460.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch introduces a test to check the compatibility
of the Haskell and the Python representation of instances.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch introduces and enhances generators for
instances:
- 'genInstWithNets' is split into the generation of an
  arbitrary instance and enhancing an instance with nets
- 'genInst' calls 'genInstWithNets' with an empty set
  of initial networks to provide a reasonable default
- the Arbitrary instance of 'Instance' uses now 'genDisks'
  to create instances with a reasonable set of disks

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The Arbitrary instance of the 'Instance' object is written
using the <*> syntax. Since it often uses the 'arbitrary'
generator for the instance's fields it is hard to figure
out which 'arbitrary' fills which instance field. This
patch annotates all fields with their name to make
maintenance of this code easier.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This patch adds generators for Disk instances to the Haskell
test code. It uses somewhat more reasonable generators to
fill the fields instead of just arbitrary values.
'genDiskWithChildren' is a generator that generates a disk
with a specified number of disk children. To avoid shooting
ourselves in the foot we do not generate further (grand)
child disks for the child disks. 'genDisk' calls
'genDiskWithChildren' by requesting three children as a
resonable default.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This simplifies my previous commit (820bade9) by using os.statvfs
instead of parsing the output of 'df'.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

In a previous commit, I accidentially changed the order of
unit tests in Makefile.am to not be alphabetically anymore.
This fixes it.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This adds functionality to retrieve disk space information
for file storage. It calls the 'df' tool and parses its
output to extract the total and free amount of disk space
on the disk where the given path is located.
The code is not integrated yet, but thoroughly unit-tested.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Bernardo dal Seno <bdalseno@google.com>

Also, properly set the date of the last modification.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The Haskell ConfD client was assuming internet addresses to be IPv4. This
patch modifies the client so that it is able to automatically detect the
protocol it should use by analyzing the address it is told to connect to.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This function can be useful to many parts of the code to convert the string
representation of an IP (v4 or v6) address into the proper data type.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

The monitoring daemon should always be alive, therefore it's added to the
watcher.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add the monitoring daemon to the command starting the node daemon, given that
they both have to be started on all nodes.

Note that daemon-util only supports starting one daemon at the time, so the
actual command has to be composed as a sequence of two different daemon-util
invocations.

Also, the monitoring daemon invocation is conditional, depending on whether it
was enabled at configure time.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

So far QA only recreated the whole set of disks at once.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Just a very basic test that adds and then removes a disk.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

cmdlib.LUInstanceSetParams now uses helper functions to create and wipe
disks, so that when the creation of a disk fails, any leftover device is
cleaned up. As a bonus, exceptions raised by _CreateBlockDev() are caught
correctly.

Now cmdlib._CreateDisks() is used every time there are disks to create.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>