Reviewed-by: imsnah

os.path.join does not like None as argument and fails with
AttributeError: 'NoneType' object has no attribute 'startswith'.

This patch makes sure the passed argument is a string in any case.

Reviewed-by: iustinp

Reviewed-by: imsnah

Reviewed-by: ultrotter, iustinp

This patch does two things:
- Modify LURenameInstance.Exec to rename directory
  when a file-based instance is renamed
- Modify config.RenameInstance() to replace the directory name in
  config.data for file devices

Reviewed-by: iustinp

- Modfiy _GenerateDiskTemplate to support file-based disk template
- Modify _CreateDisks to create directory needed for file-based
  instances before creating the actual files
- Modify _RemoveDisks to delete directory for file-based instances
  after deleting their VBDs
- Add Prereq-check to check if given file-driver is valid
- Add Prereq-check to check if given file-storage-dir path is relative

Reviewed-by: iustinp

Having the individual parts in the code allows us to build version
numbers like "1.2" while leaving "3" out in a clean fashion, that is
without regular expressions or the like. This might be used together
with configuration format versions.

Why m4 code? AM_INIT_AUTOMAKE, which could take a shell variable, is
considered deprecated[1] and should be replaced by AC_INIT. Unfortunately,
AC_INIT is expanded at build time, so one has to use m4 to build
composite values like this version number[2].

[1] http://www.gnu.org/software/libtool/manual/automake/Public-macros.html
[2] http://www.mail-archive.com/autoconf@gnu.org/msg16720.html

Reviewed-by: iustinp

The driver in the xen config file needs to be changed when dealing with
files rather then bdevs.

This patch does two things:
- Add _GetConfigFileDiskData to XenHypervisor which returns the correct
  disk xen config line. This function checks the logical disk type of
  every given block device, such that also hybrid (e.g. mixed drbd and
  file VBDs) are possible
- Make Xen[Pvm|Hvm]Hypervisor._WriteConfigFile() a classmethod to be
  able to call the helper function _GetConfigFileDiskData() in their
  parent XenHypervisor

Reviewed-by: iustinp

This patch implements query-ing of only selected jobs instead of all.

Reviewed-by: ultrotter

The idea of cli.py and luxi.py is that all protocol checks should be in
luxi, and cli.py should just offer some helpful shortcuts for the
command line scripts.

This patch removes the result checks from cli and adds some other checks
to luxi. It does no longer check the success/failure since it's not yet
clear how that should be handled - probably exceptions.

Reviewed-by: ultrotter

This small patch fixed the opcodes.OpCode.LoadOpcode capitalization to
what was intented to be (as the comment says): LoadOpCode.

Reviewed-by: ultrotter

This patch adds checks for the master role and daemonize support to
ganeti-masterd.

The patch modifies the startup/shutdown of the server because:
  - we want bind()/listen() to the master socket to occur before forking
    so that we can return a correct exit code and write messages to
    stderr
  - but we want thread startup to occur after fork(), otherwise python
    threading gets confused

The patch also has some small cleanups:
  - remove the unix socket after closing it, so we don't need to remove
    it manually
  - instead of just telling the threads to terminate via the new_queue,
    we also join() them so that the logs show what thread clinging to
    life
  - the daemon logs to its own logfile now
  - there is command line parameter support :)

Reviewed-by: imsnah

This is the representation of file VBDs on the backend. It's the first
implementation an supports only raw files.

Reviewed-by: iustinp

Reviewed-by: ultrotter

Add _[Create,Remove,Rename]FileStorageDir function which are needed for
file-based instance management. These function check whether the given
directory to operate on is under the cluster-wide defined default file
storage dir. If this is not the case the won't do anything and return
False. This is to prevent cluster manipulation or damage.

Reviewed-by: ultrotter

Reviewed-by: imsnah

This patch implements an optional parameter to utils.Daemonize() which
allows that function to not close some file descriptors.

This will allow the master daemon to open the listening socket before
fork - in order to be able to notify errors and return a meaningful exit
code, and then when we fork we don't close that fd.

Reviewed-by: imsnah

This patch adds a very basic gnt-job script that allows job querying.
This goes on top of the previous master daemon patches.

Currently, because of the not-changed cmd lock, you can't query the jobs
as long as a job is running - you have to rm the cmd lock and then you
can query the jobs.

Reviewed-by: imsnah

Currently, in ganeti-noded we have the createDaemon function. Since
we'll need the same in other daemons, we move this function to utils.py

With the move, a few changes were also done:
  - change the name to Daemonize()
  - add a parameter, logfile, as different daemons will want to log to
    different files
  - remove the try.. except.. around the fork calls, since they were
    only re-raising the OS exception with less data; unless we want to
    actually handle fork error (not just re-raising), these try blocks
    are not useful
  - change the return style at the end of the function

Reviewed-by: imsnah

This new version includes all the possible failure options.

Reviewed-by: iustinp

_CheckDiskConsistency outputs "Can't get any data from node NODE" when no drbd
is found on the target node. This causes a misleading error message to be
output for example on failover (when the primary node is down, or the instance
is not running), stating that no data could be got from the secondary node,
which scares the user and misleads him. Changing this to "Disk degraded or not
found on node %s" still reports that something is missing, but on the other
hand doesn't make the user think the node is down, or has no data at all...

Reviewed-by: imsnah

Right now if you try to failover an instance which is not marked as up the
operation will fail unless you pass the --ignore-consistency flag because the
disks won't be considered to be consistent. Allow them to be if we know the
instance shouldn't be up.

Reviewed-by: imsnah

Currently gnt-backup export chains the ShutdownInstance and StartupInstance
opcodes to itself. This works but (a) it's suboptimal, because there's no need
to deactivate the instance's disks as we are about to restart it anyway, and
(b) doesn't take care of instances which are already down (and should be). This
patch takes care of this by just calling the shutdown rpc function instead of
the whole opcode, and just starting up the instance if it's configured as up in
the first place.

Reviewed-by: imsnah

ConfigParser.SafeConfigParser doesn't support unicode string objects.
Unicode string objects are returned by simplejson.

Reviewed-by: iustinp

gnt-instance failover on an instance marked as down will mistakenly bring it
up. The watcher will then shut it down again, but it's a lot better (and safer)
not to start it at all.

Reviewed-by: imsnah

Reviewed-by: imsnah

Reviewed-by: imsnah

Reviewed-by: ultrotter

This patch adds function that submit jobs or queries over the unix socket
interface to lib/cli.py. The will be used by the scripts instead of the
SubmitOpCode function.

Reviewed-by: ultrotter

This patch adds a very dumb in-memory only queue implementation.

Reviewed-by: imsnah

This patch adds an initial implementation of the client-side of the unix
socket interface. The code is not yet used by other parts of ganeti.

Reviewed-by: imsnah

This patch makes the command execute last on the master (if the master
is selected). The order for the other nodes is unchanged.

The patch also updates the man page with some explanations and an
example.

Reviewed-by: imsnah

- Substitute all occurences of name 'parms' with 'params'
- Small codestyle fix

Reviewed-by: ultrotter

Reviewed-by: iustinp

Reviewed-by: iustinp

This patch does two things:
- Remove "vg_name" from _OP_REQP due to the introduction of
  --no-lvm-storage. Since vg_name option has as default now None and is
  only set to the DEFAULT_VG if lvm_storage is enabled, this is needed
- It changes LUInitCluster.CheckPrereq() to skip _HasValidVG check
  initializing the cluster with --no-lvm-storage. Furthermore it adds to
  the help message the statement of the possibility to use
  --no-lvm-storage if no 'xenvg' volume group is found.

Reviewed-by: iustinp

Add LUSetClusterParams, which is the LU to modify cluster options.
This includes checks:
- not to disable lvm storage when it's already disabled
- not to enable lvm storage when it is already enabled
- not to disable lvm when lvm-based instances are present
- that the specified volume group is valid on all cluster-nodes
  when lvm-storage is going to be enabled

Reviewed-by: iustinp

Reviewed-by: iustinp

This adds a check to fail instance creation if lvm-storage is disabled
(cluster-wide). If lvm-storage is disabled (by initializing the cluster
with --no-lvm-storage) only instances with disk template in frozenset
DTS_NOT_LVM are allowed to create.

Reviewed-by: iustinp

DTS_NOT_LVM:
This constant is needed when checking if an instance can be created with
the given disk template if no lvm-storage is available, i.e. the ganeti
cluster does not have a volume group

DEFAULT_VG:
'xenvg' has been hardcoded before.

Reviewed-by: iustinp