GitLab

This patch modifies the watcher to keep on-disk a file with the instance
status; this can be used from outside of ganeti to react to instances
being down (when the watcher cannot restart them).
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch adds constants for the mouse and boot order strings; while
there are still some issues remaining, we're trying to cleanup hardcoded
strings from the hypervisors.

Since the formatting of frozensets is currently wrong, we also add an
utility function for this and change all the error messages to use it.
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Since the cluster tags are/should be more-or-less static, add them as an
ssconf key, so that querying them is possible without creating a
job/requiring the masterd to be running.

Reviewed-by: imsnah

This updates the iallocator documentation to 2.0, bumps up the
iallocator version (and moves a constants to lib/constants.py), and
fixes a style on install.rst.

Reviewed-by: ultrotter

Since we are quite close to final RPC and hooks APIs, we update the hooks and
protocol_version constants.

Reviewed-by: imsnah

This patch adds a new hypervisor parameter for the hypervisors that can
actually start and instance with external kernels.

Reviewed-by: imsnah

As discussed with 2.0 msdos partition style should be the default in the
instance OS, so we're changing the default instance params accordingly.
A followup patch will update the debootstrap os.

Reviewed-by: iustinp

Currently we use the HTS_USE_VNC constant only to copy the vnc password
file. While KVM uses vnc it currently has no password support, nor we'll
be on time making one for 2.0, so renaming the constant to
HTS_COPY_VNC_PASSWORD and only putting Xen HVM in it. In the future
(2.1) password handling will need to be reworked anyway.

Reviewed-by: iustinp

We want all the hv/be parameters to have a known type, rather than a
random mix of empty string, boolean values, and None, so we declare the
type of each variable and we enforce/convert it.

- Add some new constants for enforceable value types
- Add new constants dicts HVS_PARAMETER_TYPES and BES_PARAMETER_TYPES
  holding not only the valid parameters but also their types
- Drop the old HVS_PARAMETERS and BES_PARAMETERS constants and calculate
  the values from the type dict
- Convert all the default parameters to a valid type value
- Create a new ForceDictType utils function, to check/enforce a dict's
  element value types, with relevant unit tests
- Drop a few custom functions to check/convert the BE param types in
  utils and cli, in favor of ForceDictType
- Double-check the parameter types using ForceDictType in both scripts
  and LogicalUnits, when possible.

As a bonus:
- Remove some old commented-out code in gnt-instance
- Remove some already fixed FIXME
- Fix a bug which prevented VALUE_DEFAULT to be applied to BE parameters
  in SetInstanceParams because the value was checked for validity before
  that transformation was made
- Fix a bug which prevented initing a cluster and passing hvparams to
  work at all
- ForceDictType allows an allowed_values for exceptions, which makes us
  able to do the checking even when some values must not be
  converted/typechecked (for example the 'default' string in
  SetInstanceParameters)

Reviewed-by: iustinp

In some cases 'mouse' may work better than 'tablet', so we'll handle
both by allowing the user to specify a parameter. By default no mouse is
used.

Reviewed-by: iustinp

- Add a bunch of NICs and DISKs types
- Specify which one are valid disks and nics for KVM (the new ones
  toghether with some of the old ones)
- Add the default values (paravirtual)
- Allow the disk and nic types as parameters and check their validity

Reviewed-by: iustinp

These are not HVM specific, so have been given an HT generic name.

Reviewed-by: iustinp

The VNC base port has nothing to do with HVM itself, and is general to
VNC itself, so we're removing the HT_HVM prefix to the constant.

Reviewed-by: iustinp

The cdrom image has the same meaning than in Xen HVM, and so does
boot_order, even though it has a slightly different syntax, and uses the
value 'disk' too boot from disk and 'cdrom' to boot from cdrom.

Reviewed-by: iustinp

Confusingly, as a leftober from 1.2, there was a
constants.HT_HVM_DEFAULT_BOOT_ORDER constant, with a value opposite to
the default HV_BOOT_ORDER hv param that got enabled only if
HV_BOOT_ORDER was set to None. Since setting it to None is very
hard/impossible for the user, and we didn't handle other "empty" values
(False, ''), we'll just force the parameter to have a valid value (after
all we have a default, and that's the way we use hvparams) and get rid
of the old constant altoghether.

Reviewed-by: iustinp

With this parameters VNC for KVM is able to be protected by tls,
optionally with an x509 certificate, and optionally verifying the
client as well. Additionally in this patch we limit the bind address to
being a directory, rather than a file or a directory, for simplicity, as
it allows for the same level of control anyway.

Reviewed-by: iustinp

This patch adds the online node list and instance list to the ssconf
keys. In order to do distribute correctly the instance list, we need to
update the cluster serial number on instance additions and removals.

The patch also changes the permissions on the ssconf files to be 0444:
  - no write for root, in order to signal that these file should not be
    modified
  - read for everyone since the files don't contain sensitive data
    anymore (and permissions can be controlled via the parent directory
    if needed)

The patch also fixes a small typo on gnt-cluster.

Reviewed-by: ultrotter

This patch only changes some indentation in constants.py.

Reviewed-by: imsnah

Reviewed-by: iustinp

Before we used to copy the file if xen-hvm was enabled on the cluster,
no we'll do that if any enabled hypervisor is in the new HTS_USE_VNC
group.

Reviewed-by: iustinp

HT_KVM doesn't technically require a port, but if it has one it can give
vnc displays to instances.

Reviewed-by: iustinp

Up until now a KVM instance was forced to have a serial port.
With this change this is no longer mandatory, by default we'll use one,
but if the HV_SERIAL_CONSOLE parameter is set to False we'll do without.

Reviewed-by: iustinp

This parameter allows a different path to be passed to the instance
kernel. The new parameter is mandatory, and by default has the value of
the old hardcoded value for both kvm and xen.

Beta1 clusters will need to have this parameter added for their
instances to be able to boot.

Reviewed-by: iustinp

The tcp port used for migrating KVM instances is selectable at
./configure time. We use a single port as nodes are locked anyway during
a migration, so no two migrations can happen at the same time to the
same node.

Reviewed-by: iustinp

This patch adds support for verification of drbd minors space in cluster
verify: minors which belong to running instances and should be online
but are not, and minors which do not belong to any instace but are in
use.

The patch requires exposing some methods from bdev.DRBD8 and
config.ConfigWriter which were until now private methods.

Reviewed-by: ultrotter

Instead of having the default live in the gnt-cluster script, we move it
to the constants file. The patch also fixes a typo on constants.py.

Reviewed-by: ultrotter

This is a forward-port of commit 2219 on the 1.2 branch.

Reviewed-by: ultrotter

The patch adds a new ssconf file containing the ganeti version.

Reviewed-by: imsnah

This patch updates burnin to the latest replace disks constant, and
changes the constants' values to be more accurate.

Reviewed-by: imsnah

In 1.2, due to the md+drbd7 legacy, we had a complex choice of replace
modes, and the new drbd8 modes where forced into this syntax, with some
complicated rules of transition from one mode to another (if REPLACE_ALL
but not new node passed, switch to REPLACE_SEC, etc.).

This patch cleans this situation by making a clear separation between
the two main modes:
  - replace on current nodes (with the two sub-cases on primary and on
    secondary)
  - change to a new node (either via manually specified node or via
    iallocator)

Reviewed-by: imsnah

Reviewed-by: amishchenko

Passwords are stored in "$localstatedir/lib/ganeti/rapi_users". User
options specify the access permissions of a user (see docstring for
ganeti.http.ReadPasswordFile), for which only "write" is supported
to grant write access. Every other user has read-only access.

Reviewed-by: amishchenko

A job queue with too many jobs can increase memory usage and/or make
the master daemon slow. The current limit is just an arbitrary number.
A "soft" limit for automatic job archival is prepared.

Reviewed-by: iustinp

Call socat with a full path specified at configure time, rather than
just by its name, and check for the binary to exist at hypervisor
verify.

Reviewed-by: iustinp

It makes sense for the default kvm kernel not to be called "xenU".

Reviewed-by: iustinp

Adding compression to larger amounts of data is more efficient than
transferring it (len(nodes) - 1) times over the network without
compression. We were able to compress a 800KB config file to about
30 KB, which is about 40 KB with Base64 encoding (required due to
the way SimpleJson handles strings).

Reviewed-by: ultrotter

The patch also changes the various node list generation to be more
consistent.

Reviewed-by: imsnah

This constant will be used at cluster init time.

Reviewed-by: imsnah

This partially reorganizes the cluster verify LU:
  - introduce constants for the node verify rpc call
  - move from additional rpc calls to a single rpc call, the
    call_node_info, which gaters all data needed

Also fix a small error (self.LogWarning instead of self.Warning).

Reviewed-by: imsnah

These two operations were missing from burnin. The reboot is done with
all valid modes (a new constant is added), and the reinstall is done
both with and without specifying the OS (to account for the two code
paths in the LU).

Reviewed-by: imsnah