The mcpu.py:HooksMaster class needs to have a proc attribute/argument to
init in ordet to call its LogWarning method. However, this is available
from the 'lu' attribute, so we can remove this dependency.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

We don't format with subsecond-precision.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The salt needs to be returned anyway, so we don't have to add another
key for the sender to recognize which request an answer is answering, so
all that infrastructure is useless. :(

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The method is changed to a normal loop, to avoid calling getattr()
twice. Also __getstate__ is changed to just use ToDict() by default.

This should also make __getstate__ work for objects which have to
override the ToDict function because they contain other objects.

__setstate__ is probably still broken in this case, but so it was
before, and it's not used inside our code, so I'll pretend not to have
noticed, as there is no "nice" way to fix it, without overriding it all
over the place :(

Some unittests are added as a bonus, to make sure we behave well.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

File "../test/ganeti.hooks_unittest.py", line 239, in setUp
  self.lu = FakeLU(FakeProc(), self.op, self.context, None)
File "…/ganeti/cmdlib.py", line 92, in __init__
  self.LogStep = processor.LogStep
AttributeError: FakeProc instance has no attribute 'LogStep'

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This parameter is now mandatory for the cluster config to work.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently we have both a default_hypervisor and an enabled_hypervisors
list. The former is only settable at cluster init time, while the latter
can be changed with cluster modify.

This becomes cumbersome in a few ways: at cluster init time for example
if we pass in a list of enabled hypervisors which doesn't include the
"default" xen-pvm one, we're also forced to pass a default hypervisor,
or an error will be reported. It is also currently possible to disable
the default hypervisor in cluster-modify (with unknown results).

In order to avoid this we get rid of this field altogether, and define
the "first" enabled hypervisor as the default one. This allows ease of
changing which one is the default, and at the same time maintains
coherency.

At configuration upgrade we make sure that the old default is first in
the list, so that 2.0 cluster defaults are preserved.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This patch includes HMAC authenticated json messages to the serializer.
The new interface works on any json-encodable data type, and can sign it
with a private key and an optional salt. The same private key must be
used upon message loading to verify the message.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

If an ident member of an IdentKeyVal relationship starts with no_ or -,
handle it the same way we do for a key. Some unittests are added to
check that check_ident_key_val behaves as expected.

This patch also changes ForceDictType to, for now, fail on such an
entry, and the same to happen when creating an instance or modifying its
nics or disks.

This behavior will be used later on to allow deletion of os entries in
os parameters.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If an empty string is passed to _SplitKeyVal, we should return {},
rather than {'': True}. Also test for the correct behavior.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Patch "Simplify the RPC result framework in backend.py" changed all
RPCs, and thus the hooks results where also changed. This needs changes
to the hooks unittests too.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This also converts (and fixes) unittests and mock objects to deal with
this change, and the custom hook verifier in cmdlib.LUClusterVerify.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Only the custom handling in rpc.py for call_node_info used this, so now
it can be removed.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The target test cluster should have nic params, rather than a default
bridge.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This function will be used to check the NIC parameters for validity.
Unittests are included.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We used to refer to normalized paths as "normal" which might be
confusing. This fixes the syntax in all current IsNormAbsPath users and
in the docstring.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently most of the time we check for absolute path, but that doesn't
protect us from some invalid paths. In some places we should be more
strict, and this function should help us to.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Currently we miss in hooks the instance's hypervisor, hypervisor
parameters and backend parameters. This forces hooks to query back into
ganeti, which is dangerous due to possible luxi sockets exhaustion.

This patch adds these three as INSTANCE_HYPERVISOR, INSTANCE_HV_*,
INSTANCE_BE_*. The hook environment prefixes all keys with “GANETI”, so
a default settings for a xen-pvm instance would be:

  GANETI_INSTANCE_HV_initrd_path=
  GANETI_INSTANCE_HV_kernel_args=ro
  GANETI_INSTANCE_HV_kernel_path=/boot/vmlinuz-2.6-xenU
  GANETI_INSTANCE_HV_root_path=/dev/sda1

Any dashes in parameter names are changed to underscores, since
variables with dashes are not easy to access from the shell
(alternatively we could deny those via an unittest for constants.py).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently we have bad behaviour in SafeEncode:
  - binary strings are actually not handled correctly (ahem)
  - the encoding is not stable, due to use of string_escape

For this reason, we replace the use of string_escape with part of the
code of string escape (PyString_Repr in Objects/stringobject.c); we
don't escape backslashes or single quotes, since that is that makes it
nonstable. Furthermore, we only use the encode('ascii', ...) for unicode
inputs.

The patch also adds unittests for the function that test basic
behaviour.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

DRBD 8.3 changes two more things compared to 8.2:
  - /proc/drbd format changed in multiple ways; the part we're
    interested is the ‘st:’ to ‘ro:‘ change (in the changelog named as
    “Renamed 'state' to 'role'”
  - “drbdsetup /dev/drbdN show” changed the ‘device’ stanza from:
      device "/dev/drbd0";
    to:
      device                  minor 0;

This patch fixes these both and adds data files and unittests for DRBD
8.3.1.

Signed-off-by: Iustin Pop <iustin@google.com>

If /proc/drbd can't be opened, this raises an IOError, but all the
error-handling behaviour in backend treats only BlockDeviceErrors. This
creates a plain failure in cluster verify and in other RPC calls.

This patch simply converts EnvironmentErrors into BlockDeviceErrors, and
also changes the RPC result for NV_DRBDLIST and its handling to be able
to show the error. The other RPC calls work by default now, due the
existing error handling.

Reviewed-by: ultrotter

In the current Ganeti version when modifying /etc/hosts we mistakenly
give it the permissions of the temporary file we create to define its
content, which is by default 0600. This breaks most non-root
applications, and thus must be corrected. This patch forces the mode to
be 0644 (but we might decide to just use the mode of the previous
/etc/hosts, if we want to be more polite against any eventual
administrative choice). We also add a new assertFileMode() method for
unit tests and actually check in the SetEtcHostsEntry and
RemoveEtcHostsEntry tests that the mode is correct, to be sure not to
reintroduce this bug again. Also, a FIXME is added in the original
functions stating that it would be nice to use WriteFile+fn() rather
than reimplementing its functionality again.

Reviewed-by: iustinp

We want all the hv/be parameters to have a known type, rather than a
random mix of empty string, boolean values, and None, so we declare the
type of each variable and we enforce/convert it.

- Add some new constants for enforceable value types
- Add new constants dicts HVS_PARAMETER_TYPES and BES_PARAMETER_TYPES
  holding not only the valid parameters but also their types
- Drop the old HVS_PARAMETERS and BES_PARAMETERS constants and calculate
  the values from the type dict
- Convert all the default parameters to a valid type value
- Create a new ForceDictType utils function, to check/enforce a dict's
  element value types, with relevant unit tests
- Drop a few custom functions to check/convert the BE param types in
  utils and cli, in favor of ForceDictType
- Double-check the parameter types using ForceDictType in both scripts
  and LogicalUnits, when possible.

As a bonus:
- Remove some old commented-out code in gnt-instance
- Remove some already fixed FIXME
- Fix a bug which prevented VALUE_DEFAULT to be applied to BE parameters
  in SetInstanceParams because the value was checked for validity before
  that transformation was made
- Fix a bug which prevented initing a cluster and passing hvparams to
  work at all
- ForceDictType allows an allowed_values for exceptions, which makes us
  able to do the checking even when some values must not be
  converted/typechecked (for example the 'default' string in
  SetInstanceParameters)

Reviewed-by: iustinp

Due to the fact that we sanitize now the output from environment
scripts, the unittest needs to be adjusted. My bad for not checking it.

Reviewed-by: imsnah

The resources we still need moved to rlib2.

Reviewed-by: iustinp

Reviewed-by: iustinp

It is impossible to keep backward compatibility due to
significant changes in the Ganeti core.

Reviewed-by: iustinp

This patch enhances the duplicate DRBD minors checks (currently just a
few) and adds automatic checks of configuration consistency at
configuration file writing time.

In order to do so and show meaningful error messages, the
_UnlockedComputeDRBDMap function is changed to not raise errors in case
of duplicates, but instead return both the minors map and the duplicate
list, and its callers now raise the error. This allows the VerifyConfig
function to return a complete list of duplicates.

The new checks required some small updates to the unittests for the
config module.

Reviewed-by: ultrotter

In order to prevent errors with old, in-use DRBD minors, we check and
abort at create time if our minor is already in use. For this we need to
also modify DRBD8Status to be able to parse cs:Unconfigured devices.

Reviewed-by: ultrotter

This patch adds a tail file function, to be used for parsing and returning in
the job log OS installation failures.

Reviewed-by: ultrotter

This patch adds unified temporary file handling to the
testutils.GanetiTestCase class, which adds easy creation and automated
cleanup of temporary files.

The patch allows a simpler handling in a couple of test cases but
requires all child classes to call the parent setUp and tearDown
methods.

Reviewed-by: ultrotter

Reviewed-by: amishchenko

Unfortunately, os.makedirs in Python 2.4 is not safe against multiple
processes creating the same directory tree at the same time. This is
only fixed in Python 2.5 and up. Adding more checks in our code doesn't
make it any better.

Reviewed-by: iustinp

More complex unittests will need some refactoring in the HTTP code.

Reviewed-by: amishchenko

It should be consistent.

Reviewed-by: amishchenko

We don't need it anymore and it wouldn't work as it is, anyway.

Reviewed-by: ultrotter

Reviewed-by: iustinp

Reviewed-by: imsnah