This is useful if you have nested parameter dicts like in diskparams.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* devel-2.5:
  Update NEWS and bump version for 2.5.1 release
  Fix gnt-group --help display
  Fix hardcoded Xen kernel path
  Fix grow-disk handling of invalid units
  Accept both PUT and POST in noded
  Preserve bridge MTU in KVM ifup script
  Update synopsis for “gnt-cluster repair-disk-sizes”
  Reconcile Makefile.am and test data files
  Workaround changed LVM behaviour
  Enable lvmstrap to run under Linux 3.x
  Add a default PATH variable to OS scripts env
  Move hooks PATH environment variable to constants
  Add note to the install doc about bridge MAC issues
  Fix exception re-raising in Python Luxi clients
  Fix LVM volume listing with newer LVM

Conflicts:
        NEWS (trivial)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

* commit 'v2.5.1':
  Update NEWS and bump version for 2.5.1 release
  Fix gnt-group --help display
  Fix hardcoded Xen kernel path
  Fix grow-disk handling of invalid units
  Accept both PUT and POST in noded
  Preserve bridge MTU in KVM ifup script
  Update synopsis for “gnt-cluster repair-disk-sizes”
  Reconcile Makefile.am and test data files
  Workaround changed LVM behaviour
  Enable lvmstrap to run under Linux 3.x
  Add a default PATH variable to OS scripts env
  Move hooks PATH environment variable to constants
  Add note to the install doc about bridge MAC issues
  Fix exception re-raising in Python Luxi clients
  Fix LVM volume listing with newer LVM

Conflicts:
        lib/client/gnt_group.py (trivial, fixup conflicts from 6bc3ed14)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Should be straightforward; the only difference from before is that we
will remove the 'auto' file even during migration finalise.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Copy-paste mismatch :)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
(cherry picked from commit 36c70d4d)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

We already have a ./configure-time variable for this, but it seems to
be actually unused.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 3c4afa2e)

Signed-off-by: Iustin Pop <iustin@google.com>
(trivial patch, let's cherry-pick it)
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The reason why grow-disk was doing:

$ gnt-instance grow-disk instance3 0 -64
Unhandled Ganeti error: Invalid format

Is because it does it's own ParseUnit call, and doesn't transform that
into a nicer message.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
(cherry picked from commit c8bde61e)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is a partial cherry-pick from
7530364d on master:

Currently, noded requires PUT, even though the semantics of the RPC
calls do not match a PUT. We change the code accept both PUT and POST,
with the intention to remove the PUT support in a later version.

Additionally, we add a message to the HttpBadRequest exception to make
clear the failure mode (not seeing any error message was what made me
send this patch…). This was the only description-less use of this
exception, by the way.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
(cherry picked from commit 7530364d)

What was not cherry-picked is the rpc change (to switch to PUT). The
reason I want to backport this to devel-2.5 is that when upgrading to
2.6, having noded accept both makes for an easier upgrade path.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 5d0566de)

Signed-off-by: Iustin Pop <iustin@google.com>

Yet another cherry-pick (must go deeper!); since we might not make a
new release from the devel-2.5 branch, let's add this to stable-2.5.

Reviewed-by: Michael Hanselmann <hansmi@google.com>

Closes: #201 - KVM_IFUP does not set bridge-MTU on tap devices
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit a1ec8695)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Mention that instances can be passed on the CLI when “--help” is used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
(cherry picked from commit eb5ac108)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Sorry, forgot this in previous commit.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 1a1e7ab3)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The vgreduce command has changed behaviour from when we initially
wrote the code (2.02.02 versus 2.02.66, 4 years delta):

- if there are LVs which will be impacted, it requires --force
- otherwise refuses to proceed, but it still returns exit code 0

We handle this by looking to see if it returns "Wrote out consistent
volume group" (behaviour unchanged), or if it complains about
"--force"; in the case it didn't complete, we retry the operation.

We improve a bit the checking of "vgs", as it uses to fail silently
and we didn't detect it.

New tests for this function should test, I believe, all the expected
variations; at the least we now have data files with the expected
output.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 048eeb2b)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Extend the kernel version check to also accept Linux 3.x as valid.

Signed-off-by: Alexander Schreiber <als@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit 1bf72492)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Similiar to blockdev_create we sometimes do find on children. This fixes
those cases. However, this is not very nice.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is due to the nature of bdev. We spread some logic into cmdlib and
deal for example with it's children recursively. This makes it hard to
annotate the disk parameters in a generic way as we don't always deal
with the top most disk. But the disk parameters are depending on the top
device not on the children.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The following (blockdev) RPC calls are not converted yet (as they are
not straight forward or need more research):

* bdev_sizes
* blockdev_remove
* blockdev_shutdown
* blockdev_removechildren
* blockdev_close
* blockdev_getsize
* drbd_disconnect_net
* blockdev_rename (has already a special encoder, needs further research
  if needed at all)
* blockdev_getmirrorstatus (not sure if we have everywhere a clear link
  to the instance the disk belongs)
* blockdev_getmirrorstatus_multi (same here, further research)

Then special cases where we take care later in the patch series:

* blockdev_create (special cased)
* blockdev_find (special cased, like blockdev_create)

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

We change the cmdlib side on the next patch.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Start using “InstanceCheck” decorator where appropriate. Use direct call
for check in places where the instance status changes during the test.

The parameters of “TestInstanceImport” had to be re-ordered to have the
instance as the first argument.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This script can be used to check if an instance is running or stopped at
various points during a QA run. Environment variables are used to pass
the most essential information.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Until now this wasn't possible.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In commit 896a03f6 I cleaned up the environment for OS scripts,
however I think that was a bit too extreme - it breaks our own
instance-debootstrap hooks, because for example dpkg (called from the
grub script) requires PATH to be set.

Instead of requiring every OS to define a path, let's set a default
PATH for the OS scripts, which should cover most common uses. A more
specialised PATH can be set, if needed, in the OS scripts.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the contents of the PATH environment variable for hooks to
constants, and use its value in the code and in the hooks documentation.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
(cherry picked from commit fe5ca2bb)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Thanks to Faidon Liambotis for explaining this on the external IRC
channel.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Faidon Liambotis <paravoid@gmail.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commit e687ec01 (present in 2.5 since the 2.5 beta 3) did consistency
fixes across the code-base. Unfortunately this was done without enough
checks on the actual meaning of one of the fixes, which means error
re-raising in lib/errors.py is broken.

The problem is that:

  raise cls, args

is different than:

  raise cls(args)

And our unit-tests didn't catch this (this patch updates the tests).

This breakage is usually trivial, like wrong error messages:

  $ gnt-instance remove no-such-instance
  Failure: prerequisites not met for this operation:
  ("Instance 'no-such-instance' not known", 'unknown_entity')

versus:

  $ gnt-instance remove no-such-instance
  Failure: prerequisites not met for this operation:
  error type: unknown_entity, error details:
  Instance 'no-such-instance' not known

or:

  $ gnt-instance add … no-such-instance
  Failure: prerequisites not met for this operation:
  ('The given name (no-such-instance) does not resolve: Name or service not known', 'resolver_error')

versus:

  $ gnt-instance add … no-such-instance
  Failure: prerequisites not met for this operation:
  error type: resolver_error, error details:
  The given name (no-such-instance) does not resolve: Name or service not known

But in some cases where we rely on a certain data representation
(e.g. HooksAbort), this actually breaks because we try to iterate over
the wrong type:

  File "/usr/lib/python2.6/dist-packages/ganeti/cli.py", line 1907, in FormatError
     for node, script, out in err.args[0]:
  ValueError: need more than 1 value to unpack

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently we allow for up to NODE_MAX_CLOCK_SKEW time difference
between nodes in some operations, but not everywhere: SSL certificate
verification (import/export, both intra and inter-cluster) has a zero
limit (downwards), and a week upwards. This can cause even
intra-cluster backup problems, if the source node has a time even two
seconds in the future.

To fix this, when we verify certificates compare with a time offset
with the max skew, which fixes the lower bound and reduces the upper
bound by an insignificant amount (0.04%).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The pyeval role generates literal (e.g. ``OP_CLUSTER…``) formatting,
which differs from the other headers. This is confusing, so let's add
a note that these are virtual operations, which makes the output more
consistent.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch adds a hand-coded decoder for LuxiCall arguments, as the
data-structure is not uniform enough for automated generation (even
for the serialisation, we had to add hints for some fields,
de-serialisation is even harder).

It also fixes a tiny issue with WaitForJobChange job ID encoding, and
adds unittests for the encoding/decoding of LuxiOp structures.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>