* commit 'v2.5.1':
  Update NEWS and bump version for 2.5.1 release
  Fix gnt-group --help display
  Fix hardcoded Xen kernel path
  Fix grow-disk handling of invalid units
  Accept both PUT and POST in noded
  Preserve bridge MTU in KVM ifup script
  Update synopsis for “gnt-cluster repair-disk-sizes”
  Reconcile Makefile.am and test data files
  Workaround changed LVM behaviour
  Enable lvmstrap to run under Linux 3.x
  Add a default PATH variable to OS scripts env
  Move hooks PATH environment variable to constants
  Add note to the install doc about bridge MAC issues
  Fix exception re-raising in Python Luxi clients
  Fix LVM volume listing with newer LVM

Conflicts:
        lib/client/gnt_group.py (trivial, fixup conflicts from 6bc3ed14)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Copy-paste mismatch :)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
(cherry picked from commit 36c70d4d)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

We already have a ./configure-time variable for this, but it seems to
be actually unused.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 3c4afa2e)

Signed-off-by: Iustin Pop <iustin@google.com>
(trivial patch, let's cherry-pick it)
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The reason why grow-disk was doing:

$ gnt-instance grow-disk instance3 0 -64
Unhandled Ganeti error: Invalid format

Is because it does it's own ParseUnit call, and doesn't transform that
into a nicer message.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
(cherry picked from commit c8bde61e)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This is a partial cherry-pick from
7530364d on master:

Currently, noded requires PUT, even though the semantics of the RPC
calls do not match a PUT. We change the code accept both PUT and POST,
with the intention to remove the PUT support in a later version.

Additionally, we add a message to the HttpBadRequest exception to make
clear the failure mode (not seeing any error message was what made me
send this patch…). This was the only description-less use of this
exception, by the way.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
(cherry picked from commit 7530364d)

What was not cherry-picked is the rpc change (to switch to PUT). The
reason I want to backport this to devel-2.5 is that when upgrading to
2.6, having noded accept both makes for an easier upgrade path.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 5d0566de)

Signed-off-by: Iustin Pop <iustin@google.com>

Yet another cherry-pick (must go deeper!); since we might not make a
new release from the devel-2.5 branch, let's add this to stable-2.5.

Reviewed-by: Michael Hanselmann <hansmi@google.com>

Closes: #201 - KVM_IFUP does not set bridge-MTU on tap devices
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit a1ec8695)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Mention that instances can be passed on the CLI when “--help” is used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
(cherry picked from commit eb5ac108)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Sorry, forgot this in previous commit.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 1a1e7ab3)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The vgreduce command has changed behaviour from when we initially
wrote the code (2.02.02 versus 2.02.66, 4 years delta):

- if there are LVs which will be impacted, it requires --force
- otherwise refuses to proceed, but it still returns exit code 0

We handle this by looking to see if it returns "Wrote out consistent
volume group" (behaviour unchanged), or if it complains about
"--force"; in the case it didn't complete, we retry the operation.

We improve a bit the checking of "vgs", as it uses to fail silently
and we didn't detect it.

New tests for this function should test, I believe, all the expected
variations; at the least we now have data files with the expected
output.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
(cherry picked from commit 048eeb2b)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Extend the kernel version check to also accept Linux 3.x as valid.

Signed-off-by: Alexander Schreiber <als@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit 1bf72492)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

In commit 896a03f6 I cleaned up the environment for OS scripts,
however I think that was a bit too extreme - it breaks our own
instance-debootstrap hooks, because for example dpkg (called from the
grub script) requires PATH to be set.

Instead of requiring every OS to define a path, let's set a default
PATH for the OS scripts, which should cover most common uses. A more
specialised PATH can be set, if needed, in the OS scripts.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the contents of the PATH environment variable for hooks to
constants, and use its value in the code and in the hooks documentation.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
(cherry picked from commit fe5ca2bb)
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Thanks to Faidon Liambotis for explaining this on the external IRC
channel.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Faidon Liambotis <paravoid@gmail.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commit e687ec01 (present in 2.5 since the 2.5 beta 3) did consistency
fixes across the code-base. Unfortunately this was done without enough
checks on the actual meaning of one of the fixes, which means error
re-raising in lib/errors.py is broken.

The problem is that:

  raise cls, args

is different than:

  raise cls(args)

And our unit-tests didn't catch this (this patch updates the tests).

This breakage is usually trivial, like wrong error messages:

  $ gnt-instance remove no-such-instance
  Failure: prerequisites not met for this operation:
  ("Instance 'no-such-instance' not known", 'unknown_entity')

versus:

  $ gnt-instance remove no-such-instance
  Failure: prerequisites not met for this operation:
  error type: unknown_entity, error details:
  Instance 'no-such-instance' not known

or:

  $ gnt-instance add … no-such-instance
  Failure: prerequisites not met for this operation:
  ('The given name (no-such-instance) does not resolve: Name or service not known', 'resolver_error')

versus:

  $ gnt-instance add … no-such-instance
  Failure: prerequisites not met for this operation:
  error type: resolver_error, error details:
  The given name (no-such-instance) does not resolve: Name or service not known

But in some cases where we rely on a certain data representation
(e.g. HooksAbort), this actually breaks because we try to iterate over
the wrong type:

  File "/usr/lib/python2.6/dist-packages/ganeti/cli.py", line 1907, in FormatError
     for node, script, out in err.args[0]:
  ValueError: need more than 1 value to unpack

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Per commit 0304f0ec, newer LVM has extended the lv_attr field. However,
that commit was incomplete as we examine this attribute in another
place in the code.

Thanks to user alperhome, the _LVSLINE_REGEX in lib/backend.py also
needs fixing. I've used the same change as in the above commit: accept
at minimum 6 characters, but allow for more.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

- Listing tags is a query, so neither “--priority” nor “--submit” make
  sense
- Support both options for adding/removing tags
- Also remove “--submit” from “gnt-node health”; it doesn't work and
  doesn't make sense for listing node health

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* stable-2.5:
  Bump version for 2.5.0 final release
  configure.ac: Fix “too many arguments” error
  Fix extra whitespace
  Further fixes concerning drbd port release
  Fix a bug concerning TCP port release
  Fix extra whitespace
  Fix a bug concerning TCP port release

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Also update NEWS file.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* devel-2.4:
  Fix extra whitespace
  Further fixes concerning drbd port release
  Fix a bug concerning TCP port release

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If GHC_PKG_QUICKCHECK contains multiple values, the test would fail
with “too many arguments”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Sorry, didn't catch this before…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit 54b010ca)

Signed-off-by: Michael Hanselmann <hansmi@google.com>

Commit 3b3b1bca does not entirely fix the bug introduced in commit
f396ad8c. It fixes consistency of config data in permanent storage, but
does not ensure consistency in data held in runtime memory of masterd.

The bug of duplicate ports is still triggered when LUInstanceRemove()
invokes _RemoveDisks() and this returns False (in case
call_blockdev_remove RPC fails). The drbd ports get returned in the
pool, but execution is aborted and RemoveInstance() is never invoked.

Due to the fact that port handling is not done with
TemporaryReservationManager, ensure that ports are released, only if
disk related config data is deleted.

In _RemoveDisks() release ports only if all RPCs succeed.

Extend _RemoveDisks() to include ignore_failures argument passed by
_RemoveInstance() to handle the ports appropriately.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Commit f396ad8c returns the TCP port used by DRBD disk back to the
TCP/UDP port pool using AddTcpUdpPort().

However, AddTcpUdpPort() writes the config on every invocation,
using _WriteConfig(). This causes two problems:

 * it causes critical errors logged by VerifyConfig(), after the DRBD
   disk removal, and until the actual instance removal.
 * if the code following AddTcpUdpPort() fails, the port is already
   returned back the pool, which causes the port to have duplicates
   (inconsistent config).

AddTcpUdpPort() is invoked in three cases:

 * during InstanceRemove() through _RemoveDisks().
 * during InstanceSetParams() in case of disk removal.
 * during InstanceSetParams() through _ConvertDrbdToPlain().

This commit fixes the problem by removing the _WriteConfig() call from
AddTcpUdpPort(), delegate it to Update() via the
TemporaryReservationManager and ensure AddTcpUdpPort() precedes
Update().

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
[iustin@google.com: small comments adjustements]
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit 3b3b1bca)

Sorry, didn't catch this before…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Commit f396ad8c returns the TCP port used by DRBD disk back to the
TCP/UDP port pool using AddTcpUdpPort().

However, AddTcpUdpPort() writes the config on every invocation,
using _WriteConfig(). This causes two problems:

 * it causes critical errors logged by VerifyConfig(), after the DRBD
   disk removal, and until the actual instance removal.
 * if the code following AddTcpUdpPort() fails, the port is already
   returned back the pool, which causes the port to have duplicates
   (inconsistent config).

AddTcpUdpPort() is invoked in three cases:

 * during InstanceRemove() through _RemoveDisks().
 * during InstanceSetParams() in case of disk removal.
 * during InstanceSetParams() through _ConvertDrbdToPlain().

This commit fixes the problem by removing the _WriteConfig() call from
AddTcpUdpPort(), delegate it to Update() via the
TemporaryReservationManager and ensure AddTcpUdpPort() precedes
Update().

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
[iustin@google.com: small comments adjustements]
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Eric Rostetter sent a patch adding a “status” action, but unfortunately
his code was apparently specific to Red Hat. I hope this implementation
is more distribution-agnostic; after all “status_of_proc” is part of
LSB. Example output:

$ /etc/init.d/ganeti status
ganeti-noded is not running ... failed!
ganeti-masterd is running.
ganeti-rapi is running.
ganeti-confd is running.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* stable-2.5:
  LUOobCommand: acquire BGL in shared mode

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fixed a typo so that now LUOobCommand acquires the BLG in shared mode, as
intended.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fix a typo introduced in commit c85b15c1, which breaks epydoc.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

* stable-2.5:
  LUNodeAdd: Verify version in Prereq
  Fix LV status parsing to accept newer LVM
  Bump version for 2.5.0~rc6 release
  Revert "Stop acquiring BGL for LUXI queries"
  LUClusterVerifyConfig: Share BGL, acquire all locks in shared mode
  KVM: don't add -nographic using spice
  Stop acquiring BGL for LUXI queries
  Fix type error in LUInstanceChangeGroup

Conflicts:
	lib/hypervisor/hv_kvm.py
	  - trivial, keep both changes

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

There are other ways to leave the cluster in a broken state than just
the version check. However they are not very trivial to fix in 2.5. So
leave it up to 2.6 for a nicer fix.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
(cherry picked from commit e2ea8de1)

There are other ways to leave the cluster in a broken state than just
the version check. However they are not very trivial to fix in 2.5. So
leave it up to 2.6 for a nicer fix.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

LVM version 2.02.93 (or at least, sometimes after .88) has extend the
lv_attr field with two more flag; we only care about the first digit,
so let's change the "!= 6" check to "< 6".

Thanks to Robin H Johnson <robbat2@gentoo.org> for finding this issue.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This requires acquiring the node group locks in shared mode.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The “cur_group_uuid” parameter is optional to prepare for using the
factorized code from LUInstanceQueryData.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

While debugging another issue we realized that LUClusterQuery forks.
This turned out to be the “platform.architecture” function from the
Python library. It uses the “file” command to determine the architecture
of the Python binary.

This patch adds two new functions to the “runtime” module to get this
information once per process instead of doing it every single time
LUClusterQuery is used. Forking is a no-go in a multi-threaded
environment anyway.

A future change will also have to change the terminology in “gnt-cluster
info”: it reports the binary architecture simply as “architecture”, when
it's actually the binaries' architecture. Kernel and userland can be
different.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

Don't notify for every released lock in shared mode. The last one is
enough.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>