* devel-2.3:
  Fix typo in gnt-instance manpage
  jqueue: Fix cancelling while in waitlock in queue
  cli: Extend message for LUXI timeouts
  Fix timeout handling in LUXI client

Conflicts:
	man/gnt-instance.sgml: Trivial merge in gnt-instance.rst

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

- Remove non-ASCII character from manpage
- Reformat docstring for epydoc in script

These caused build breakage on some but not all distributions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

s/os-name/os-type/. This was reported in issue 133.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This should reduce the amount of damage in case of accidential copy &
paste.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This tool was used and worked on internally for quite a long time. We
decided to include it in Ganeti.

Known issues:
- Code doesn't match rest of Ganeti (e.g. using “print” all over the
  place, hardcoded calls to sys.exit deep in functions)
- Code duplication from Ganeti library (e.g. PingByTcp/netutils.TcpPing,
  GetHosts/utils.ReadFile)
- Using ssh-agent doesn't work with more than one worker (Paramiko keeps
  the socket open and the file descriptor is used from different
  workers)
  - No clear separation between parent and child process in code
- Uses getopt instead of optparse

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Since the recent change to leave jobs in the “waitlock” status (commit
5fd6b694), cancelling a job while it's back in the queue would break.
This patch handles these cases and adds a unittest.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

While the LU does return the final name, it's useful to log the actual
DNS resolving process (input and output) in order to help with the
diagnose of failures.

The patch also fixes the docstring of the Exec() function.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The list of fields is not only sorted, but sorted in a nice way.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Long story short: time.strftime("%Z", time.localtime()) doesn't work,
even though it's documented to be equivalent to time.strftime("%Z").

$ TZ=America/Sao_Paulo python -c 'import time; print
time.strftime("%Z"), time.strftime("%Z", time.localtime())'
BRST LMT

References:
http://bugs.python.org/issue762963
https://bugs.launchpad.net/ubuntu/+source/python2.6/+bug/564607
http://stackoverflow.com/questions/4367896/issue-with-timezone-with-time-strftime



Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

As per issue 124, some Xen versions (or packaging) don't deal nicely
with the colon being part of a disk name. Therefore we add a
configure-time option for customising this.

Note: setting the separator to interesting values like / is not
handled by the code. This being a configure-time option (e.g. to be
set by distribution packagers), we assume the person building the code
knows what they are doing.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

- Update docstrings to explicitely mention Epoch
- Fix timezone bug in FormatTimestampWithTZ, where it would
  use GMT/UTC when it should use the local timezone
- Add unittests for time formatting functions

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

It'll be used for querying locks.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Also replace “sorted” with “utils.NiceSort” now that it supports a key
function.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Point out that jobs already submitted continue to run.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

If the socket can't be read in time, it raises “socket.timeout”, for
which there is special handling code. Unfortunately the exception block
was in the wrong order and “socket.error” caught it before.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* devel-2.3:
  Prepare 2.3.1 release
  Fix disk status verification in LUClusterVerify

Conflicts:
	NEWS: Trivial

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* stable-2.3:
  Prepare 2.3.1 release
  Fix disk status verification in LUClusterVerify

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

pylint is not yet included as the code needs some work for that.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

As we can't test this on master anymore (if we flag the node offline we
would change master role on master) we use the first non master node we
find in the configuration

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch makes sure we cross verify the state the node is
in with our view:

power off -> Node has to be set offline
modify -O no -> Node has to be powered

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* devel-2.3:
  QA: Run cluster-verify as part of all instance tests
  QA: Fix typo and add “not”
  ensure-dirs: Speed up when using big queues
  Fix gnt-cluster verify with diskless instances

Conflicts:
	lib/cmdlib.py: Trivial
	qa/ganeti-qa.py: Trivial

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This patch changes utils.NiceSort to use the built-in “sorted()” and
gets rid of the intermediate list. Instead of wrapping the items
ourselves, a key function is used. The caller can specify another key
function (useful to sort objects by their name, e.g.
“utils.NiceSort(instances, key=operator.attrgetter("name"))”.

Unittests are provided.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

“gnt-cluster verify” looks at some per-instance information as well, so
it should be run for each instance type QA tests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

For secondary node that is offline, we should not consider that the
disk shutdown has failed, as it can never succeed under this cluster
state and (by virtue of the fact that the secondary node is offline)
the disks are already "shutdown".

The patch also fixes a tiny typo.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

data ≫ code, eom.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The “ensure-dirs” script as included in Ganeti 2.3 is very slow when
working with big queues requiring a change of permissions on many or all
files.

$ find /var/lib/ganeti/queue/ | wc -l
52354

Before this change:
$ time /usr/local/lib/ganeti/ensure-dirs -f
real    16m4.739s

While not adressed in this patch, I'd like to record the overall
ineffiency of the “ensure-dirs” script, even after this change:

$ time /usr/local/lib/ganeti/ensure-dirs -f
real    5m57.362s
[…]
$ strace -e clone,execve -f -c /usr/local/lib/ganeti/ensure-dirs -f
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 50.08    5.147090          49    104774           clone
 49.92    5.131094          49    104739           execve

More changes will be needed. Just for comparision, a small Python
snippet changing permissions on all files (“ensure-dirs” changes the
owner too):

$ time python -c 'import os; from ganeti import utils;
[os.chmod(i, 0644) for i in
utils.ListVisibleFiles("/var/lib/ganeti/queue/archive/big")]'
real    0m0.605s
[…]

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

`gnt-cluster verify` was failing with KeyError if there was any
diskless instance in the cluster. This was because _CollectDiskInfo()
was not including these instances in the returned dictionary, but they
were expected to be present in LUVerifyCluster.Exec().

With this commit, we ensure that the dictionary returned by _CollectDiskInfo
includes entries for diskless instances as well.

Signed-off-by: Adeodato Simo <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

The error contained a typo and is slightly cumbersome. It changes from:

- ERROR: node a: not enough memory on to accommodate failovers should peer node
  b fail

to:

- ERROR: node a: not enough memory to accomodate instance failovers should node
  b fail

Signed-off-by: Miguel Di Ciurcio Filho <miguel.filho@gmail.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

* devel-2.3:
  jqueue: Keep jobs in “waitlock” while returning to queue
  Improve jqueue unittests
  Update manpages to display version 2.3

Conflicts:
	man/ganeti-cleaner.sgml: Removed
	man/ganeti-confd.sgml: Removed
	man/ganeti-masterd.sgml: Removed
	man/ganeti-noded.sgml: Removed
	man/ganeti-os-interface.sgml: Removed
	man/ganeti-rapi.sgml: Removed
	man/ganeti-watcher.sgml: Removed
	man/ganeti.sgml: Removed
	man/gnt-backup.sgml: Removed
	man/gnt-cluster.sgml: Removed
	man/gnt-debug.sgml: Removed
	man/gnt-instance.sgml: Removed
	man/gnt-job.sgml: Removed
	man/gnt-node.sgml: Removed
	man/gnt-os.sgml: Removed

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>