Commit 5d9bfd87 moved tap interface handling from KVM to Ganeti, partly
to also solve the problem of routed interfaces getting configured too
early during live migrations, causing network anomalies. In that
direction, configuration of NICs of incoming instances was deferred to
FinalizeMigration time.

However, this causes minor issues with bridged interfaces; KVM sends out
an ARP-like packet upon migration finish, which is lost because the tap
interface is not yet configured. As a consequence, intermediate network
equipment (i.e. switches) does not get notified about the topology
change, until the instance transmits another packet after the bridge has
been configured, or the switch's ARP cache expires.

The proper solution to that is to support different phases in network
configuration (pre/post migration), which also requires separate ifup
scripts. Until then we fall back to configuring bridged interfaces on
incoming instances at migration start, instead of finish.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Currently the drbd8 replace-disks on the same node (i.e. -p or -s) has
a bug in that it does modify the instance disk temporarily before
changing it back to the same value. However, we don't need to, and
shouldn't do that: what this operation do is simply change the LVM
configuration on the node, but otherwise the instance disks keep the
same configuration as before.

In the current code, this change back-and-forth is fine *unless* we
fail during attaching the new LVs to DRBD; in which case, we're left
with a half-modified disk, which is entirely wrong.

So we change the code in two ways:

- use temporary copies of the disk children in the old_lvs var
- stop updating disk.children

Which means that the instance should not be modified anymore (except
maybe for SetDiskID, which is a legacy and unfortunate decision that
will have to cleaned up sometime).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Under newer kvm this prevents the vm from starting.
Ah, change!

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

When using the pool security model, _ExecuteKVMRuntime was storing the
instance's UID using str(uid), which would result in storing the
LockedUid.__repr__() result:

 $ cat /var/run/ganeti/kvm-hypervisor/uid/xxxxxxxxxxxxx
 <ganeti.uidpool.LockedUid object at 0x1f30610>

This patch restores the intended behaviour, by using LockedUid.AsStr().

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This was left out during the fix/refactoring

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

- Move the calculation at the beginning of CheckPrereq, since it doesn't
  modify any state, but still keeps locks
- Only perform the calculation if the actual disk template is filebased
- Error out if there is no defined file storage dir
- Only join the optional --file-storage-dir extra-path if one is passed

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

As the manpage says, and the code does, self.op.file_storage_dir is an
additional relative path under the cluster file storage dir. As such it
should not be absolute.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

When a job was cancelled, its status would be changed and the file
written again. Since this was a final status, the job file could be
moved anytime for archival. If the job was still in the queue, however,
it would be processed (not fully, just updating the “end_timestamp”
attribute) and written again. This was bad as it could leave the same
job in two different files.

With this patch the processor is changed to return early for finished
jobs. Cancelling a queued job will finalize it right away. Unittests are
updated.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

ConfigWriter.ReserveLV() and Configwriter.ReserveMAC() called
TemporaryReservationManager.Reserve() with the ec_id and resource arguments
swapped. As a result, two reservation attempts for the same resource type
within the same LU would fail, even if the resources requested were different,
e.g.:

  $ gnt-instance add -t sharedfile -o debootstrap+default \
       --net 0:mac=00:01:02:03:04:00 \
       --net 1:mac=00:01:02:03:04:ff \
       --disk 0:size=2g  test_instance
  Failure: prerequisites not met for this operation:
  error type: resource_not_unique, error details:
  MAC address 00:01:02:03:04:ff already in use in cluster

This patch fixes the argument order in the call to Reserve().

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Commit 1bee66f3 added assertions for ensuring only the necessary locks
are kept while replacing disks. One of them makes sure locks have been
released during the operation. Unfortunately the commit added the check
as part of a “finally” branch, which is also run when an exception is
thrown (in which case the locks may not have been released yet). Errors
could be masked by the assertion error. Moving the check out of the
“finally” branch fixes the issue.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Currently we generate an empty list only for the '-n node' invocation,
but for iallocator we still call the iallocator (which needs an RPC
call, etc.). By moving the computation of instances outside of the if
block, we can return early from the LU.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

In many cases the opcode ID was incorrect. A unittest for this will
be added in the master branch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

"This can be overriden" can be read as either the port we listen on or
the address we bind to. Replace with "The port" for great clarity!

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Instead of hardcoded Xen commands. This will make it work for all
hypervisors, instead of duplicating hypervisor functionality in QA
itself.

The timeout has been removed as gnt-instance stop itself will make
sure the instance is down before returning. We just double-check that
it is indeed down.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This will allow stopping or starting an instance without changing the
remembered state. While this seems counter-intuitive at first (it will
create cluster verify errors), it can help in a few corner cases:

- shutting down an entire cluster for maintenance but without having
  to remember state
- doing testing of Ganeti itself

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

I think we should stop finding bugs and instead release this :)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The opcode parameter ignore_consistency was used in the LU, but not
actually declared in the OpCode. The patch adds it in the opcode and
the command line client.

ObQuote — Please, please, can I have static typing?

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Two opcodes already use it and we need it for a third, time to add a
constant for it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This encoding, part of the standard Python installation, is used by
the pickle module (in turn used by subprocess when handling
failures in program execution). Preloading it means that Python will
cache it in memory so that even if the disk goes away or just the
module, we're not going to fail in reporting errors.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

The code uses the disk object's “mode” attribute, which uses the
constants DISK_RDONLY (“ro”) and DISK_RDWR (“rw”).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

There are multiple bugs with the code checking for N+1 failures in the
instance memory changes which needs significant changes, in the
meantime we can at least:

- change the warning message into an error (--force will skip checks)
- only make checks when we increase the memory

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Hi all,
this patch will add 3 new KVM parameters and a new option.

New Parameters:
 - floppy_image_path = "" -> Specify the floppy image to load as
floppy disk.
 - cdrom2_image_path = "" -> Specify a second cdrom image to load on
the system (note: this in not intended to be used as a boot device. To
boot the system from cdrom you must use the "cdrom_image_path"
parameter as always).
 - cdrom_disk_type = "" -> it can be one of the kvm supported types as
"ide,scsi,paravirtual,ecc". I introduced this optional parameter to
make possible to specify a different virtual device for cdroms. It is
useful if you want to install a windows system

New option for "boot_device" parameter:
 -  "floppy": with this value you should be able to boot a KVM
instance from floppy image.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit cc130cc7)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

A simple doc patch to document how to configure the kernels for the
instances.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This will allow us an easy migration to pv-grub, because a set root_path
confused pv-grub.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This adds documentation for both the short and long form of many
options (which was inconsistent before: in some cases only the short
form was used, in others only the long form).

Note that the standard this patch adopts is to document both forms as
such:

  {-O|--os-parameters} …

This makes it a bit uglier in complex situations, but the alternatives
considered were not perfect either. Other suggestions (with patches)
welcome.

Additionally, it fixes two doc bugs:

- in gnt-cluster.rst, the --prealloc-wipe-disks section was in the
  middle of a paragraph
- in gnt-instance.rst, a list was not typed correctly, thus it was
  mangled as a single paragraph

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add INSTANCE_PRIMARY_NODE and INSTANCE_SECONDARY_NODES. These new
values are useful for OS scripts that needs to know the nodes where
the instance lives.. or has lived.

Signed-off-by: Iustin Pop <iustin@google.com>
[iustin@google.com: fixed small issue with SECONDARY_NODES]
Reviewed-by: Iustin Pop <iustin@google.com>

Currently, when converting an instance from plain to DRBD, the
instance is blocked during the entire resync period. This patch adds
the --no-wait-for-sync so that the operation finishes as soon as the
DRBD sync has started, without waiting for the entire sync. This makes
the instance available much faster.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This patch introduces the option of changing an instance's nodes when
doing the disk recreation. The rationale is that currently if an
instance lives on a node that has gone down and is marked offline,
it's not possible to re-create the disks and reinstall the instance on
a different node without hacking the config file.

Additionally, the LU now locks the instance's nodes (which was not
done before), as we most likely allocate new resources on them.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>