Perform suitable pool operations on NIC addition, removal and modification.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Call cluster.SimpleFillNIC to get the default link when link is None.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Change the semantics of ReleaseIp, so that it can be called from outside
without knowing the exact network UUID, by passing only node and link.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Add a new method, TemporaryReservationManager.GetECReserved(), which returns all
reservations held by a given ec_id.

Also add a new method, ConfigWriter._UnlockedCommitReservedIps(), which can be
called to commit all temporarily reserved IP addresses to the respective
address pools. For this to work, the IP reservation manager must hold both, the
address and the network UUID of the respective address pool.

Finally, we automatically commit IP address reservations during
ConfigWriter.Update() of Instance objects. For this to work, we add an optional
ec_id keyword, which must hold the calling job's unique id to get the
relevant reservations.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

ToStdOut -> ToStdout

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

The IP pool management infrastructure currently needs to know an instance's
node to fill in IP addresses. For this to happen, the iallocator must run
before we generate IP addresses, yet the iallocator itself needs the instance's
NIC dictionaries.

We break this cycle by passing ip="pool" to the iallocator and filling-in the
IP address after the iallocator run.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Prototype implementation of IP address management during instance
addition/removal.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

gnt-network manipulates network objects. Required parameters have been added to
ganeti.cli.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Add opcodes for network addition and query.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Modify LUGroupSetParams to accept network addition/removal. Networks are
connected to nodegroups' links.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Add Logical Units for network adding and querying network config objects.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Add unit tests to check proper IP address pool management logic.
The tests cover all simple operations on an IP address pool.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Add NetworkQueryData data container for network-related queries. Also add a
network query field generator and helper functions to generate usage statistics
for IP address pools.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Add helper methods to ganeti.config.ConfigWriter to handle Network addition and
removal, IP address reservation and commitment to the address pools.

Also add the “networks” ssconf key to make network names/UUIDs available to all
nodes via ssconf.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

lib.network contains AddressPool, a helper class for ganeti.objects.Network.
AddressPool wraps a Network object and provides all IPv4 Address Pool
management logic, manipulating the respective fields of the Network object.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Add a network object definition, with the appropriate slots, to lib.objects.

Also add a “networks” top-level configuration slot to host network definitions
and a “networks” slot to the NodeGroup objects to hold network ←→ node-group
mappings.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

A new ip keyword is added ("pool") to signify that LUCreateInstance should get
the instance's IP from an IP pool (rather than manually or by DNS resolution).

IP and link checks are re-ordered so that a NIC's link is available at the time
of IP address validation.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

This patch introduces the following changes to lib.config and lib.objects to
facilitate IP pool management.

- Add a "networks" key to objects.cluster to hold link -> subnet relations.
  Each link is assumed to be associated with at most one IPv4 and at most one
  IPv6 subnet. Currently only IPv4 subnets are used.
- Add a TemporaryReservationManager for IP address reservations.
- Add config.{_UnlockedCommitIp,CommitIp,_UnlockedReleaseIp,ReleaseIp} to write
  IP pool changes to the config file.
- Add config.{ReserveIp,GenerateIp} for use with the _temporary_ips
  TemporaryReservationManager.
- Commit succesful reservations/releases to the IP pools in config.AddInstance
  and config.RemoveInstance

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

This patch introduces primitive structures for IP pool management. The core of
the IP Pool management framework is the ippool.IPv4Network class, which encodes
an IPv4 subnet configuration with its IP address pool. The pool functionality
depends on python-bitarray and python-ipaddr is used for address manipulation.

ippool.IPv4Network has 3 key attributes:
 - net: The IPv4 subnet definition (CIDR notation)
 - gateway: The default gateway used, if any
 - _pool: a bitarray.bitarray storing the reservation status of individual IPs

The purpose of including ippool.IPv4Network in ganeti is two-fold:
 - It will allow including subnet information useful for network configuration
   (e.g. by being available during an OS provider invocation)
 - It will be used for automatic IP address allocation during instance creation
   for both, routed and bridged modes.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Commit 5d9bfd87 moved tap interface handling from KVM to Ganeti, partly to
also solve the problem of routed interfaces getting configured too early during
live migrations, causing network anomalies. In that direction, configuration of
NICs of incoming instances was deferred to FinalizeMigration time.

However, this causes minor issues with bridged interfaces; KVM sends out an
ARP-like packet upon migration finish, which is lost because the tap interface
is not yet configured. As a consequence, intermediate network equipment (i.e.
switches) does not get notified about the topology change, until the instance
transmits another packet after the bridge has been configured, or the switch's
ARP cache expires.

The proper solution to that is to support different phases in network
configuration (pre/post migration), which also requires separate ifup scripts.
Until then we fall back to configuring bridged interfaces on incoming instances
at migration start, instead of finish.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Introduced during merge, because of fba7f911. Remove the second definition, as
it broke KVM HTTP CDROM boot.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

ConfigWriter.ReserveLV() and Configwriter.ReserveMAC() called
TemporaryReservationManager.Reserve() with the ec_id and resource arguments
swapped. As a result, two reservation attempts for the same resource type
within the same LU would fail, even if the resources requested were different,
e.g.:

  $ gnt-instance add -t sharedfile -o debootstrap+default \
       --net 0:mac=00:01:02:03:04:00 \
       --net 1:mac=00:01:02:03:04:ff \
       --disk 0:size=2g  test_instance
  Failure: prerequisites not met for this operation:
  error type: resource_not_unique, error details:
  MAC address 00:01:02:03:04:ff already in use in cluster

This patch fixes the argument order in the call to Reserve().

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Conflicts:
	lib/hypervisor/hv_kvm.py
	lib/opcodes.py
	 - Merge

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

Check that the instance is not being migrated to its current primary node
during CheckPrereq. Otherwise migration is aborted because the instance is
already running and cleaned-up, which causes the running instance to be killed.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>

I think we should stop finding bugs and instead release this :)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The opcode parameter ignore_consistency was used in the LU, but not
actually declared in the OpCode. The patch adds it in the opcode and
the command line client.

ObQuote — Please, please, can I have static typing?

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Two opcodes already use it and we need it for a third, time to add a
constant for it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This encoding, part of the standard Python installation, is used by
the pickle module (in turn used by subprocess when handling
failures in program execution). Preloading it means that Python will
cache it in memory so that even if the disk goes away or just the
module, we're not going to fail in reporting errors.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

The code uses the disk object's “mode” attribute, which uses the
constants DISK_RDONLY (“ro”) and DISK_RDWR (“rw”).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

There are multiple bugs with the code checking for N+1 failures in the
instance memory changes which needs significant changes, in the
meantime we can at least:

- change the warning message into an error (--force will skip checks)
- only make checks when we increase the memory

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Hi all,
this patch will add 3 new KVM parameters and a new option.

New Parameters:
 - floppy_image_path = "" -> Specify the floppy image to load as
floppy disk.
 - cdrom2_image_path = "" -> Specify a second cdrom image to load on
the system (note: this in not intended to be used as a boot device. To
boot the system from cdrom you must use the "cdrom_image_path"
parameter as always).
 - cdrom_disk_type = "" -> it can be one of the kvm supported types as
"ide,scsi,paravirtual,ecc". I introduced this optional parameter to
make possible to specify a different virtual device for cdroms. It is
useful if you want to install a windows system

New option for "boot_device" parameter:
 -  "floppy": with this value you should be able to boot a KVM
instance from floppy image.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit cc130cc7)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>