Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Support:
 - gnt-network add --tags.
 - gnt-network list-tags/add-tags/remove-tags/.
 - gnt-network list -o +tags

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

If a nic has a network field then encapsulate a network object in
netinfo slot for every rpc. This is needed to pass network info to
scripts managing nics (kvm-vif-bridge).

Introduce _BuildNetworkEnv().

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Add networks slot in config data.
Add networks slot in nodegroups.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Modify existing GenerateMAC so that it takes network as an argument.

Substitute _GenerateOneMAC with a decorator _GenerateMACPrefix that
chooses the prefix (network's or cluster's) and wraps
_GenerateMACSuffix that generates the three remaining bytes.

Check mac prefix validity when adding/modifing a network.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Support:
 - GetNetwork(s)
 - CreateNetwork
 - ConnectNetwork
 - DisconnectNetwork
 - RemoveNetwork

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

gnt-network is used to manipulate and handle networks that currently
provides the following operations:

 * Add a new network:
   gnt-network add --network=1.2.3.0/28 --gateway=1.2.3.1
                   --add-reserved-ips=1.2.3.4,1.2.3.5 testnet

 * Remove an existing network:
   gnt-network remove testnet

 * Modify an existing network:
   gnt-network modify --gateway=1.2.3.6
                      --network-type=private
                      --network6=2001:648::/64
                      --gateway6=none testnet
                      --add-reserved-ips=1.2.3.10,1.2.3.10,
                      --remove-reserved-ips=1.2.3.20
                      testnet

 * Connect an existing network to a nodegroup:
   gnt-network connect testnet default bridged br100
   gnt-network connect testnet <nodegroup> <mode> <link>
   (pass all for <nodegroup> to connect to all nodegroups)

 * Disconnect an existing network from a nodegroup:
   gnt-network disconnect testnet <nodegroup>
   (pass all for <nodegroup> to disconnect from all nodegroups)

 * List available networks:
   gnt-network list

 * Show network info:
   gnt-network info [testnet]

Introduce new option NOCONFLICTSCHECK_OPT for not checking for
conflicting IPs. Using this might cause data inconsistency.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Add --no-conflicts-check option.
Modify instance query methods to return nic's network info.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Specifically:
 * export a new environment variable for the gnt-os-interface
 * export a new environment variable for KVM specific scripts
 * ovf support for networks

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Keep backwards compatibility just like LUInstanceCreate() and support
network parameter for NIC object.

Add ec_id argument in Update() to be able to commit temporary IPs that
were previously reserved.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Release any IPs held the instance before actually removing.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Implement backend support, to export the IP pool management
functionality to the clients. When the new NIC parameter 'network' is
given, the ippool management system is triggered.  If a NIC belongs to
a network, it inherits the netparams (mode, link) as its nicparams. If
a network is requested, then no 'mode' or 'link' should be
specified. Backwards compatibility is preserved, if 'network'
parameter is omitted.

Examples for IP pool management system:

--net 0:network=net1
  for an IPless NIC in a network

--net 0:ip=pool,network=net1
  for automatically generated IP inside the network

--net 0:ip=1.2.3.4,network=net1
  for specific IP inside the network

--net 0:network=none
  for IPless NIC with default nicparams

Examples for traditional assignment:

--net 0:ip=1.2.3.4,link=br100,mode=bridged
--net 0:ip=none,link=rt2,mode=routed
--net 0

Check for conflicting IPs to ensure IP uniqueness inside nodegroups.

IAllocator should be extended to become network aware. If an instance
has a NIC inside a network, then IAllocator must pick a node that its
nodegroup is already connected with that network.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Use TemporaryReservationManager for IP handling to ensure config data
consistency.

Implement:
 - methods for reserving/releasing IPs
 - _UnlockedCommitTemporaryIps()
 - CheckIPInNodeGroup() used for conflicting IPs
 - GetGroupNetParams() used for inheriting netparams.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

LUNetworkConnect() connects a network to a nodegroup and defines
its netparams (mode and link). Specifically:
 * Check if network already exists
 * Check if netparams are valid
 * Check if already connected to the same nodegroup
 * Produce errors if any conflicting IPs found
 * Update the 'networks' slot in NodeGroup config object
   with the network UUID and netparams requested. E.g.
   nodegroup_uuid {
     networks {
       network_uuid: {mode: 'bridged', link: 'br100'}
     }
   }

LUNetworkDisconnect() unmaps a network from a nodegroup:
 * Produce errors if an instance has a NIC with IP inside the network
 * Remove corresponding entry in 'networks' slot of NodeGroup object

Check for conflicting ips can be avoided with corresponding option.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Support modifying all network parameters except for network (ip
range). Cannot modify gateway and reserved ips at the same time.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Summarily list all existing networks
Supply detailed info for every existing network
 - List used/free IPs
 - List instances with NICs assigned to the corresponding network
 - List NIC index and IP for the above instances

Implement complementary config methods for retrieving networks.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Implement LUs for corresponding opcodes:
 * LUNetworkAdd:
   - Check for IP validity
   - Reserves all necessary IPs
   - Create new Network config object
 * LUNetworkRemove:
   - Checks if connected to any nodegroup
   - Remove a Network config object

Implement basic config methods:
 * LookupNetwork()
   - Given the network name return the network UUID
 * AddNetwork()
   - Add a new network to the config
 * RemoveNetwork()
   - Remove a network from the config

Add new locking level: LEVEL_NETWORK

Add various useful config methods for retrieving network info.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Add new library module lib/network.py.
Introduce new class: AddressPool.

AddressPool implements all operations needed for managing IPs inside
the IP pool.

Given a Network config object (nobj), the class:

 * initializes the corresponding IP pool object via
   network.AddressPool.InitializeNetwork(nobj)
 * obtains the corresponding IP pool object via
   network.AddressPool(nobj)
 * manipulates IPs inside the pool

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

Config objects:
 * Introduce L{Network} with
  - IPv4 network field (mandatory)
  - IPv4 gateway, IPv6 (network/gateway), mac prefix, type (optional)
 * Modify existing config objects to support networks:
  - Add new slot 'network' to L{NIC} config object
  - Add new slot 'networks' to L{NodeGroup} config object

Opcodes:
 * Introduce new opcodes for networks
  - add/remove/modify/query/connect/disconnect.
 * In InstanceCreate/InstanceSetParams add conflicts_check option

Constants:
 * INIC_PARAM 'INIC_NETWORK'
 * NIC_IP_POOL for automaticaly obtain an IP from a pool
 * NETWORK_TYPE_PUBLIC/PRIVATE for network types

Checking of network_type handled by the opcode parameter validation.
Introduce _CheckCIDR*Notation() functions for network parameters
validation.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

This mirrors the ht.py types PositiveInt, NonNegative, etc., except
that they work at a more generic level (any numeric type, respectively
any non-empty list).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

This is yet-another version-specific warning (fixed in later hlint
versions), but it's better to keep it clean.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This adds tests that existing test files can be parsed by the Haskell
parser as well, plus one new test file.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

A new directory for haskell modules about block devices has been created
The parser is divided in two modules:
* one exports the data types describing the DRBD status
* one exports the parser itself

Signed-off-by: Michele Tartara <mtartara@google.com>
[iustin@google.com: indentation/alignment fixes]
Reviewed-by: Iustin Pop <iustin@google.com>

While writing the pipe-based reporting and trying various ways to
break the startup, I fought for a while trying to understand why error
reporting was _different_ when running the daemon as a user (with no
rights). It turns out that setupDaemonFDs wants to open the log file
in append mode way before, so we are not protected by the 'prepare'
phase.

This patch explicitly runs the 'setupDaemonFDs' function under the
same handler as the prepare phase, with the only change that here we
instruct handlePrepErr to not log the message via log*, since logging
is not yet set up.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently, the OpCode definitions are using normal constructors:

  data OpCode = OpTestDelay Double Bool [String]
              | OpInstanceFailover String Bool (Maybe String)
              …

While this works for a few opcodes, it becomes unwieldy when dealing
with a bigger number of opcode definitions and/or with opcodes having
many fields.

This patch changes the opcodes to record-based constructors, so that
we get for free accessor functions:

  data OpCode
    = OpTestDelay {
        opDuration :: Double,
        opOnMaster :: Bool,
        opOnNodes :: [String]
      }
      | OpInstanceFailover {
         opInstanceName :: String,
         opIgnoreConsistency :: Bool,
         opTargetNode :: Maybe String
      }
      …

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

While looking at the opcode docs and clicking accidentally on the
filter type, I saw that the haddock formatting is broken due to
non-escaped use of special chars. Let's convert the ascii-like
formatting to haddock, and have nicer apidoc.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Use more monadic combinators instead of explicit code.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

Currently, the cluster verify errors are defined as follows:

CV_ECLUSTER_FOO = (TCLUSTER, "ECLUSTER_FOO", "description")

This means there's no standalone name for the string "ECLUSTER_FOO",
which makes it hard to derive automatically a type for this union in
Haskell.

There are three possible fixes:

- manually separate the CV_ECLUSTER_FOO_STR = "ECLUSTER_FOO" in
  constants.py
- manually extract the strings in Haskell code
- change convert-constants to automatically export virtual constants
  for those

After discussion on IRC, I've taken the latter approach; even though a
bit hack-ish, it avoids manual work and potential errors.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

When given an empty string as value to a parameter that doesn't accept
empty strings, the error message was confusing, since it was showing
just as a double space.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

Currently, ht.py uses a bad terminology for positive/non-negative
numbers. Per http://en.wikipedia.org/wiki/Positive_number

, this is the
correct terminology:

- A number is positive if it is greater than zero.
- A number is negative if it is less than zero.
- A number is non-negative if it is greater than or equal to zero.
- A number is non-positive if it is less than or equal to zero.

So this patch renames things as follows:

- TPositiveInt            ⇒ TNonNegativeInt
- TStrictPositiveInt      ⇒ TPositiveInt
- TMaybePositiveInt       ⇒ dropped, not used anywhere
- TMaybeStrictPositiveInt ⇒ TMaybePositiveInt
- TPositiveFloat          ⇒ TNonNegativeFloat
- TStrictNegativeInt      ⇒ TNegativeInt

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Due to TemplateHaskell stage restrictions, we can't define parameters
in the same module as we're using them for TH, so we have to define
all module parameters in a separate module.

This patch therefore splits OpCodes.hs in two, adding that module and
moves most code there (types, parameters, etc.). The remaining parts
in OpCodes.hs, the actual opcode definitions, now use more parameters
instead of direct field definitions (more will come later)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

There are already three cases where we copied type definitions between
the htools-specific types into the main ganeti code. Let's stop doing
this ☺️

 and create a common types module that holds these.

Note that there already exists BasicTypes.hs, but that refers to very
low-level types, and can't use TH derivation itself.

A side effect of this unification is that there is a small conflict
between AdminStatus/AdminOffline and InstanceStatus/AdminOffline. As
such, I renamed AdminOffline and AdminDown to StatusOffline/StatusDown
in the InstanceStatus type.

The patch also moves the tests related to these types to a new test
module.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>