1. 20 Nov, 2012 21 commits
    • Dimitris Aragiorgis's avatar
      Modify LUInstanceCreate to support networks · e8e079f3
      Dimitris Aragiorgis authored
      
      
      Implement backend support, to export the IP pool management
      functionality to the clients. When the new NIC parameter 'network' is
      given, the ippool management system is triggered.  If a NIC belongs to
      a network, it inherits the netparams (mode, link) as its nicparams. If
      a network is requested, then no 'mode' or 'link' should be
      specified. Backwards compatibility is preserved, if 'network'
      parameter is omitted.
      
      Examples for IP pool management system:
      
      --net 0:network=net1
        for an IPless NIC in a network
      
      --net 0:ip=pool,network=net1
        for automatically generated IP inside the network
      
      --net 0:ip=1.2.3.4,network=net1
        for specific IP inside the network
      
      --net 0:network=none
        for IPless NIC with default nicparams
      
      Examples for traditional assignment:
      
      --net 0:ip=1.2.3.4,link=br100,mode=bridged
      --net 0:ip=none,link=rt2,mode=routed
      --net 0
      
      Check for conflicting IPs to ensure IP uniqueness inside nodegroups.
      
      IAllocator should be extended to become network aware. If an instance
      has a NIC inside a network, then IAllocator must pick a node that its
      nodegroup is already connected with that network.
      Signed-off-by: default avatarApollon Oikonomopoulos <apollon@noc.grnet.gr>
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      Reviewed-by: default avatarIustin Pop <iustin@google.com>
      e8e079f3
    • Dimitris Aragiorgis's avatar
      Config methods for reserving/releasing IPs · ad4a9ae7
      Dimitris Aragiorgis authored
      
      
      Use TemporaryReservationManager for IP handling to ensure config data
      consistency.
      
      Implement:
       - methods for reserving/releasing IPs
       - _UnlockedCommitTemporaryIps()
       - CheckIPInNodeGroup() used for conflicting IPs
       - GetGroupNetParams() used for inheriting netparams.
      Signed-off-by: default avatarApollon Oikonomopoulos <apollon@noc.grnet.gr>
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      Reviewed-by: default avatarIustin Pop <iustin@google.com>
      ad4a9ae7
    • Dimitris Aragiorgis's avatar
      Implement network/nodegroup mappings · 0fbd466b
      Dimitris Aragiorgis authored
      
      
      LUNetworkConnect() connects a network to a nodegroup and defines
      its netparams (mode and link). Specifically:
       * Check if network already exists
       * Check if netparams are valid
       * Check if already connected to the same nodegroup
       * Produce errors if any conflicting IPs found
       * Update the 'networks' slot in NodeGroup config object
         with the network UUID and netparams requested. E.g.
         nodegroup_uuid {
           networks {
             network_uuid: {mode: 'bridged', link: 'br100'}
           }
         }
      
      LUNetworkDisconnect() unmaps a network from a nodegroup:
       * Produce errors if an instance has a NIC with IP inside the network
       * Remove corresponding entry in 'networks' slot of NodeGroup object
      
      Check for conflicting ips can be avoided with corresponding option.
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      Reviewed-by: default avatarIustin Pop <iustin@google.com>
      0fbd466b
    • Dimitris Aragiorgis's avatar
      Implement LUNetworkSetParams · 1e0d3321
      Dimitris Aragiorgis authored
      
      
      Support modifying all network parameters except for network (ip
      range). Cannot modify gateway and reserved ips at the same time.
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      Reviewed-by: default avatarIustin Pop <iustin@google.com>
      1e0d3321
    • Apollon Oikonomopoulos's avatar
      Implement LUNetworkQuery · 306bed0e
      Apollon Oikonomopoulos authored
      
      
      Summarily list all existing networks
      Supply detailed info for every existing network
       - List used/free IPs
       - List instances with NICs assigned to the corresponding network
       - List NIC index and IP for the above instances
      
      Implement complementary config methods for retrieving networks.
      Signed-off-by: default avatarApollon Oikonomopoulos <apollon@noc.grnet.gr>
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      Reviewed-by: default avatarIustin Pop <iustin@google.com>
      306bed0e
    • Dimitris Aragiorgis's avatar
      Basic IP pool management logic · 6c0a75db
      Dimitris Aragiorgis authored
      
      
      Implement LUs for corresponding opcodes:
       * LUNetworkAdd:
         - Check for IP validity
         - Reserves all necessary IPs
         - Create new Network config object
       * LUNetworkRemove:
         - Checks if connected to any nodegroup
         - Remove a Network config object
      
      Implement basic config methods:
       * LookupNetwork()
         - Given the network name return the network UUID
       * AddNetwork()
         - Add a new network to the config
       * RemoveNetwork()
         - Remove a network from the config
      
      Add new locking level: LEVEL_NETWORK
      
      Add various useful config methods for retrieving network info.
      Signed-off-by: default avatarApollon Oikonomopoulos <apollon@noc.grnet.gr>
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      Reviewed-by: default avatarIustin Pop <iustin@google.com>
      6c0a75db
    • Apollon Oikonomopoulos's avatar
      Introduce new module for IP pool management · 1de1cf25
      Apollon Oikonomopoulos authored
      
      
      Add new library module lib/network.py.
      Introduce new class: AddressPool.
      
      AddressPool implements all operations needed for managing IPs inside
      the IP pool.
      
      Given a Network config object (nobj), the class:
      
       * initializes the corresponding IP pool object via
         network.AddressPool.InitializeNetwork(nobj)
       * obtains the corresponding IP pool object via
         network.AddressPool(nobj)
       * manipulates IPs inside the pool
      Signed-off-by: default avatarApollon Oikonomopoulos <apollon@noc.grnet.gr>
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      Reviewed-by: default avatarIustin Pop <iustin@google.com>
      1de1cf25
    • Dimitris Aragiorgis's avatar
      IP pool related objects, opcodes and constants · eaa4c57c
      Dimitris Aragiorgis authored
      
      
      Config objects:
       * Introduce L{Network} with
        - IPv4 network field (mandatory)
        - IPv4 gateway, IPv6 (network/gateway), mac prefix, type (optional)
       * Modify existing config objects to support networks:
        - Add new slot 'network' to L{NIC} config object
        - Add new slot 'networks' to L{NodeGroup} config object
      
      Opcodes:
       * Introduce new opcodes for networks
        - add/remove/modify/query/connect/disconnect.
       * In InstanceCreate/InstanceSetParams add conflicts_check option
      
      Constants:
       * INIC_PARAM 'INIC_NETWORK'
       * NIC_IP_POOL for automaticaly obtain an IP from a pool
       * NETWORK_TYPE_PUBLIC/PRIVATE for network types
      
      Checking of network_type handled by the opcode parameter validation.
      Introduce _CheckCIDR*Notation() functions for network parameters
      validation.
      Signed-off-by: default avatarApollon Oikonomopoulos <apollon@noc.grnet.gr>
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
      Reviewed-by: default avatarIustin Pop <iustin@google.com>
      eaa4c57c
    • Iustin Pop's avatar
      Add more basic validation types · edb5a1c8
      Iustin Pop authored
      
      
      This mirrors the ht.py types PositiveInt, NonNegative, etc., except
      that they work at a more generic level (any numeric type, respectively
      any non-empty list).
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarAdeodato Simo <dato@google.com>
      edb5a1c8
    • Iustin Pop's avatar
      Fix a hlint warning · 469a1490
      Iustin Pop authored
      
      
      This is yet-another version-specific warning (fixed in later hlint
      versions), but it's better to keep it clean.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarGuido Trotter <ultrotter@google.com>
      469a1490
    • Michele Tartara's avatar
      Add DRBD parser unit tests · a52f8e72
      Michele Tartara authored
      
      
      This adds tests that existing test files can be parsed by the Haskell
      parser as well, plus one new test file.
      Signed-off-by: default avatarMichele Tartara <mtartara@google.com>
      Reviewed-by: default avatarIustin Pop <iustin@google.com>
      a52f8e72
    • Michele Tartara's avatar
      Add parser for DRBD /proc file · 3c1915df
      Michele Tartara authored
      
      
      A new directory for haskell modules about block devices has been created
      The parser is divided in two modules:
      * one exports the data types describing the DRBD status
      * one exports the parser itself
      Signed-off-by: default avatarMichele Tartara <mtartara@google.com>
      [iustin@google.com: indentation/alignment fixes]
      Reviewed-by: default avatarIustin Pop <iustin@google.com>
      3c1915df
    • Iustin Pop's avatar
      Properly log errors when setting up daemon FDs · 1a865afe
      Iustin Pop authored
      
      
      While writing the pipe-based reporting and trying various ways to
      break the startup, I fought for a while trying to understand why error
      reporting was _different_ when running the daemon as a user (with no
      rights). It turns out that setupDaemonFDs wants to open the log file
      in append mode way before, so we are not protected by the 'prepare'
      phase.
      
      This patch explicitly runs the 'setupDaemonFDs' function under the
      same handler as the prepare phase, with the only change that here we
      instruct handlePrepErr to not log the message via log*, since logging
      is not yet set up.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarMichael Hanselmann <hansmi@google.com>
      1a865afe
    • Iustin Pop's avatar
      Switch opcode data type from normal to record constructors · 8ee2994a
      Iustin Pop authored
      
      
      Currently, the OpCode definitions are using normal constructors:
      
        data OpCode = OpTestDelay Double Bool [String]
                    | OpInstanceFailover String Bool (Maybe String)
                    …
      
      While this works for a few opcodes, it becomes unwieldy when dealing
      with a bigger number of opcode definitions and/or with opcodes having
      many fields.
      
      This patch changes the opcodes to record-based constructors, so that
      we get for free accessor functions:
      
        data OpCode
          = OpTestDelay {
              opDuration :: Double,
              opOnMaster :: Bool,
              opOnNodes :: [String]
            }
            | OpInstanceFailover {
               opInstanceName :: String,
               opIgnoreConsistency :: Bool,
               opTargetNode :: Maybe String
            }
            …
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarAdeodato Simo <dato@google.com>
      8ee2994a
    • Iustin Pop's avatar
      Fix docstrings for the Filter type · 51d991d7
      Iustin Pop authored
      
      
      While looking at the opcode docs and clicking accidentally on the
      filter type, I saw that the haddock formatting is broken due to
      non-escaped use of special chars. Let's convert the ascii-like
      formatting to haddock, and have nicer apidoc.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarMichael Hanselmann <hansmi@google.com>
      51d991d7
    • Iustin Pop's avatar
      Simplify a few test helpers · 32f2e1e1
      Iustin Pop authored
      
      
      Use more monadic combinators instead of explicit code.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarAdeodato Simo <dato@google.com>
      32f2e1e1
    • Iustin Pop's avatar
      Add custom code for CV_E* constants to convert-constants · 9ba02574
      Iustin Pop authored
      
      
      Currently, the cluster verify errors are defined as follows:
      
      CV_ECLUSTER_FOO = (TCLUSTER, "ECLUSTER_FOO", "description")
      
      This means there's no standalone name for the string "ECLUSTER_FOO",
      which makes it hard to derive automatically a type for this union in
      Haskell.
      
      There are three possible fixes:
      
      - manually separate the CV_ECLUSTER_FOO_STR = "ECLUSTER_FOO" in
        constants.py
      - manually extract the strings in Haskell code
      - change convert-constants to automatically export virtual constants
        for those
      
      After discussion on IRC, I've taken the latter approach; even though a
      bit hack-ish, it avoids manual work and potential errors.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarMichael Hanselmann <hansmi@google.com>
      9ba02574
    • Iustin Pop's avatar
      Improve error message for wrong opcode parameter values · e1ebbfcf
      Iustin Pop authored
      
      
      When given an empty string as value to a parameter that doesn't accept
      empty strings, the error message was confusing, since it was showing
      just as a double space.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarAdeodato Simo <dato@google.com>
      e1ebbfcf
    • Iustin Pop's avatar
      Cleanup ht's use of positive/strictpositive · 2c9fa1ff
      Iustin Pop authored
      Currently, ht.py uses a bad terminology for positive/non-negative
      numbers. Per http://en.wikipedia.org/wiki/Positive_number
      
      , this is the
      correct terminology:
      
      - A number is positive if it is greater than zero.
      - A number is negative if it is less than zero.
      - A number is non-negative if it is greater than or equal to zero.
      - A number is non-positive if it is less than or equal to zero.
      
      So this patch renames things as follows:
      
      - TPositiveInt            ⇒ TNonNegativeInt
      - TStrictPositiveInt      ⇒ TPositiveInt
      - TMaybePositiveInt       ⇒ dropped, not used anywhere
      - TMaybeStrictPositiveInt ⇒ TMaybePositiveInt
      - TPositiveFloat          ⇒ TNonNegativeFloat
      - TStrictNegativeInt      ⇒ TNegativeInt
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarMichael Hanselmann <hansmi@google.com>
      2c9fa1ff
    • Iustin Pop's avatar
      Split OpCode.hs and add module for opcode parameters · 92f51573
      Iustin Pop authored
      
      
      Due to TemplateHaskell stage restrictions, we can't define parameters
      in the same module as we're using them for TH, so we have to define
      all module parameters in a separate module.
      
      This patch therefore splits OpCodes.hs in two, adding that module and
      moves most code there (types, parameters, etc.). The remaining parts
      in OpCodes.hs, the actual opcode definitions, now use more parameters
      instead of direct field definitions (more will come later)
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarAdeodato Simo <dato@google.com>
      92f51573
    • Iustin Pop's avatar
      Create a new Ganeti/Types.hs module · 5e9deac0
      Iustin Pop authored
      There are already three cases where we copied type definitions between
      the htools-specific types into the main ganeti code. Let's stop doing
      this 
      
       and create a common types module that holds these.
      
      Note that there already exists BasicTypes.hs, but that refers to very
      low-level types, and can't use TH derivation itself.
      
      A side effect of this unification is that there is a small conflict
      between AdminStatus/AdminOffline and InstanceStatus/AdminOffline. As
      such, I renamed AdminOffline and AdminDown to StatusOffline/StatusDown
      in the InstanceStatus type.
      
      The patch also moves the tests related to these types to a new test
      module.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarAdeodato Simo <dato@google.com>
      5e9deac0
  2. 19 Nov, 2012 6 commits
  3. 16 Nov, 2012 8 commits
  4. 15 Nov, 2012 4 commits
    • Michael Hanselmann's avatar
      Move gnt_cluster.SHOW_MACHINE_OPT to cli · f92ed8ab
      Michael Hanselmann authored
      
      
      This allows the option to be re-used in other places.
      Signed-off-by: default avatarMichael Hanselmann <hansmi@google.com>
      Reviewed-by: default avatarGuido Trotter <ultrotter@google.com>
      f92ed8ab
    • Guido Trotter's avatar
      Add ganeti-node-role ocf example file · 17071597
      Guido Trotter authored
      
      
      This allows offlining nodes that don't respond if they are part of a
      linux-HA cluster.
      Signed-off-by: default avatarGuido Trotter <ultrotter@google.com>
      Reviewed-by: default avatarMichael Hanselmann <hansmi@google.com>
      17071597
    • Guido Trotter's avatar
      Add ganeti-master-role.ocf example file · aa75500a
      Guido Trotter authored
      
      
      This allows controlling the cluster master role if the nodes are part of
      a linux-HA cluster.
      Signed-off-by: default avatarGuido Trotter <ultrotter@google.com>
      Reviewed-by: default avatarMichael Hanselmann <hansmi@google.com>
      aa75500a
    • Iustin Pop's avatar
      Cleanup THH function use from built module namespace · 32a569fe
      Iustin Pop authored
      
      
      Currently, THH.hs "injects" into the built code names of library
      functions like Text.JSON.makeObj, Ganeti.JSON.fromObj, etc. built
      directly from strings, via (e.g.)
      
        varE (mkName "makeObj")
      
      This means that the "makeObj" name must exist in the target module,
      i.o.w. must be imported there. This leads to the strange case of
      having to have imports that do not appear at all in the used
      (template) code, but are needed to satisfy this "hidden" dependency;
      look at Ganeti/Jobs.hs before this patch, for example.
      
      This is also not very obvious, because we usually import Text.JSON
      anyway; I only stumbled upon it while doing some cleanup work.
      
      So to clean this up, the current patch changes the THH.hs to use not
      string-derived, but identifier-derived names («'identifier» versus
      «mkName "identifier"»); this is better, as the names must be
      resolvable when compiling THH itself (once), and not when compiling
      the multiple derived modules. As you can see, this allows removal of
      extraneous imports from various modules.
      
      Background information: an `mkName "foo"` results in a name of flavour
      NameS (“An unqualified name; dynamically bound”) or alternatively to a
      qualified name, but still dynamically bound. Whereas what we want is a
      statically bound name: `'foo` results in a NameG flavour, “Global name
      bound outside of the TH AST: An original name”.
      
      One more explanation: the change is similar to going from 'x = eval
      "map"' to 'x = map'; the name is no longer dynamically evaluated, but
      statically when the module is compiled. In our case, previously names
      were bound at target module compile time, now they are bound at THH.hs
      compile time.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarGuido Trotter <ultrotter@google.com>
      32a569fe
  5. 14 Nov, 2012 1 commit