1. 29 Sep, 2008 3 commits
    • Iustin Pop's avatar
      Move a hardcoded constant to constants.py · 3c03759a
      Iustin Pop authored
      For now we only use the ‘C’ protocol so we can put it in constants.py
      instead of hardcoding it.
      
      Reviewed-by: imsnah
      3c03759a
    • Iustin Pop's avatar
      Enable the use of shared secrets · 2899d9de
      Iustin Pop authored
      This patch enables the use of the shared secrets for DRBD8 disks, using
      (hardcoded in constants.py) the md5 digest algorithm.
      
      For making this more flexible, either we implement a cluster parameter
      (once the new model is in place), or we can make it ./configure-time
      selectable.
      
      Reviewed-by: imsnah
      2899d9de
    • Iustin Pop's avatar
      Extend DRBD disks with shared secret attribute · f9518d38
      Iustin Pop authored
      This patch, which is similar to r1679 (Extend DRBD disks with minors
      attribute), extends the logical and physical id of the DRBD disks with a
      shared secret attribute. This is generated at disk creation time and
      saved in the config file.
      
      The generation of the secret is done so that we don't have duplicates in
      the configuration (otherwise the goal of preventing cross-connection
      will not be reached), so we add to config.py more than just a simple
      call to utils.GenerateSecret().
      
      The patch does not yet enable the use of the secrets.
      
      Reviewed-by: imsnah
      f9518d38
  2. 23 Sep, 2008 1 commit
    • Iustin Pop's avatar
      Switch to static minors for DRBD · a1578d63
      Iustin Pop authored
      With some todos remaining, this patch switches the DRBD devices to use
      the passed minors, and the cmdlib code (add instance and replace disks)
      to request and assign minors to the DRBD disks.
      
      Todos:
        - look at the disk RPC calls to see which can be optimized away, since
          we now know the minor beforehand
        - remove the _FindUnusedMinor usage from the few places it's still
          used (not for actual disks, but for temporary use in meta devs) and
          eventually replace with _CheckMinorUnused or such
      
      Of course, this and/or the previous two patches break existing clusters.
      Again.
      
      Reviewed-by: imsnah
      a1578d63
  3. 22 Sep, 2008 1 commit
    • Iustin Pop's avatar
      Extend DRBD disks with minors attribute · ffa1c0dc
      Iustin Pop authored
      This patch converts the DRBD disks to contain also a minor (per each
      node) attribute. This minor is not yet used and is always initialized
      with None, so the patch does not have any real-world impact - except for
      automatically upgrading config files (it adds the minors as None, None).
      
      Reviewed-by: imsnah
      ffa1c0dc
  4. 09 Jul, 2008 2 commits
    • Iustin Pop's avatar
      Reduce duplicate Attach() calls in bdev · cb999543
      Iustin Pop authored
      Currently, the 'public' functions of bdev (FindDevice and
      AttachOrAssemble) will call the Attach() method right after class
      instantiation.
      
      But the constructor itself calls this function, and therefore we have
      duplicate Attach() calls (which are not cheap at all).
      
      The patch introduces a new 'attached' instance attribute that tells if
      the last Attach() was successful. The public functions reuse this so
      that we only do the minimum required number of calls.
      
      Reviewed-by: imsnah
      cb999543
    • Iustin Pop's avatar
      Convert bdev.py to the logging module · 468c5f77
      Iustin Pop authored
      This does not enhance in any way the messages; it just switches to the
      new module.
      
      Reviewed-by: imsnah
      468c5f77
  5. 25 Jun, 2008 1 commit
    • Iustin Pop's avatar
      Cleanup LV status computation · 99e8295c
      Iustin Pop authored
      Currently, when seeing if a LV is degraded or not (i.e. virtual volume),
      we first attach to the device (which does an lvdisplay), then do a lvs
      in order to display the lv_attr. This generates two external commands to
      do (almost) the same thing.
      
      This patch changes the Attach() method for LVs to call lvs and display
      both the major/minor (needed for attach) and the lv_status (needed for
      GetSyncStatus). Thus, later in GetSyncStatus, we don't need to run lvs
      again, and instead just return the value computed in Attach().
      
      Reviewed-by: imsnah
      99e8295c
  6. 18 Jun, 2008 1 commit
    • Iustin Pop's avatar
      Rework the DRBD8 device status computation · 6b90c22e
      Iustin Pop authored
      Currently, compute the status of a drbd8 device in GetSyncStatus and
      return only the values that we need (and fit in the framework of
      GetSyncStatus). However, the full status details are useful (and needed)
      in other places, so the patch attempts to improve this situation.
      
      We abstract the status of a device outside in a separate class, that
      knows how to parse contents from /proc/drbd and set easily accessible
      attributes. We then simplify the GetSyncStatus to use this and return
      the values that it needs, and add a separate method that returns the
      full status object.
      
      The move to a separate class cleans up a little bit the old
      sync-progress computation from GetSyncStatus, but it's still many
      regexes.
      
      The patch also adds unittests for a few statuses, and modifies one
      BaseDRBD call to accept a custom filename instead of '/proc/drbd' to
      ease unittests.
      
      Reviewed-by: imsnah
      6b90c22e
  7. 16 Jun, 2008 1 commit
    • Iustin Pop's avatar
      bdev: implement disk resize for lvm/drbd8 · 1005d816
      Iustin Pop authored
      This patch implements disk resize at the bdev level for the LVM and
      DRBD8 disk types. It is not implemented for DRBD7 and MD since the way
      MD works with its underlaying devices makes it harder and this
      combination is also deprecated.
      
      The LVM resize operation is tried three times, with different allocation
      policies:
        - contiguous first, since this is best for allocation purposes (it
          won't fragment too much the PV)
        - cling, which is supported only by more recent LVM versions, will try
          to place the new extents on the same PV as the rest of the LV
        - and finally normal, which is the default
      
      Reviewed-by: imsnah
      1005d816
  8. 30 May, 2008 1 commit
    • Iustin Pop's avatar
      Complete removal of md/drbd 0.7 code · abdf0113
      Iustin Pop authored
      This patch removes the last of the md and drbd 0.7 code. Cluster which
      have the old device types will be broken if they have this applied.
      
      Reviewed-by: imsnah
      abdf0113
  9. 15 May, 2008 2 commits
    • Guido Trotter's avatar
      Fix drbd show parser to handle valueless keywords · 63012024
      Guido Trotter authored
      It turns out in some cases there can exist keywords without an
      associated value exported by drbdsetup show. This patch makes the value
      part optional in our parser, so that if it's not present the parsing
      result will contain an array with just the keyword in it. This is not a
      problem since we check all keyword names before accessing their values,
      so we won't mistakenly try to access the value of a valueless keyword.
      
      Reviewed-by: iustinp
      
      63012024
    • Guido Trotter's avatar
      Split drbd command creation and execution · 333411a7
      Guido Trotter authored
      Make _AssembleDisk more similar to _AssembleNet by splitting the
      generation of the drbdsetup command and its execution. While not
      changing anything this makes it easier to manipulate the command just in
      certain cases, which in the future we'll need to do.
      
      Reviewed-by: iustinp
      
      333411a7
  10. 12 May, 2008 1 commit
    • Iustin Pop's avatar
      bdev: always log command output if it failed · 6c896e2f
      Iustin Pop authored
      Currently many error handling code paths in bdev.py log only
      result.fail_reason (i.e. exit code or signal that killed the command)
      but not its output. This makes debugging very hard.
      
      The patch changes all places where we only log fail_reason to also log
      result.output.
      
      Reviewed-by: ultrotter
      6c896e2f
  11. 10 May, 2008 1 commit
    • Iustin Pop's avatar
      DRBD: Fix another bug in diskless activation · ab6cc81c
      Iustin Pop authored
      DRBD8 requires that we pass ‘--create-device’ to the first command that
      wants to activate a new DRBD minor. We do this currently when we run the
      “drbdsetup ... disk” command which we run before the network setup.
      
      But if the LVs are missing, we skip the ‘disk’ subcommand and run only
      the ‘net’ one, so it might be that the activation fails because the
      minor we selected was never created in the first place.
      
      The patch adds the required parameter to the DRBD8._AssembleNet() call.
      Since it's a no-op for existing minors, it should not create any
      problems (tested and works both with configured and unconfigured
      minors).
      
      Reviewed-by: ultrotter
      ab6cc81c
  12. 08 May, 2008 3 commits
    • Guido Trotter's avatar
      Improve DRBD8.Open's docstring a bit more · f860ff4e
      Guido Trotter authored
      Reviewed-by: iustinp
      
      f860ff4e
    • Guido Trotter's avatar
      Fix comment typo in bdev.py · 7b62772e
      Guido Trotter authored
      Reviewed-by: iustinp
      
      7b62772e
    • Iustin Pop's avatar
      Fix DRBD8 diskless assembling · bf25af3b
      Iustin Pop authored
      The algorithm for attaching to existing DRBD devices is not trivial. It
      has four alternatives, and there is a bug in the last one when we have
      diskless devices.
      
      The last case (local disk info matches but remote/network configuration
      doesn't match) we disconnect from the network and reattach with the
      correct info. We do this because correct local device has higher
      priority than remote device.
      
      However, the test we use (self._MatchesLocal) can succeed in two cases:
        - we have a disk and it's the same as the one attached
        - we don't have a disk and the drbd is in diskless mode
      
      But this creates problems for the fourth case as when we already have
      one diskless DRBD, activating then next one will do:
        - _MatchesLocal? yes, because both config data and system have no
          disks (with the effect that all diskless devices are identical)
        - _MatchesRemote? no, because this disk is configured to its current
          remote peer, not to our new one
      
      The fix is trivial, although the algorithm not: we only allow overriding
      the network configuration when the disk information matches and we are
      not diskless, by adding the <"local_dev" in info'> test.
      
      Reviewed-by: ultrotter
      bf25af3b
  13. 15 Apr, 2008 1 commit
  14. 05 Apr, 2008 1 commit
    • Manuel Franceschini's avatar
      Add FileStorage class · 6f695a2e
      Manuel Franceschini authored
      This is the representation of file VBDs on the backend. It's the first
      implementation an supports only raw files.
      
      Reviewed-by: iustinp
      6f695a2e
  15. 20 Mar, 2008 1 commit
    • Iustin Pop's avatar
      Remove some deprecated code in bdev.py · 39e9e213
      Iustin Pop authored
      The BlockDev.GetStatus() method and some associated constants were never
      used in the rest of the code. This patch removes them.
      
      Reviewed-by: imsnah
      39e9e213
  16. 22 Feb, 2008 1 commit
  17. 12 Feb, 2008 1 commit
    • Guido Trotter's avatar
      Parse double protocol version in drbd8.2 · c3f9340c
      Guido Trotter authored
      DRBD 8.2 uses a double integer field ad protocol version, rather than a single
      one. This patch fixes the ganeti parsing code, allowing both the old and the
      new version type. In order to do so the internal _GetVersion function is
      changed to return a dict, rather than a list, and the second protocol field is
      added, only if present, as proto2.
      
      This is a fix for issue 24.
      
      Reviewed-by: iustinp
      c3f9340c
  18. 05 Feb, 2008 1 commit
    • Iustin Pop's avatar
      Reduce the chance of DRBD errors with stale primaries · fdbd668d
      Iustin Pop authored
      This patch is a first step in reducing the chance of causing DRBD
      activation failures when the primary node has not-perfect data.
      
      This issue is more seen with DRBD8, which has an 'outdate' state (in
      which it can get more often). But it can (and before this patch, usually
      will) happen with both 7 and 8 in the case the primary has data to sync.
      
      The error comes from the fact that, before this patch, we activate the
      primary DRBD device and immediately (i.e. as soon as we can run another
      shell command) we try to make it primary. This might fail - since the
      primary knows it has some data to catch up to - but we ignored this
      error condition. The failure was visible later, in either md failing to
      activate over a read-only storage or by instance failing to start.
      
      The patch has two parts: one affecting bdev.py, which changes failures
      in BlockDev.Open() from returning False to raising
      errors.BlockDeviceError; noone (except a generic method inside bdev.py)
      checked this return value and we logged it but the master didn't know
      about it; now all classes raise errors from Open if they have a failure.
      
      The other part, affecting cmdlib.py, changes the activation sequence
      from:
        - activate on primary node as primary and secondary as secondary, in
          whatever order a function returns the nodes
      to the following:
        - activate all drives as secondaries, on both the primary and the
          secondary nodes of the instance
        - after that, on the primary node, re-activate the device stack as
          primary
      
      This is in order to give the chance to DRBD to connect and make the
      handshake. As noted in the comments, this just increases the chances of
      a handshake/connect, not fixing entirely the problem. However, it is a
      good first step and it passes all tests of starting with stale (either
      full or partial) primaries, with both drbd 7 and 8, and also passes a
      burnin.
      
      Note that the patch might make the device activation a little bit
      slower, but it is a reasonable trade-off.
      
      Reviewed-by: imsnah
      fdbd668d
  19. 07 Jan, 2008 3 commits
    • Iustin Pop's avatar
      Activate logical volumes at Assemble() time · 5574047a
      Iustin Pop authored
      This patch changes the Assemble() method for logical volumes from a noop
      to do a `lvchange -ay` on the logical volume; this ensures that if the
      logical volume is not active, we are able to activate and use it.
      
      Reviewed-by: imsnah
      5574047a
    • Iustin Pop's avatar
      Improve speed of activating block devs · be1ba2bd
      Iustin Pop authored
      This patch fixes the double attach operation in bdev.AttachOrAssemble,
      which was an indentation mistake in the first place.
      
      Reviewed-by: imsnah
      be1ba2bd
    • Iustin Pop's avatar
      Add unittest for DRBD8 drdbsetup show parser · 3840729d
      Iustin Pop authored
      This patch changes the bdev.DRBD8._GetDevInfo to take a string instead
      of a minor, separates the `drbdsetup show` invocation into a new
      separate method (bdev.DRBD8._GetShowData) and modifies the rest of the
      DRBD8 class to make the appropriate calls.
      
      It also adds a unittest script and data files for testing various cases
      of device output.
      
      Reviewed-by: imsnah
      3840729d
  20. 30 Nov, 2007 1 commit
    • Iustin Pop's avatar
      Fix parsing of drbdsetup show output · c522ea02
      Iustin Pop authored
      This fixes the parsing of integers in the drbdsetup show output with
      newer pyparsing versions.
      
      Basically, the convert-to-int action that we use was taken from the
      example documentation of an older pyparsing version that automatically
      uses only the second element if the result is returned as a tuple, but
      this was deprecated for a while and removed in 1.4.3.
      
      Based on a nice report by Jorge Cabello <jorge@aspl.es>, and
      confirmation about pyparsing behaviour from its author. Thanks!
      
      Reviewed-by: imsnah
      c522ea02
  21. 26 Nov, 2007 3 commits
    • Iustin Pop's avatar
      Drop the Rename method of DRBD8 devices · a70f34d1
      Iustin Pop authored
      Since we now do changes of remote peers in two steps (Attach with no
      network information and then Attach with the new peer's data), we don't
      need this operation anymore.
      
      Reviewed-by: imsnah
      a70f34d1
    • Iustin Pop's avatar
      Relax even more the identification of DRBD8 devices · 9cdbe77f
      Iustin Pop authored
      This patch adds a new success condition for identifying DRBD8 devices:
      if the local device is correct, but the network peer is not, we shutdown
      the network and attach to the correct one (this also handles going to
      standalone mode).
      
      This new check (and its actions) is based on the fact that we don't pass
      incorrect data from the master to the node (otherwise we could start
      syncing with an incorrect peer.
      
      Reviewed-by: imsnah
      9cdbe77f
    • Iustin Pop's avatar
      Add support for DRBD8 devices to work standalone · 52857176
      Iustin Pop authored
      This patch adds a check in _AssembleNet so that if we are passed None
      value in the unique_id, we instead switch to standalone mode.
      
      Reviewed-by: imsnah
      52857176
  22. 21 Nov, 2007 1 commit
  23. 14 Nov, 2007 1 commit
  24. 07 Nov, 2007 1 commit
    • Iustin Pop's avatar
      Enhance secondary node replace for drbd8 · 0834c866
      Iustin Pop authored
      This (big) patch does two things:
        - add "local disk status" to the block device checks
          (BlockDevice.GetSyncStatus and the rpc calls that call this
          function, and therefore cmdlib._CheckDiskConsistency)
        - improve the drbd8 secondary replace operation using the above
          functionality
      
      The "local disk status" adds a new variable to the result of
      GetSyncStatus that shows the degradation of the local storage of the
      device. Of course, not all device support this - for now, we only modify
      LogicalVolumes and DRBD8 to return degraded in some cases, other devices
      always return non-degraded. This variable should be a subset of
      is_degraded - whenever this variable is true, the is_degraded should
      also be true.
      
      The drbd8 secondary replace uses this variable as we don't care if the
      primary drbd device is network-degraded, only if it has good local disk
      data (ldisk is False).
      
      The patch also increases the protocol version (due to rpc changes).
      
      Reviewed-by: imsnah
      0834c866
  25. 06 Nov, 2007 5 commits
    • Iustin Pop's avatar
      Implement degraded status for logical volumes · 9db6dbce
      Iustin Pop authored
      Logical volumes can be 'degraded' in a similar way to mirrored devices,
      when their underlying storage has gone away (i.e. after a physical disk
      failure and 'vgreduce --removemissing'). If we can detect this, we can
      prevent mistaken replaces of disks that would use this LV (or its
      parent) as source data.
      
      This patch adds support for computing the degraded attribute and
      modifies gnt-instance to warn if the LV is virtual.
      
      Reviewed-by: imsnah
      9db6dbce
    • Iustin Pop's avatar
      Enhance mirror operations for DRBD8 · 03ece5f3
      Iustin Pop authored
      Currently, the mirror operations (add and remove children) test against
      the instance's attributes. This patch changes the check tests to work
      against the actual status of the device (i.e. live data) which is more
      realistic.
      
      The changes are:
        - allow add children if the device doesn't have local storage (even if
          we believe it has)
        - early return from remove children if the device is already without
          local storage
      
      Reviewed-by: imsnah
      03ece5f3
    • Iustin Pop's avatar
      Allow DRBD8 operation without backing storage · fc1dc9d7
      Iustin Pop authored
      This patch adds the following functionality:
        - DRBD8 devices can assemble without local storage (done by allowing
          None in the list of children, and making DRBD8 to ignore all
          children if any is None)
        - DRBD8 devices can attach (i.e. identify a device) which is not
          connected to backing storage but to the correct network ports; this
          is a rare case in normal operation (it's what would happen if one
          manually detaches the local disk, and the backing LV still exists)
      
      Reviewed-by: imsnah
      fc1dc9d7
    • Iustin Pop's avatar
      Make DRBD8 disks show 'degraded' status if diskless · 583e3f6f
      Iustin Pop authored
      This patch enables the bdev.DRBD8 class report a degraded status if the
      local disk is missing. This allows `gnt-instance info` to report the
      actual situation in this case.
      
      Note that DRBD7 should also behave like this, however the diskless case
      is less often met there and we also don't want to change behaviour.
      
      The patch also fixes some docstrings for the GetSyncStatus methods.
      
      Reviewed-by: imsnah
      583e3f6f
    • Iustin Pop's avatar
      Change the way remove children is called in bdev · e739bd57
      Iustin Pop authored
      For some cases, we don't have to have access to the children of a device
      in order to remove them (e.g. md over lvs, or drbd over lvs). In order
      to ease the removal process, skip over finding the child if it provides
      a static dev path.
      
      This is needed in order to support removal of children when the
      underlying storage has gone away.
      
      Reviewed-by: imsnah
      e739bd57
  26. 05 Nov, 2007 1 commit
    • Iustin Pop's avatar
      Miscellaneous style fixes · 65fe4693
      Iustin Pop authored
      This patch fixes some minor pylint warnings (unused variables, wrong
      indentation, etc.) and a real bug in the recovery for drbd8 rename
      procedure.
      
      Reviewed-by: imsnah
      65fe4693