1. 19 Jan, 2009 1 commit
    • Iustin Pop's avatar
      Block device creation cleanup · 6c626518
      Iustin Pop authored
      Currently when creation LVM-based instances, we always get the
      extremely-confusing message "ERROR Can't find LV /dev/xenvg/..." which
      is actually expected. This behaviour was introduced before we had
      UUID-style LV names, since at that point it was not a unexpected to have
      such volumes laying around after a failed creation.
      
      Today, it's much more of an error to see existing volumes, and it's
      better to abort with a failure. Since bdev.LogicalVolume.Create() method
      will raise an error in case it exists, we can remove this check in
      backend before creating the device.
      
      The Create methods for DRBD and FileStorage currently don't raise
      exception, as behaviour is not very well defined here.
      
      We also change some exception types raised in bdev so that all
      exceptions raised by device creation are a subclass of GenericError.
      
      Reviewed-by: ultrotter
      6c626518
  2. 13 Jan, 2009 4 commits
    • Iustin Pop's avatar
      Forward-port DrbdNetReconfig · 6b93ec9d
      Iustin Pop authored
      This is a modified forward-port of DrbdNetReconfig and their associated
      RPCs. In Ganeti 2.0, these functions will be used for two things:
        - live migration (as in 1.2)
        - and for other network reconfiguration tasks, since DRBD8.Attach()
          doesn't do them anymore
      
      Because of the Attach() changes, we can now implement the
      AttachNet/DisconnectNet functions as independent entities, and we don't
      need the cache anymore.
      
      Note these functions are copies of the latest 1.2 code, and not
      cherry-picks of the (many) patches that went into 1.2.
      
      Reviewed-by: ultrotter
      6b93ec9d
    • Iustin Pop's avatar
      backend: rename AttachOrAssemble to Assemble · f96e3c4f
      Iustin Pop authored
      Since now the Assemble function is different than Attach, we rename this
      backend function to show that the intent is to fully assemble the device
      (and it's always allowed to modify the device).
      
      Reviewed-by: ultrotter
      f96e3c4f
    • Iustin Pop's avatar
      drbd: change the semantics of Attach vs. Assemble · 2d0c8319
      Iustin Pop authored
      Currently, both the Attach and Assemble methods for DRBD8 devices will use and
      alter the device state. This is suboptimal, and it has been worked
      around in 1.2 via a special cache in the node daemon so that we don't
      need to call Attach() again in migration, for example.
      
      Since in 2.0 we have static minors, we can change these functions so
      that:
        - Attach() does not affect the device in any way, and only checks if
          the minor is already in use or not
        - Assemble() has two logic paths, one for startup from unused minor
          (the old Assemble, now renamed _FastAssemble) and one for
          re-checking/fixing an in-use minor (the old Attach, now renamed
          _SlowAttach)
      
      Basically Attach was renamed to _SlowAttach, Assemble to _FastAssemble,
      and we have a new, simple Assemble that calls one or the other based on
      the result of the new Attach.
      
      The LUReplaceDisks (with new secondary) is relying on the special
      semantics of Attach modifying the device and is broken until the end of
      the patch series.
      
      Reviewed-by: ultrotter
      2d0c8319
    • Iustin Pop's avatar
      bdev: Do not call Assemble() on children · f87548b5
      Iustin Pop authored
      The caller of dev.Assemble() (backend._RecursiveAssembleBD) is doing an
      explicit recursion over all the children of the device, with better
      error reporting. As such, we don't need this repeated assembly inside
      the base BlockDev class.
      
      Reviewed-by: ultrotter
      f87548b5
  3. 09 Jan, 2009 1 commit
    • Iustin Pop's avatar
      Work around a DRBD sync speed race condition · 7d585316
      Iustin Pop authored
      This is modified forward-port of commit 1544 on the 1.2 branch:
      
        When DRBD is doing its dance to establish a connection with its
        peer, it also sends the synchronization speed over the wire. In
        some cases setting the sync speed only after setting up both
        sides can race with DRBD connecting, hence we set it here before
        telling DRBD anything about its peer.
      
        Reviewed-by: iustinp
      
      The modification we make is that we split SetSyncSpeed in two so that we
      don't need to modify our minor temporarily, and the fact that we call
      this function from within _AssembleNet (right before enabling network),
      instead of Assemble()/Attach().
      
      Original-Author: imsnah
      7d585316
  4. 08 Jan, 2009 1 commit
    • Iustin Pop's avatar
      bdev: forward-port ReAttachNet/DisconnectNet · cf8df3f3
      Iustin Pop authored
      This is plain copy of the 1.2 ReAttachNet and DisconnectNet methods on
      the DRBD8 device, with the logger to logging module changes and the
      ReAttachNet method renamed to AttachNet.
      
      These methods are not used anywhere right now, but will be used for
      migration and a simpler disk-replace.
      
      The code was originally committed on the 1.2 branch as revision numbers
      1165 and 1204.
      
      Originally-Reviewed-by: imsnah, ultrotter
      cf8df3f3
  5. 11 Dec, 2008 1 commit
    • Iustin Pop's avatar
      Fix epydoc format warnings · c41eea6e
      Iustin Pop authored
      This patch should fix all outstanding epydoc parsing errors; as such, we
      switch epydoc into verbose mode so that any new errors will be visible.
      
      Reviewed-by: imsnah
      c41eea6e
  6. 27 Nov, 2008 1 commit
    • Iustin Pop's avatar
      Fix file-based block devices · ecb091e3
      Iustin Pop authored
      We changed a while ago the protocol for opening block devices, but
      FileStorage was not changed. This patch makes it work again.
      
      Reviewed-by: imsnah
      ecb091e3
  7. 29 Sep, 2008 3 commits
    • Iustin Pop's avatar
      Move a hardcoded constant to constants.py · 3c03759a
      Iustin Pop authored
      For now we only use the ‘C’ protocol so we can put it in constants.py
      instead of hardcoding it.
      
      Reviewed-by: imsnah
      3c03759a
    • Iustin Pop's avatar
      Enable the use of shared secrets · 2899d9de
      Iustin Pop authored
      This patch enables the use of the shared secrets for DRBD8 disks, using
      (hardcoded in constants.py) the md5 digest algorithm.
      
      For making this more flexible, either we implement a cluster parameter
      (once the new model is in place), or we can make it ./configure-time
      selectable.
      
      Reviewed-by: imsnah
      2899d9de
    • Iustin Pop's avatar
      Extend DRBD disks with shared secret attribute · f9518d38
      Iustin Pop authored
      This patch, which is similar to r1679 (Extend DRBD disks with minors
      attribute), extends the logical and physical id of the DRBD disks with a
      shared secret attribute. This is generated at disk creation time and
      saved in the config file.
      
      The generation of the secret is done so that we don't have duplicates in
      the configuration (otherwise the goal of preventing cross-connection
      will not be reached), so we add to config.py more than just a simple
      call to utils.GenerateSecret().
      
      The patch does not yet enable the use of the secrets.
      
      Reviewed-by: imsnah
      f9518d38
  8. 23 Sep, 2008 1 commit
    • Iustin Pop's avatar
      Switch to static minors for DRBD · a1578d63
      Iustin Pop authored
      With some todos remaining, this patch switches the DRBD devices to use
      the passed minors, and the cmdlib code (add instance and replace disks)
      to request and assign minors to the DRBD disks.
      
      Todos:
        - look at the disk RPC calls to see which can be optimized away, since
          we now know the minor beforehand
        - remove the _FindUnusedMinor usage from the few places it's still
          used (not for actual disks, but for temporary use in meta devs) and
          eventually replace with _CheckMinorUnused or such
      
      Of course, this and/or the previous two patches break existing clusters.
      Again.
      
      Reviewed-by: imsnah
      a1578d63
  9. 22 Sep, 2008 1 commit
    • Iustin Pop's avatar
      Extend DRBD disks with minors attribute · ffa1c0dc
      Iustin Pop authored
      This patch converts the DRBD disks to contain also a minor (per each
      node) attribute. This minor is not yet used and is always initialized
      with None, so the patch does not have any real-world impact - except for
      automatically upgrading config files (it adds the minors as None, None).
      
      Reviewed-by: imsnah
      ffa1c0dc
  10. 09 Jul, 2008 2 commits
    • Iustin Pop's avatar
      Reduce duplicate Attach() calls in bdev · cb999543
      Iustin Pop authored
      Currently, the 'public' functions of bdev (FindDevice and
      AttachOrAssemble) will call the Attach() method right after class
      instantiation.
      
      But the constructor itself calls this function, and therefore we have
      duplicate Attach() calls (which are not cheap at all).
      
      The patch introduces a new 'attached' instance attribute that tells if
      the last Attach() was successful. The public functions reuse this so
      that we only do the minimum required number of calls.
      
      Reviewed-by: imsnah
      cb999543
    • Iustin Pop's avatar
      Convert bdev.py to the logging module · 468c5f77
      Iustin Pop authored
      This does not enhance in any way the messages; it just switches to the
      new module.
      
      Reviewed-by: imsnah
      468c5f77
  11. 25 Jun, 2008 1 commit
    • Iustin Pop's avatar
      Cleanup LV status computation · 99e8295c
      Iustin Pop authored
      Currently, when seeing if a LV is degraded or not (i.e. virtual volume),
      we first attach to the device (which does an lvdisplay), then do a lvs
      in order to display the lv_attr. This generates two external commands to
      do (almost) the same thing.
      
      This patch changes the Attach() method for LVs to call lvs and display
      both the major/minor (needed for attach) and the lv_status (needed for
      GetSyncStatus). Thus, later in GetSyncStatus, we don't need to run lvs
      again, and instead just return the value computed in Attach().
      
      Reviewed-by: imsnah
      99e8295c
  12. 18 Jun, 2008 1 commit
    • Iustin Pop's avatar
      Rework the DRBD8 device status computation · 6b90c22e
      Iustin Pop authored
      Currently, compute the status of a drbd8 device in GetSyncStatus and
      return only the values that we need (and fit in the framework of
      GetSyncStatus). However, the full status details are useful (and needed)
      in other places, so the patch attempts to improve this situation.
      
      We abstract the status of a device outside in a separate class, that
      knows how to parse contents from /proc/drbd and set easily accessible
      attributes. We then simplify the GetSyncStatus to use this and return
      the values that it needs, and add a separate method that returns the
      full status object.
      
      The move to a separate class cleans up a little bit the old
      sync-progress computation from GetSyncStatus, but it's still many
      regexes.
      
      The patch also adds unittests for a few statuses, and modifies one
      BaseDRBD call to accept a custom filename instead of '/proc/drbd' to
      ease unittests.
      
      Reviewed-by: imsnah
      6b90c22e
  13. 16 Jun, 2008 1 commit
    • Iustin Pop's avatar
      bdev: implement disk resize for lvm/drbd8 · 1005d816
      Iustin Pop authored
      This patch implements disk resize at the bdev level for the LVM and
      DRBD8 disk types. It is not implemented for DRBD7 and MD since the way
      MD works with its underlaying devices makes it harder and this
      combination is also deprecated.
      
      The LVM resize operation is tried three times, with different allocation
      policies:
        - contiguous first, since this is best for allocation purposes (it
          won't fragment too much the PV)
        - cling, which is supported only by more recent LVM versions, will try
          to place the new extents on the same PV as the rest of the LV
        - and finally normal, which is the default
      
      Reviewed-by: imsnah
      1005d816
  14. 30 May, 2008 1 commit
    • Iustin Pop's avatar
      Complete removal of md/drbd 0.7 code · abdf0113
      Iustin Pop authored
      This patch removes the last of the md and drbd 0.7 code. Cluster which
      have the old device types will be broken if they have this applied.
      
      Reviewed-by: imsnah
      abdf0113
  15. 15 May, 2008 2 commits
    • Guido Trotter's avatar
      Fix drbd show parser to handle valueless keywords · 63012024
      Guido Trotter authored
      It turns out in some cases there can exist keywords without an
      associated value exported by drbdsetup show. This patch makes the value
      part optional in our parser, so that if it's not present the parsing
      result will contain an array with just the keyword in it. This is not a
      problem since we check all keyword names before accessing their values,
      so we won't mistakenly try to access the value of a valueless keyword.
      
      Reviewed-by: iustinp
      
      63012024
    • Guido Trotter's avatar
      Split drbd command creation and execution · 333411a7
      Guido Trotter authored
      Make _AssembleDisk more similar to _AssembleNet by splitting the
      generation of the drbdsetup command and its execution. While not
      changing anything this makes it easier to manipulate the command just in
      certain cases, which in the future we'll need to do.
      
      Reviewed-by: iustinp
      
      333411a7
  16. 12 May, 2008 1 commit
    • Iustin Pop's avatar
      bdev: always log command output if it failed · 6c896e2f
      Iustin Pop authored
      Currently many error handling code paths in bdev.py log only
      result.fail_reason (i.e. exit code or signal that killed the command)
      but not its output. This makes debugging very hard.
      
      The patch changes all places where we only log fail_reason to also log
      result.output.
      
      Reviewed-by: ultrotter
      6c896e2f
  17. 10 May, 2008 1 commit
    • Iustin Pop's avatar
      DRBD: Fix another bug in diskless activation · ab6cc81c
      Iustin Pop authored
      DRBD8 requires that we pass ‘--create-device’ to the first command that
      wants to activate a new DRBD minor. We do this currently when we run the
      “drbdsetup ... disk” command which we run before the network setup.
      
      But if the LVs are missing, we skip the ‘disk’ subcommand and run only
      the ‘net’ one, so it might be that the activation fails because the
      minor we selected was never created in the first place.
      
      The patch adds the required parameter to the DRBD8._AssembleNet() call.
      Since it's a no-op for existing minors, it should not create any
      problems (tested and works both with configured and unconfigured
      minors).
      
      Reviewed-by: ultrotter
      ab6cc81c
  18. 08 May, 2008 3 commits
    • Guido Trotter's avatar
      Improve DRBD8.Open's docstring a bit more · f860ff4e
      Guido Trotter authored
      Reviewed-by: iustinp
      
      f860ff4e
    • Guido Trotter's avatar
      Fix comment typo in bdev.py · 7b62772e
      Guido Trotter authored
      Reviewed-by: iustinp
      
      7b62772e
    • Iustin Pop's avatar
      Fix DRBD8 diskless assembling · bf25af3b
      Iustin Pop authored
      The algorithm for attaching to existing DRBD devices is not trivial. It
      has four alternatives, and there is a bug in the last one when we have
      diskless devices.
      
      The last case (local disk info matches but remote/network configuration
      doesn't match) we disconnect from the network and reattach with the
      correct info. We do this because correct local device has higher
      priority than remote device.
      
      However, the test we use (self._MatchesLocal) can succeed in two cases:
        - we have a disk and it's the same as the one attached
        - we don't have a disk and the drbd is in diskless mode
      
      But this creates problems for the fourth case as when we already have
      one diskless DRBD, activating then next one will do:
        - _MatchesLocal? yes, because both config data and system have no
          disks (with the effect that all diskless devices are identical)
        - _MatchesRemote? no, because this disk is configured to its current
          remote peer, not to our new one
      
      The fix is trivial, although the algorithm not: we only allow overriding
      the network configuration when the disk information matches and we are
      not diskless, by adding the <"local_dev" in info'> test.
      
      Reviewed-by: ultrotter
      bf25af3b
  19. 15 Apr, 2008 1 commit
  20. 05 Apr, 2008 1 commit
    • Manuel Franceschini's avatar
      Add FileStorage class · 6f695a2e
      Manuel Franceschini authored
      This is the representation of file VBDs on the backend. It's the first
      implementation an supports only raw files.
      
      Reviewed-by: iustinp
      6f695a2e
  21. 20 Mar, 2008 1 commit
    • Iustin Pop's avatar
      Remove some deprecated code in bdev.py · 39e9e213
      Iustin Pop authored
      The BlockDev.GetStatus() method and some associated constants were never
      used in the rest of the code. This patch removes them.
      
      Reviewed-by: imsnah
      39e9e213
  22. 22 Feb, 2008 1 commit
  23. 12 Feb, 2008 1 commit
    • Guido Trotter's avatar
      Parse double protocol version in drbd8.2 · c3f9340c
      Guido Trotter authored
      DRBD 8.2 uses a double integer field ad protocol version, rather than a single
      one. This patch fixes the ganeti parsing code, allowing both the old and the
      new version type. In order to do so the internal _GetVersion function is
      changed to return a dict, rather than a list, and the second protocol field is
      added, only if present, as proto2.
      
      This is a fix for issue 24.
      
      Reviewed-by: iustinp
      c3f9340c
  24. 05 Feb, 2008 1 commit
    • Iustin Pop's avatar
      Reduce the chance of DRBD errors with stale primaries · fdbd668d
      Iustin Pop authored
      This patch is a first step in reducing the chance of causing DRBD
      activation failures when the primary node has not-perfect data.
      
      This issue is more seen with DRBD8, which has an 'outdate' state (in
      which it can get more often). But it can (and before this patch, usually
      will) happen with both 7 and 8 in the case the primary has data to sync.
      
      The error comes from the fact that, before this patch, we activate the
      primary DRBD device and immediately (i.e. as soon as we can run another
      shell command) we try to make it primary. This might fail - since the
      primary knows it has some data to catch up to - but we ignored this
      error condition. The failure was visible later, in either md failing to
      activate over a read-only storage or by instance failing to start.
      
      The patch has two parts: one affecting bdev.py, which changes failures
      in BlockDev.Open() from returning False to raising
      errors.BlockDeviceError; noone (except a generic method inside bdev.py)
      checked this return value and we logged it but the master didn't know
      about it; now all classes raise errors from Open if they have a failure.
      
      The other part, affecting cmdlib.py, changes the activation sequence
      from:
        - activate on primary node as primary and secondary as secondary, in
          whatever order a function returns the nodes
      to the following:
        - activate all drives as secondaries, on both the primary and the
          secondary nodes of the instance
        - after that, on the primary node, re-activate the device stack as
          primary
      
      This is in order to give the chance to DRBD to connect and make the
      handshake. As noted in the comments, this just increases the chances of
      a handshake/connect, not fixing entirely the problem. However, it is a
      good first step and it passes all tests of starting with stale (either
      full or partial) primaries, with both drbd 7 and 8, and also passes a
      burnin.
      
      Note that the patch might make the device activation a little bit
      slower, but it is a reasonable trade-off.
      
      Reviewed-by: imsnah
      fdbd668d
  25. 07 Jan, 2008 3 commits
    • Iustin Pop's avatar
      Activate logical volumes at Assemble() time · 5574047a
      Iustin Pop authored
      This patch changes the Assemble() method for logical volumes from a noop
      to do a `lvchange -ay` on the logical volume; this ensures that if the
      logical volume is not active, we are able to activate and use it.
      
      Reviewed-by: imsnah
      5574047a
    • Iustin Pop's avatar
      Improve speed of activating block devs · be1ba2bd
      Iustin Pop authored
      This patch fixes the double attach operation in bdev.AttachOrAssemble,
      which was an indentation mistake in the first place.
      
      Reviewed-by: imsnah
      be1ba2bd
    • Iustin Pop's avatar
      Add unittest for DRBD8 drdbsetup show parser · 3840729d
      Iustin Pop authored
      This patch changes the bdev.DRBD8._GetDevInfo to take a string instead
      of a minor, separates the `drbdsetup show` invocation into a new
      separate method (bdev.DRBD8._GetShowData) and modifies the rest of the
      DRBD8 class to make the appropriate calls.
      
      It also adds a unittest script and data files for testing various cases
      of device output.
      
      Reviewed-by: imsnah
      3840729d
  26. 30 Nov, 2007 1 commit
    • Iustin Pop's avatar
      Fix parsing of drbdsetup show output · c522ea02
      Iustin Pop authored
      This fixes the parsing of integers in the drbdsetup show output with
      newer pyparsing versions.
      
      Basically, the convert-to-int action that we use was taken from the
      example documentation of an older pyparsing version that automatically
      uses only the second element if the result is returned as a tuple, but
      this was deprecated for a while and removed in 1.4.3.
      
      Based on a nice report by Jorge Cabello <jorge@aspl.es>, and
      confirmation about pyparsing behaviour from its author. Thanks!
      
      Reviewed-by: imsnah
      c522ea02
  27. 26 Nov, 2007 3 commits
    • Iustin Pop's avatar
      Drop the Rename method of DRBD8 devices · a70f34d1
      Iustin Pop authored
      Since we now do changes of remote peers in two steps (Attach with no
      network information and then Attach with the new peer's data), we don't
      need this operation anymore.
      
      Reviewed-by: imsnah
      a70f34d1
    • Iustin Pop's avatar
      Relax even more the identification of DRBD8 devices · 9cdbe77f
      Iustin Pop authored
      This patch adds a new success condition for identifying DRBD8 devices:
      if the local device is correct, but the network peer is not, we shutdown
      the network and attach to the correct one (this also handles going to
      standalone mode).
      
      This new check (and its actions) is based on the fact that we don't pass
      incorrect data from the master to the node (otherwise we could start
      syncing with an incorrect peer.
      
      Reviewed-by: imsnah
      9cdbe77f
    • Iustin Pop's avatar
      Add support for DRBD8 devices to work standalone · 52857176
      Iustin Pop authored
      This patch adds a check in _AssembleNet so that if we are passed None
      value in the unique_id, we instead switch to standalone mode.
      
      Reviewed-by: imsnah
      52857176