1. 02 May, 2014 6 commits
    • Dimitris Aragiorgis's avatar
      deploy: Use archipelago related components · 523cb739
      Dimitris Aragiorgis authored
      Make Cyclades and Pithos require PithosBackend and Archip.
      Make VMC require also ExtStorage and ArchipGaneti.
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
    • Dimitris Aragiorgis's avatar
      deploy: Refactor components module · 3699d1fe
      Dimitris Aragiorgis authored
      Up until now components were simple objects decoupled from fabric
      execution. Their methods were just returning a list of bash commands
      which utils module was executing in the correct context.
      This patch changes this rational. Specifically:
      A Component() gets initialized with an execution context that is a
      configuration snapshot of the target setup, cluster and node. A
      component implements the following helper methods: check, install,
      prepare, configure, restart, initialize, and test. All those methods
      will be executed on the target node with this order during setup.
      Additionally each Component class implements admin_pre, and
      admin_post methods which invoke actions on different components on
      the same execution context before and after installation. For
      example before a backend gets installed, its FQDN must resolve to
      the master floating IP, so we have to run some actions on the ns
      node and after installation we must add it to cyclades (snf-manage
      backend-add on the cyclades node).
      Component() inherits ComponentRunner() which practically exports the
      setup() method. This will first check if the required components are
      installed, will install them if not and update the status of target
      ComponentRunner() inherits FabricRunner() which practically wraps
      basic fabric commands (put, get, run) with the correct execution
      The fabfile practically gets the target nodes for each role from the
      configuration and uses the roles module which knows the
      role-component mapping to get the target component with the proper
      execution context. Then it just invokes the setup() method.
      Each component gets initialized with an execution context and uses
      the config module for accessing global wide options. The context
      provides node, cluster, and setup related info.
      We introduce some helper decorators that wrap methods of Component
      class and update its execution context (self, self.ctx):
      - parse_* executes the wrapped method and parses the output.
        The desired vars (user_id, backend_id, user_uuid, etc.) are
        stored in the execution context and made available to other
      - update_admin initializes the admin roles (NS, Astakos,
        Cyclades, etc.) of the current context and make them available
        under self.NS, self.ASTAKOS, etc. These have the same execution
        context of the current components besides the target node
        which gets derived from the corresponding config.
      - update_cluster_admin initializes the cluster's admin role
        and make it available under self.MASTER.
      - run_cmds runs the list of returned commands in the target node
      The update_ns() method of NS component gets the
      self.ctx.admin_fqdn and invokes nsupdate accordingly.
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
    • Dimitris Aragiorgis's avatar
      deploy: Add new component GanetiDev · e330e80a
      Dimitris Aragiorgis authored
      This is used to create a development environment for Ganeti.
      It clones the official repo, installs all packages needed,
      and adds a qa-sample.json file that reflects to the
      Ganeti cluster found in the configuration.
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
    • Dimitris Aragiorgis's avatar
      deploy: Add ganeti-qa command · 9093eb07
      Dimitris Aragiorgis authored
      To install a minimal Ganeti cluster for QA run:
       snf-deploy ganeti-qa
      This will create a ganeti cluster. The test instances will resolve on
      the local nameserver. On the client node it clones the official repo
      and installs all required packages required for ganeti development.
      A sample qa json file is uploaded under /tmp.
      To run the qa go to the client node and run:
       # cd ganeti
       # python ganeti-qa.py /tmp/qa-sample.json --yes-do-it
      Note that the installed version in the qa cluster should be the same
      used in the qa source.
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
    • Dimitris Aragiorgis's avatar
      deploy/ci: Setup lvm in case extra disk exists · 79707d0e
      Dimitris Aragiorgis authored
      Ganeti needs a VG in order to support plain/drbd disk templates.
      In case an extra disk exists, deploy will create a PV and a VG
      on this disk.
      Use options `extra_disk` in nodes.conf and `vg` in ganeti.conf for
      this purpose.
      VMs spawned currently during ci do not have extra disks. Update the
      corresponding schema.
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
    • Dimitris Aragiorgis's avatar
      deploy: Extract synnefo's components from ganeti · de81d118
      Dimitris Aragiorgis authored
      The components PithosBackend, Image, GTools, and GanetiCollectd
      are related to Ganeti but used only by Synnefo.
      Define new role, named 'snf-ganeti' to include those components and use
      it with the `snf-deploy ganeti` command.
      Install ganeti-instance-debootstrap so that we have a working os
      snf-image takes a lot of time to be configured (curl,
      snf-image-update-helper, wget debian-base image) and is not needed
      for a simple Ganeti cluster (i.e. QA cluster).
      Move wget of debian base image to Image component and not nfs.
      In order not to download many times check if it already exists
      in the proper path.
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
  2. 28 Apr, 2014 2 commits
  3. 14 Apr, 2014 1 commit
    • Vangelis Koukis's avatar
      Switch license to GPLv3 · 02071b96
      Vangelis Koukis authored
      According to the decision of the GRNET Board of Directors,
      switch license to GPLv3.
      This commit will be propagated to the release
      and master branches based on git flow, and the next
      release will be licensed as GPLv3.
  4. 31 Mar, 2014 6 commits
  5. 28 Feb, 2014 2 commits
  6. 13 Feb, 2014 3 commits
  7. 03 Feb, 2014 1 commit
  8. 29 Jan, 2014 1 commit
  9. 16 Jan, 2014 1 commit
  10. 09 Jan, 2014 3 commits
  11. 08 Jan, 2014 3 commits
    • Dimitris Aragiorgis's avatar
      More pep8 fixes related to snf-deploy · f8c8467d
      Dimitris Aragiorgis authored
      Those fixes concern errors that are introduced in latest
      pep8 version (and not the wheezy's one).
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>
    • Dimitris Aragiorgis's avatar
    • Dimitris Aragiorgis's avatar
      Huuuuge snf-deploy refactor · df284363
      Dimitris Aragiorgis authored
      Introduce the concept of Roles and Components.
      Roles are ns, mq, db, nfs, astakos, cyclades, pithos, cms, stats,
      ganeti, master, and client.
      Each role consists of various SynnefoComponents which  should define:
      * commands to check installation prerequisites (check)
      * packages to install (REQUIRED_PACKAGES)
      * commands to prepare installation (prepare)
      * configuration templates (configure)
      * reload commands (restart)
      * initialization commands (initialize)
      * test commands (test)
      SynnefoComponents are unaware of fabric environment. They
      get initialized with a Host object and a Env object.
      Host includes all the necessary info for the installation node
      (ip, hostname, alias, fqdn..) and Env includes all the configuration info
      (created after parsing config files)
      After defining components, fabfile should just do the following:
      def setup_somerole_role():
      In case you want to run a component's specific method just run
      RunComponentMethod(SomeComponent, "method_name", *args, **kwargs)
      During a role setup you might have to retrieve info from other
      components already installed (user token, backend id, etc.) Use
      'execute' fabric method and fill env object with required info.
      Make deployment re-entrant.
      Check if specific component is already installed and if yes then skip it.
      Currently this is done on node/component level. The component's
      status on a target node is either ok or nothing.
      Introduce conflicting components.
      Two components might not be able/or should not coexist in the same
      node, e.g. NFS and Mount. These conflicts are defined in CONFLICTS
      dict in roles.py.
      Sync ci and deploy conf files.
      ci: Change node's password in nodes.conf
      Always enable CSRF. Install CMS only if it resides on different
      node than pithos, cyclades and astakos.
      Add copyright headers.
      Signed-off-by: default avatarDimitris Aragiorgis <dimara@grnet.gr>