1. 25 Feb, 2013 1 commit
    • Iustin Pop's avatar
      Improve TemplateHaskell code to support empty objects · 08f7d24d
      Iustin Pop authored
      
      
      Currently, an empty objects will generate warnings as the arguments of
      various functions are unused. By adding conditional code for this, we
      can support generation of empty objects, e.g. like needed in Rpc code.
      
      Additionally, the patch also converts RpcCallVersion to THH, now that
      it can build it. We change the serialisation for this (from JSNull to
      JSObject []), but this shouldn't matter as this is not used in
      production.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarGuido Trotter <ultrotter@google.com>
      08f7d24d
  2. 24 Dec, 2012 1 commit
  3. 04 Dec, 2012 5 commits
  4. 30 Nov, 2012 1 commit
    • Iustin Pop's avatar
      Remove read instances from our Haskell code · 139c0683
      Iustin Pop authored
      
      
      It turns out that optimising 'read' derived instances (via -O) for
      complex data types (like OpCode, or the various objects) can be slow
      to very slow. Disabling such instances results in (time make
      $all_our_haskell_binaries) large compile-time savings and also smaller
      (unstripped) binaries (by a significant amount):
      
      ghc 6.12:        time  htools sz  hconfd sz
        with read:    4m50s 12,244,694 14,927,928
        no read:      3m30s 10,234,305 12,536,745
      ghc 7.6:
        with read:   14m45s 13,694,761 15,741,755
        no read:      3m40s 11,631,373 13,245,134
      
      So let's remove these instances, since we never use read in production
      for our custom types, and even when debugging in GHCI, we can simply
      use the 'show' representation to create the types, without needing to
      actually parse from strings.
      
      Note: for the very slow ghc 7.6 compilation time, I filled a ticket
      (ghc #7450), since it is surprising(ly slow).
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarMichele Tartara <mtartara@google.com>
      139c0683
  5. 20 Nov, 2012 1 commit
    • Iustin Pop's avatar
      Switch opcode data type from normal to record constructors · 8ee2994a
      Iustin Pop authored
      
      
      Currently, the OpCode definitions are using normal constructors:
      
        data OpCode = OpTestDelay Double Bool [String]
                    | OpInstanceFailover String Bool (Maybe String)
                    …
      
      While this works for a few opcodes, it becomes unwieldy when dealing
      with a bigger number of opcode definitions and/or with opcodes having
      many fields.
      
      This patch changes the opcodes to record-based constructors, so that
      we get for free accessor functions:
      
        data OpCode
          = OpTestDelay {
              opDuration :: Double,
              opOnMaster :: Bool,
              opOnNodes :: [String]
            }
            | OpInstanceFailover {
               opInstanceName :: String,
               opIgnoreConsistency :: Bool,
               opTargetNode :: Maybe String
            }
            …
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarAdeodato Simo <dato@google.com>
      8ee2994a
  6. 15 Nov, 2012 1 commit
    • Iustin Pop's avatar
      Cleanup THH function use from built module namespace · 32a569fe
      Iustin Pop authored
      
      
      Currently, THH.hs "injects" into the built code names of library
      functions like Text.JSON.makeObj, Ganeti.JSON.fromObj, etc. built
      directly from strings, via (e.g.)
      
        varE (mkName "makeObj")
      
      This means that the "makeObj" name must exist in the target module,
      i.o.w. must be imported there. This leads to the strange case of
      having to have imports that do not appear at all in the used
      (template) code, but are needed to satisfy this "hidden" dependency;
      look at Ganeti/Jobs.hs before this patch, for example.
      
      This is also not very obvious, because we usually import Text.JSON
      anyway; I only stumbled upon it while doing some cleanup work.
      
      So to clean this up, the current patch changes the THH.hs to use not
      string-derived, but identifier-derived names («'identifier» versus
      «mkName "identifier"»); this is better, as the names must be
      resolvable when compiling THH itself (once), and not when compiling
      the multiple derived modules. As you can see, this allows removal of
      extraneous imports from various modules.
      
      Background information: an `mkName "foo"` results in a name of flavour
      NameS (“An unqualified name; dynamically bound”) or alternatively to a
      qualified name, but still dynamically bound. Whereas what we want is a
      statically bound name: `'foo` results in a NameG flavour, “Global name
      bound outside of the TH AST: An original name”.
      
      One more explanation: the change is similar to going from 'x = eval
      "map"' to 'x = map'; the name is no longer dynamically evaluated, but
      statically when the module is compiled. In our case, previously names
      were bound at target module compile time, now they are bound at THH.hs
      compile time.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarGuido Trotter <ultrotter@google.com>
      32a569fe
  7. 12 Nov, 2012 2 commits
  8. 06 Nov, 2012 1 commit
  9. 26 Oct, 2012 1 commit
    • Iustin Pop's avatar
      Add support for optional fields with null serialised · 9b156883
      Iustin Pop authored
      
      
      This follows a conversation we had for how to deal with
      optional-but-required fields in JSON serialisations: fields which are
      optional (can be either a given type or 'null'), but where the 'null'
      value is required. There are just a few of these in the Python code,
      but we should support them nevertheless.
      
      The patch changes the 'isOptional' attribute from boolean to a custom
      ADT, three-typed. This allows us to keep the same path on load (which
      deals with both cases), but use a custom save path where we explicitly
      save the 'null' value.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarGuido Trotter <ultrotter@google.com>
      9b156883
  10. 25 Oct, 2012 4 commits
  11. 11 Oct, 2012 1 commit
  12. 05 Sep, 2012 1 commit
    • Iustin Pop's avatar
      Add entire ConfigData serialisation tests · 9924d61e
      Iustin Pop authored
      
      
      Using the recently-added genArbitrary, we can now implement Arbitrary
      instances for even "huge" objects like Cluster, so let's use that to
      implement entire ConfigData serialisation tests.
      
      Note that, as we don't have yet proper types for some of the Params
      fields, we have to cheat via FlexibleInstances and
      TypeSynonymInstances, using either empty items or real arbitrary
      values.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarRené Nussbaumer <rn@google.com>
      9924d61e
  13. 03 Sep, 2012 1 commit
    • Iustin Pop's avatar
      Expand THH with more functionality for parameters · 2af78b97
      Iustin Pop authored
      
      
      This adds two related improvements to THH:
      
      - for parameters, we declare a list with all their fields, so that
        Query2 can build the list of fields (e.g. for hvparams, or ndparams)
        automatically
      
      - we declare a new type class for "DictObjects", i.e. objects which
        can be converted to a [(String, JSValue)] list of pairs; while this
        applies to all our objects, it will be used first for filled
        parameters, so that we can implement the lookup functions
        generically
      
      Note that we re-export the new class from Objects.hs, so that other
      modules don't have to import THH.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarAgata Murawska <agatamurawska@google.com>
      2af78b97
  14. 28 Aug, 2012 8 commits
  15. 19 Jul, 2012 3 commits
  16. 08 May, 2012 2 commits
    • Iustin Pop's avatar
      Add decoding of Luxi calls and unittests for LuxiOp · cdd495ae
      Iustin Pop authored
      
      
      This patch adds a hand-coded decoder for LuxiCall arguments, as the
      data-structure is not uniform enough for automated generation (even
      for the serialisation, we had to add hints for some fields,
      de-serialisation is even harder).
      
      It also fixes a tiny issue with WaitForJobChange job ID encoding, and
      adds unittests for the encoding/decoding of LuxiOp structures.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarRené Nussbaumer <rn@google.com>
      cdd495ae
    • Iustin Pop's avatar
      Auto-define a LuxiReq data type · 95d0d502
      Iustin Pop authored
      
      
      We currently auto-generate a LuxiOp data type, which holds the entire
      operation (including parameters). However, having a data type just for
      the method call would be useful, so let's change THH to also
      defineSADT for the Luxi constructors.
      
      Currently I don't know how to match automatically a LuxiReq to its
      LuxiOp counterpart (not even sure we need that), so any matching will
      remain manual.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarRené Nussbaumer <rn@google.com>
      95d0d502
  17. 26 Mar, 2012 2 commits
  18. 13 Mar, 2012 1 commit
    • Iustin Pop's avatar
      htools: add partial implementation of lib/objects.py · b1e81520
      Iustin Pop authored
      
      
      This is partial since not all object types can be easily converted for
      now (will need some changes on the Python side for this).
      
      Most importantly, the *Params types do not have a good solution now:
      the Python code, due to its dynamic typing, hides the fact that we
      actually have two different types at play: a full type which needs to
      have all keys, and the 'partial' type which has slightly different
      behaviour. I've implemented these in Haskell as two different types,
      Full* and Partial*, which are derived automatically from a single
      Parameter type, together with the associated Fill* functions.
      
      Furthermore, HVParams is even more special, as its contents is not
      fixed but varies per hypervisor type, plus it has the HV_GLOBALS part
      which should not be customisable at instance type (yay for
      exceptions). As such, this should be written in Haskell as a
      multi-constructor type, but it's the only one so far and thus we don't
      have support for it yet.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarGuido Trotter <ultrotter@google.com>
      b1e81520
  19. 13 Jan, 2012 2 commits
  20. 07 Dec, 2011 1 commit