1. 24 Apr, 2014 1 commit
  2. 24 Jan, 2013 1 commit
  3. 24 Dec, 2012 1 commit
  4. 17 Dec, 2012 3 commits
  5. 30 Nov, 2012 1 commit
    • Iustin Pop's avatar
      Remove read instances from our Haskell code · 139c0683
      Iustin Pop authored
      
      
      It turns out that optimising 'read' derived instances (via -O) for
      complex data types (like OpCode, or the various objects) can be slow
      to very slow. Disabling such instances results in (time make
      $all_our_haskell_binaries) large compile-time savings and also smaller
      (unstripped) binaries (by a significant amount):
      
      ghc 6.12:        time  htools sz  hconfd sz
        with read:    4m50s 12,244,694 14,927,928
        no read:      3m30s 10,234,305 12,536,745
      ghc 7.6:
        with read:   14m45s 13,694,761 15,741,755
        no read:      3m40s 11,631,373 13,245,134
      
      So let's remove these instances, since we never use read in production
      for our custom types, and even when debugging in GHCI, we can simply
      use the 'show' representation to create the types, without needing to
      actually parse from strings.
      
      Note: for the very slow ghc 7.6 compilation time, I filled a ticket
      (ghc #7450), since it is surprising(ly slow).
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarMichele Tartara <mtartara@google.com>
      139c0683
  6. 21 Nov, 2012 1 commit
    • Iustin Pop's avatar
      Split 'Query.Language.ItemType' in two sub-types · 1283cc38
      Iustin Pop authored
      
      
      The QR_VIA_OP/QR_VIA_LUXI types in Python are using yet another
      validation mode: QR_VIA_OP is the base type, and QR_VIA_LUXI extends
      it (when doing luxi queries). But on the wire they have the same
      representation.
      
      To accommodate this properly, we split the ItemType in two: a
      QueryTypeOp and a QueryTypeLuxi, joining them back together in
      ItemType itself. This requires custom serialisation/deserialisation,
      but allows us to express correctly that at opcode level, we only
      accept a QueryTypeOp, but at Luxi query level, we accept either of
      them.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarGuido Trotter <ultrotter@google.com>
      1283cc38
  7. 20 Nov, 2012 1 commit
  8. 15 Nov, 2012 1 commit
    • Iustin Pop's avatar
      Cleanup THH function use from built module namespace · 32a569fe
      Iustin Pop authored
      
      
      Currently, THH.hs "injects" into the built code names of library
      functions like Text.JSON.makeObj, Ganeti.JSON.fromObj, etc. built
      directly from strings, via (e.g.)
      
        varE (mkName "makeObj")
      
      This means that the "makeObj" name must exist in the target module,
      i.o.w. must be imported there. This leads to the strange case of
      having to have imports that do not appear at all in the used
      (template) code, but are needed to satisfy this "hidden" dependency;
      look at Ganeti/Jobs.hs before this patch, for example.
      
      This is also not very obvious, because we usually import Text.JSON
      anyway; I only stumbled upon it while doing some cleanup work.
      
      So to clean this up, the current patch changes the THH.hs to use not
      string-derived, but identifier-derived names («'identifier» versus
      «mkName "identifier"»); this is better, as the names must be
      resolvable when compiling THH itself (once), and not when compiling
      the multiple derived modules. As you can see, this allows removal of
      extraneous imports from various modules.
      
      Background information: an `mkName "foo"` results in a name of flavour
      NameS (“An unqualified name; dynamically bound”) or alternatively to a
      qualified name, but still dynamically bound. Whereas what we want is a
      statically bound name: `'foo` results in a NameG flavour, “Global name
      bound outside of the TH AST: An original name”.
      
      One more explanation: the change is similar to going from 'x = eval
      "map"' to 'x = map'; the name is no longer dynamically evaluated, but
      statically when the module is compiled. In our case, previously names
      were bound at target module compile time, now they are bound at THH.hs
      compile time.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarGuido Trotter <ultrotter@google.com>
      32a569fe
  9. 10 Oct, 2012 1 commit
  10. 05 Sep, 2012 3 commits
    • Iustin Pop's avatar
      Further hlint fixes · 5b11f8db
      Iustin Pop authored
      Commit 2cdaf225, “Re-enable standard hlint warnings”, got it almost
      right. The only problem is that (confusingly) the default set of hints
      is not in HLint.Default, but in HLint.HLint (it includes Default and
      some built-ins).
      
      After changing the lint file to correctly include the defaults, we had
      another 128 suggestions:
      
        - Error: Eta reduce (2)
        - Error: Redundant bracket (4)
        - Error: Redundant do (17)
        - Error: Redundant lambda (7)
        - Error: Redundant return (1)
        - Warning: Avoid lambda (2)
        - Warning: Redundant $ (42)
        - Warning: Redundant bracket (35)
        - Warning: Use : (1)
        - Warning: Use String (4)
        - Warning: Use camelCase (10)
        - Warning: Use section (3)
      
      which are fixed by the current patch. Note that the 10 "Use camelCase"
      were all due to hlint not “knowing” the idiom of ‘case_’ (it does for
      ‘prop_’), for which I filled
      http://code.google.com/p/ndmitchell/issues/detail?id=558
      
      .
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarRené Nussbaumer <rn@google.com>
      5b11f8db
    • Iustin Pop's avatar
      Fix deserialisation bug in ResultEntry · 3ce788db
      Iustin Pop authored
      
      
      Found via the newly added unit-tests, which test most of the
      serialisation code in Query/Language (except for QueryResult, for
      which we already tests both sub-components separately).
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarRené Nussbaumer <rn@google.com>
      3ce788db
    • Iustin Pop's avatar
      Add some unittests for node queries · b9bdc10e
      Iustin Pop authored
      
      
      These new tests check that:
      
      - no known fields return unknown
      - any unknown field returns unknown
      - the type of the fields is consistent between the getters and the
        field definition
      - the length of each result row corresponds with the number of fields
        queried, and the length of the field definitions returned
      - the length of the rows corresponds to the number of nodes
      - querying fields on empty fields returns all fields
      
      Finally this patch found a bug, in that the pinst_list/sinst_list
      fields were declared as QFTNumber (copy-paste error from
      pinst_cnt/sinst_cnt), yay!
      
      I also changed genEmptyCluster to ensure that it generates unique node
      names, so that the number of result rows is consistent with what we
      requested, and switched ResultEntry from a normal constructor to
      record syntax, so that we can extract the fields without having to use
      pattern matching.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarRené Nussbaumer <rn@google.com>
      b9bdc10e
  11. 04 Sep, 2012 5 commits
  12. 03 Sep, 2012 2 commits
    • Iustin Pop's avatar
      Add Query support for Nodes (no filtering, no RPC) · 046fe3f5
      Iustin Pop authored
      
      
      This is the initial support for Query2: basic infrastructure (except
      filtering) and node query support (without RPC).
      
      It implements all the fields (tests by comparison with list-fields on
      the Python side), except that:
      
      - filter is not done
      - since RPC is not integrated yet, the runtime gathering/computing is
        simply stubbed out
      
      However, the infrastructure seems pretty reasonable, so I'm sending as
      is.
      
      Note that I've split the functions/declarations into multiple files,
      to keep each file clean and readable.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarAgata Murawska <agatamurawska@google.com>
      046fe3f5
    • Iustin Pop's avatar
      Stub query2 call integration into QueryD · 4cbe9bda
      Iustin Pop authored
      This patch corrects the definitions in Qlang.hs to match what Python
      expects on the wire; this means replacing some manual data type
      definitions with 'buildObject' so that we get serialisation (and field
      names) for free, adding (manually) JSON instances for types which are
      not represented as objects in JSON, etc. Due to more TH usage, I had
      to shift some definitions around, since TH breaks the "define in any
      order" property (
      
      ).
      
      After that, we simply add a call into the stub Query/Query.hs module
      which, for all queries, responds with "query not supported". The
      reason for the deep directory structure is because I want to separate
      the functionality into multiple submodules, for smaller module size
      and readability.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarAgata Murawska <agatamurawska@google.com>
      4cbe9bda
  13. 28 Aug, 2012 4 commits
    • Iustin Pop's avatar
      Reduce some more code duplication and split code · 62377cf5
      Iustin Pop authored
      
      
      The Qlang module defines ResultStatus, but it was already defined in
      Ganeti/Luxi.hs; let's remove the duplicate definition from there since
      the proper place is in the newer module.
      
      Also, in order to ease testing, we split some confd functions into a
      separate module; this can be imported easily into QC for testing.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarAgata Murawska <agatamurawska@google.com>
      62377cf5
    • Iustin Pop's avatar
      Remove obsolete QrViaLuxi type · 8a9ee1e9
      Iustin Pop authored
      
      
      The actual query definitions are now in Qlang.hs, so let's use the
      ItemType from there instead of luxi-defined type (which is also
      incomplete).
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarAgata Murawska <agatamurawska@google.com>
      8a9ee1e9
    • Iustin Pop's avatar
      Rename Query2.hs to Qlang.hs · dc6a0f82
      Iustin Pop authored
      
      
      While starting to use the new filter types, I realised that what is
      currently implemented is the equivalent of `lib/qlang.py', not
      `lib/query.py', since we only deal with data types for now and not the
      actual query runtime functionality (RPC, config, etc.).
      
      Let's rename the file to be more consistent with the Python code base.
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarAgata Murawska <agatamurawska@google.com>
      dc6a0f82
    • Iustin Pop's avatar
      Implement Query2 filter JSON (de)serialisation · e8a25d62
      Iustin Pop authored
      
      
      This adds support for encoding/decoding Query2 filters to/from JSON,
      in (hopefully) the same format as the Python code generates.
      
      It also adds a simple unit-test to check that this conversion is
      idempotent. Of note here is that, since the Filter data type is
      recursive, we have to manually ensure that the generator for it
      correctly "shrinks" at each step (first version crashed hard my
      workstation after eating ~8GB of ram :).
      
      Compared to the current Query2 implementation, the following changes
      were done:
      
      - style: shortened some names to match the Python ones (LessEqualThan
        → LE, etc.)
      - changed FilterValue from string to an ADT that can encode both
        quoted strings and numeric values, since this is actually what
        qlang.py generates
      - added support for EmptyField, which in hindsight it's an obvious
        missing part :)
      Signed-off-by: default avatarIustin Pop <iustin@google.com>
      Reviewed-by: default avatarAgata Murawska <agatamurawska@google.com>
      e8a25d62
  14. 06 Jul, 2012 1 commit
  15. 14 Jun, 2012 1 commit
  16. 12 Jun, 2012 1 commit