- 04 Sep, 2012 12 commits
-
-
Iustin Pop authored
Except for Ganeti.HTools.JSON, which needs rename, we split all the other test suites into separate files. We have to add another common test helper, due to import dependencies (sigh), but otherwise this split is pretty straightforward. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
René Nussbaumer <rn@google.com>
-
Iustin Pop authored
… from QC.hs into their own files, again mirroring the production code source tree. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
René Nussbaumer <rn@google.com>
-
Iustin Pop authored
This required lots of other code moves, so I created it as a standalone patch. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
René Nussbaumer <rn@google.com>
-
Iustin Pop authored
This splits the confd/utils tests, and adds the TestCommon module for shared test code. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
René Nussbaumer <rn@google.com>
-
Iustin Pop authored
This is the first file split out from QC.hs - an easy one, since it has just one test. The patch changes the way we build hpc excludes, since now we'll have many modules that need to be excluded, and hpc doesn't seem to be able to do wildcards (it can do packages, but all our code is not-package-based). Further splitting will be done in bigger batches. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
René Nussbaumer <rn@google.com>
-
Iustin Pop authored
This patch starts the move of the test haskell code from `htools/Ganeti/HTools/' to its more proper place of `htest/Test/Ganeti'. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
René Nussbaumer <rn@google.com>
-
Iustin Pop authored
This moves the last (I think) htools-related bits out of test/ under htest/. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
René Nussbaumer <rn@google.com>
-
Iustin Pop authored
This is another rather trivial patch, moving all the (htools) shelltests to their own directory. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
René Nussbaumer <rn@google.com>
-
Iustin Pop authored
Following up on the program moves, we now move the test data files. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Agata Murawska <agatamurawska@google.com>
-
Iustin Pop authored
This is the first commit of a series that will attempt to cleanup the test code organisation, which evolved somewhat organically from the initial pure htools functionality. The proposed organisation of the tree will be as follows: - htools (or maybe renamed to haskell or hs): only production code - htest: top-level test directory, containing test.hs, static helper scripts, etc. - htest/Ganeti/*.hs: modules implementing the actual test properties and test cases for the correspondingly-named production code modules - htest/data: containing test data files for the test cases This particular patch moves all the test code (test.hs, hpc-htools.hs symlink) and helper scripts (offline-test.sh, etc.) from htools/ to htest/, while updating the files themselves (if they had paths mentioning htools/), .gitignore and the Makefile. The only special mention is that in Makefile, we used to have a BINARY shell variable in binary build rule; that was computed via stripping `htools/' prefix; I've cleaned that and replaced with $(notdir $@); even though it's duplicated a few times, it leads to more readable make output (and easier to copy-paste). Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Agata Murawska <agatamurawska@google.com>
-
Iustin Pop authored
Per the new query module hierarchy, rename Qlang to Query/Language and Queryd to Query/Server. This way, all query-related functionality is now "contained" in the Query/ directory. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Agata Murawska <agatamurawska@google.com>
-
Iustin Pop authored
This adds basic infrastructure for filtering (fully functional except, as usual, for runtime data), and then uses it for node queries. Since the filtering exports regex matching as an external functionality, we have to use a regex library. There are many flavours of these in Haskell (see http://www.haskell.org/haskellwiki/Regular_expressions ), but since we want to be as compatible as we can with Python's, we use the regex-pcre one, which is a wrapper to the widely used 'pcre' library. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Agata Murawska <agatamurawska@google.com>
-
- 03 Sep, 2012 3 commits
-
-
Guido Trotter authored
This design describes a tool that will perform automatic repairs on instances when they are detected to be unhealthy (living on offline or drained nodes, at the moment). These repairs can be scheduled automatically or requested as a one-off by a tool or person. Signed-off-by:
Guido Trotter <ultrotter@google.com> Reviewed-by:
Bernardo Dal Seno <bdalseno@google.com> Reviewed-by:
Iustin Pop <iustin@google.com>
-
Iustin Pop authored
This is the initial support for Query2: basic infrastructure (except filtering) and node query support (without RPC). It implements all the fields (tests by comparison with list-fields on the Python side), except that: - filter is not done - since RPC is not integrated yet, the runtime gathering/computing is simply stubbed out However, the infrastructure seems pretty reasonable, so I'm sending as is. Note that I've split the functions/declarations into multiple files, to keep each file clean and readable. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Agata Murawska <agatamurawska@google.com>
-
Iustin Pop authored
This patch corrects the definitions in Qlang.hs to match what Python expects on the wire; this means replacing some manual data type definitions with 'buildObject' so that we get serialisation (and field names) for free, adding (manually) JSON instances for types which are not represented as objects in JSON, etc. Due to more TH usage, I had to shift some definitions around, since TH breaks the "define in any order" property (
☹ ). After that, we simply add a call into the stub Query/Query.hs module which, for all queries, responds with "query not supported". The reason for the deep directory structure is because I want to separate the functionality into multiple submodules, for smaller module size and readability. Signed-off-by:Iustin Pop <iustin@google.com> Reviewed-by:
Agata Murawska <agatamurawska@google.com>
-
- 28 Aug, 2012 6 commits
-
-
Iustin Pop authored
The Qlang module defines ResultStatus, but it was already defined in Ganeti/Luxi.hs; let's remove the duplicate definition from there since the proper place is in the newer module. Also, in order to ease testing, we split some confd functions into a separate module; this can be imported easily into QC for testing. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Agata Murawska <agatamurawska@google.com>
-
Iustin Pop authored
And expand cli.GetClient() to allow opening the query socket, instead of the main master socket. Finally, enable the query socket use in gnt-cluster version, since that is already implemented fully in Queryd.hs/hconfd. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Agata Murawska <agatamurawska@google.com>
-
Iustin Pop authored
While starting to use the new filter types, I realised that what is currently implemented is the equivalent of `lib/qlang.py', not `lib/query.py', since we only deal with data types for now and not the actual query runtime functionality (RPC, config, etc.). Let's rename the file to be more consistent with the Python code base. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Agata Murawska <agatamurawska@google.com>
-
Iustin Pop authored
Commit 5a1e31b4 (Add infrastructure for, and two extra hlint rules) was intended to add two *extra* hlint rules, but I didn't realise at that time that "--hint" when first used overrides the built-in lints. As such, since then we were basically running with just those two rules, which resulted in many uncaught warnings/errors. This patch fixes that (by importing the standard lint rules in our custom hints file), and then goes to fix all the warnings that a current hlint gives me. Compared to our current style, we have just a few additions: - zipWithM instead of map foo . zip … - 'exitSuccess' instead of 'exitWith ExitSuccess' - more uses of '.' Additionally, we have to silence a case where hlint doesn't realise why we are using '\e -> const (return False (e :: IOError)' instead of just '\e -> return False' or even 'const (return False'). One warning that is generated by hlint ("Use void") can't be fixed until we deprecate GHC 6.x, as only GHC 7 has the 'void' function in Control.Monad. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Agata Murawska <agatamurawska@google.com>
-
Iustin Pop authored
This is just a new module that exports a runQueryD function, that can be imported to run a separate thread handling the luxi requests. Currently it needs access just to the configuration, in the future it will need access to an RPC runner too. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Agata Murawska <agatamurawska@google.com>
-
Agata Murawska authored
We introduce typeclasses for RPC call and result and create a typeclass that binds the two together. For that we need to use MultiParamTypeClasses and FunctionalDependencies language pragmas, which allow us to ensure that RPC result type can be deduced based on the RPC call type. Signed-off-by:
Agata Murawska <agatamurawska@google.com> Reviewed-by:
Iustin Pop <iustin@google.com>
-
- 27 Aug, 2012 1 commit
-
-
Iustin Pop authored
This is done so that all current branches can run with newer pep8; note that instead of fixing the problems (like I did on master), I've just silenced more. These should *not* be merged onto master! Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
René Nussbaumer <rn@google.com>
-
- 23 Aug, 2012 1 commit
-
-
Iustin Pop authored
Debian Wheezy will ship with this version, and it has many improved checks compared to 0.6, so let's: - bump version in the docs - silence some new checks that are wrong due to our indent=2 instead of 4 - fix lots of errors in the code where the indentation was wrong by 1 or 2 spaces - fix a few cases of == True, False, None and replace with 'is' - re-indent some cases where the code is OK, but pep8 complains Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
René Nussbaumer <rn@google.com>
-
- 14 Aug, 2012 1 commit
-
-
René Nussbaumer authored
This is the design doc for the bulk instance creation. You can more details in the doc itself. Signed-off-by:
René Nussbaumer <rn@google.com> Reviewed-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Agata Murawska <agatamurawska@google.com>
-
- 09 Aug, 2012 1 commit
-
-
René Nussbaumer authored
Due to the fact how the automake system works it doesn't rebuild already prebuild files in distcheck. This lead to a bug, where a rebuild of the documentation was failing because we missed the fact that the files were missing from the archive. By adding distrebuildcheck we workaround that issue by running a maintainer-clean which also removes prebuild files. Signed-off-by:
René Nussbaumer <rn@google.com> Reviewed-by:
Iustin Pop <iustin@google.com>
-
- 08 Aug, 2012 1 commit
-
-
René Nussbaumer authored
Signed-off-by:
René Nussbaumer <rn@google.com> Reviewed-by:
Iustin Pop <iustin@google.com>
-
- 23 Jul, 2012 8 commits
-
-
Iustin Pop authored
This is just begging to be converted to a standard replace_vars_sed rule, instead of custom sed calls. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Guido Trotter <ultrotter@google.com>
-
Iustin Pop authored
A rule of type "a/%: a/%.in" will also match "a/b/%: a/b/%.in", so no need for the explicit examples/hooks rule. As for the man rules, they are identical and thus can be collapsed. We still have the problem that globally, not all our %.in to % transformations are identical; this is suboptimal and should be cleaned sometime… Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Guido Trotter <ultrotter@google.com>
-
Iustin Pop authored
Generic rules like: %: %.in have the downside that the source (%.in) itself matches the target (via %.in: %.in.in). This leads to things like: Looking for a rule with intermediate file `doc/examples/hooks/ipsec.in.in'. Trying pattern rule with stem `ipsec.in.in'. Trying implicit prerequisite `doc/examples/hooks/ipsec.in.in.in'. Trying pattern rule with stem `ipsec.in.in'. Trying implicit prerequisite `doc/examples/hooks/ipsec.in.in.in'. Looking for a rule with intermediate file `doc/examples/hooks/ipsec.in.in.in'. Rejecting impossible implicit prerequisite `doc/examples/hooks/ipsec.in.in'. To fix this, we need to tell make that such rules are terminating, so that it doesn't recurse into them. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Guido Trotter <ultrotter@google.com>
-
Iustin Pop authored
GNU Make contains some (ancient) implicit rules, that try to _automatically_ extract source files from RCS/SCCS version control systems. This is unneeded, and it pollutes the make -d output significantly: after removing these rules (by defining empty targets for their patterns), make -d line count goes from 5.3K to 3.3K. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Guido Trotter <ultrotter@google.com>
-
Iustin Pop authored
Commit dc7d2c49 introduced the use of BUILT_SOURCES to work around missing dependencies. However, on closer reading of the gmake manual, BUILT_SOURCES is mainly used in cases where the dependencies are not know before the build starts (e.g. with auto-generated C header files). Additionally, there are a number of drawbacks to BUILT_SOURCES, which we already had to work around (e.g. in commit eb732fb5 ). Since we know all our dependencies statically, there's no need to use this special variable. Let's rename it to GENERATED_FILES, which doesn't have a special meaning to make, and add a few missing dependencies (one of which was already broken by make -j on a clean tree). After this change, running make in a fully built tree is finally "clean": $ make make: Nothing to be done for `all'. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Guido Trotter <ultrotter@google.com>
-
Iustin Pop authored
Currently, if one runs 'make' in an already fully-built tree, this is the result: cd . && test -h "ganeti" || { rm -f ganeti && ln -s lib ganeti; } make all-am make[1]: Entering directory `/tmp/test' cd . && test -h "ganeti" || { rm -f ganeti && ln -s lib ganeti; } make[1]: Leaving directory `/tmp/test' This is because commit dc7d2c49 added 'ganeti' (which is a PHONY target) to BUILT_SOURCES, and since that is a dependency of other, real targets, it means the ganeti target is always remade. To fix this, we keep ganeti as a PHONY target, but we remove it from the 'built_base_sources' target, and instead we only remake it manually in the stamp-directories target. A make run now is just: make all-am make[1]: Entering directory `/tmp/test' make[1]: Nothing to be done for `all-am'. make[1]: Leaving directory `/tmp/test' Note that we can't get rid of the all-am since we use BUILT_SOURCES. We also remove the comment of BUILT_SOURCES since it no longer depends on PHONY targets. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Guido Trotter <ultrotter@google.com>
-
Iustin Pop authored
Commit c964d962 changed the way we create directories, by two things: - unifying all dependencies and ad-hoc directory creation into a single target (all_dirfiles) - changing how directories are created from a stamp file to .dir files in each directory The first item is a very good one, but the second item is debatable: there's no per-se advantage of .dir files versus a single one, top-level, since both the .dir file and stamp-directories creation are depending on Makefile, which is the only one which can introduce new directories. On the other hand, moving back from .dir files to stamp-directories has an advantage: "make -d | wc -l" does from ~8.7K lines to ~5.3K lines, because we eliminate the many .dir files and their multiple implicit and explicit dependencies (the %/.dir files fall under multiple patterns). Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Guido Trotter <ultrotter@google.com>
-
Iustin Pop authored
Seen while debugging make rules. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Guido Trotter <ultrotter@google.com>
-
- 20 Jul, 2012 1 commit
-
-
Iustin Pop authored
Reading the automake documentation, it seems that explicit calls to $(MAKE) need $(AM_MAKEFLAGS) passed, so let's do that. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
René Nussbaumer <rn@google.com>
-
- 06 Jul, 2012 3 commits
-
-
Iustin Pop authored
There are two current issues with the coverage values: - we don't import all modules, thus leading to incomplete coverage results (too optimistic); - we use hpc in its default mode (intersection), which means that even modules which do have coverage results but are not used in all binaries we test will be dropped from the results; thanks to Agata, passing --union to hpc is enough to have better results (don't remember why this wasn't there in the first place…) After adding more modules to the import list and fixing the combining mode, we now have a complete list of modules in coverage results, many with zero coverage, so our overall coverage has dropped to about 60%. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Guido Trotter <ultrotter@google.com>
-
Iustin Pop authored
Note however that this won't work correctly with older compilers, due to ghc issue #4256; however, it's a handy way to build complete a TAGS file for Emacs. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Guido Trotter <ultrotter@google.com>
-
Iustin Pop authored
Currently, we build the Haskell unittests with custom GHC flags, because we (I) were quite lazy when initially writing the unittests. This is not a problem for the tests themselves, but it creates problem when (for example) one would want to pass all $(HS_LIB_SRCS) to the compilers; this is not doable unless we "degrade" the flags used for all modules, instead of just for QC.hs. So we do two things to fix this: - first, we go and add type declaration to all functions that were missing them (in QC), and fix the couple of cases of monomorphism restrictions; this gets us rid of -fno-warn-missing-signature and -fno-warn-monomorphism-restriction - then, we move the actually important remaining options (-fno-warn-orphans and -fno-warn-unused-imports; see the explanation for the latter in the newly added FIXME) to a compiler pragma in the file, so that when building the unittests only this file is using the extra options And finally we can then drop the other unused options (-fno-warn-missing-methods anmd -Wwarn), leaving htools/test use simply -fhpc. This is more in-line with the other files, and thus we can handle all of HS_LIB_SRCS the same. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
René Nussbaumer <rn@google.com>
-
- 04 Jul, 2012 1 commit
-
-
Iustin Pop authored
This is trivial, but can be used easily from automated builds to check that the git tree is clean: files are not modified and untracked files are not present. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Bernardo Dal Seno <bdalseno@google.com>
-
- 29 Jun, 2012 1 commit
-
-
Iustin Pop authored
This patch works around an issue in our build system. Since we don't use cabal or 'ghc -M', we don't track actual dependencies in our Makefile; this in turn means that editing a file that only is used in the main 'htools' binary will keep triggering 'ghc --make' for the hconfd binary; and this actually takes a bit of time to look at all the dependencies. So to work around this, we add a touch after each ghc --make, such that we trigger the unneeded build at most once. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
René Nussbaumer <rn@google.com>
-