GitLab

The same pattern that has been used for gathering DRBD minors will be
used for DRBD secrets etc.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. since it's a general, randomized function.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. and add a function for random secrets, just as we have in Python
code.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Add flags '--install-image' and '--zeroing-image' to cluster init and
cluster modify man pages.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

... in 'gnt-cluster modify'
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

To avoid further cluttering the already overly long Utils.hs
move the Statistics data type with its related functions into
a separate submodule. This also provides a better encapsulation
of this type where the Constructors are not exported.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

* stable-2.11
  (no changes)

* stable-2.10
  Use more efficient statistics for the standard deviation
  Use statistics updates when allocating on pairs
  Factor score computation through abstract statistics
  Verify the update of the standard deviation statistics
  Add data type for abstract statistics
  Relax test requirements
  Fix gnt-network client wrt instances report
  Fix QueryNetworks wrt instances

Conflicts:
	src/Ganeti/HTools/Cluster.hs: manually apply
	    patch b366d732 to stable-2.11 version
	src/Ganeti/Utils.hs: take all addition
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

* stable-2.10
  Use more efficient statistics for the standard deviation
  Use statistics updates when allocating on pairs
  Factor score computation through abstract statistics
  Verify the update of the standard deviation statistics
  Add data type for abstract statistics
  Relax test requirements
  Fix gnt-network client wrt instances report
  Fix QueryNetworks wrt instances

Conflicts:
	lib/cmdlib/network.py
Resolved by ignoring the change as the corresponding code
has been removed in 2.11.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Instead of using the full sample as statistics providing
enough information to compute the standard deviation, use
a slightly more elaborate one. It contains the standard
statistics count, sum, and sum of squares, which can also
very easily be updated. However, computing the standard
deviation from these values unfortunately is not precise
enough for our purpose (small difference of large values).
Therefore, we also carry the naively computed variance in
the statistics. The latter can also be updated using the
standard statistics and, moreover, this can be done in a
way that does not introduce too large inaccuracies for a
small number of updates.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

When considering the various ways of positioning an instance
on a pair of nodes, make use of the fact that the statistics
are extremely similar (only two nodes changed) and obtain the
new statistics by updating the old one, rather than by recomputing
it from scratch.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Logically separate the computation of the cluster score into
two steps: the computation of the abstract statistics and its
evaluation. In this way, we obtain an abstract value which we
can update instead of recomputing it when considering different
instance placements.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Add a test that verifies that the error introduced by updating
a standard-deviation statistics of a sample with at least two
elements is not too large, as compared to the direct computation.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Our cluster score is a weighted sum of certain sums and
standard deviations of node characteristics. When placing
a single instance, the cluster score of a big number of
quite similar clusters are computed: that of the original
cluster with one instance added, for each possible placement.
To speed up these computations, add an abstract type of
statistics that allows a potentially more efficient update
of a single value.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Instead of insisting on perfect equality of score
allow for numerical inaccuracies and consider everything
all differences in the cluster score smaller than 1e-12
negligible. Given that, by default, a cluster with a
score of less than 1e-9 is considered perfectly balanced,
this relaxation is on the save side.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

.. so that unit tests work without the presence of WConfd..
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Also call it for dropping reservations when a job finishes.

And remove the field for temporary DRBDs, that has been already moved to
WConfD.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. and also a function for dropping all reservations (currently just
MACs).
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

For this, add an internal Tempres data structure that implements
functionality similar to the one in config.py.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. collected from all network interfaces of all instances.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

If a key has no value, it returns a supplied monadic default value.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. so that it's able to handle Kind '* -> *' types such as
'MaybeForJSON'.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. which can contain multiple values for a single key.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This allow to work with sets using the standard lens mechanisms.

Also add a function that creates a lens where a setter function also
uses the information from a getter. This allows to optimize the setter,
in particular in the case of Set, keep the set unmodified, if the inner
function doesn't change anything.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Most often the inner functor is "(,) r" and "traverseOf2" is used to
traverse an effectful computation that also returns an additional
output value.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. by adding sections and moving functions into them.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The '.get' method of dicionaries takes the default
value as positional argument and not as key word
argument. Fixing this as it was accidentally added
in my last patches.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

This patch fixes some calls of subfunctions in
'move-instance'.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

... and by that fixing a TODO as well.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

Let the gnt-network client expect a list of instance names and not
UUIDs as returned by QueryNetworks (by both old and new style query
mechanism).
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Helga Velroyen <helgav@google.com>

QueryNetworks tries to find which instances are connected to which
networks. The query mechanism in Haskell was written back when NICs
referred to a network via its name and not its UUID. Fix luxi to
comply with the current implementation (network slot of NIC object
is a UUID).

Fix old style query mechanism to return a list of names instead of
UUIDs for the instances that are connected to a network.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Helga Velroyen <helgav@google.com>

The last lint fixes to move-instance were a bit too
eager and removed an import line that was actually
necessary. Adding it back again.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This 'main' function was too big anyway.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

Wrong open modes + a 'too-many-branches' error.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

A 'too-many-branches' error.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

This time a 'too-many-branches' error.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

One 'too many branches' error.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

This patch fixes the 'too many branches' lint error
in the lvmstrap tool.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

This patch fixes some lint errors in the 'move-instance'
tool that showed up with a newer version of lint.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

* stable-2.11
  (no commits)
* stable-2.10
  tiered allocation: try canonical search path first
  Add QA config flag for all performance tests
  build-bash-completion: reduce branches
  Convert all the classes to new-style classes
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

* stable-2.10
  tiered allocation: try canonical search path first
  Add QA config flag for all performance tests
  build-bash-completion: reduce branches
  Convert all the classes to new-style classes
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>