File system paths moved from constants to pathutils.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

File system paths moved from constants to pathutils.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

File system paths moved from constants to pathutils.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

File system paths moved from constants to pathutils.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

File system paths moved from constants to pathutils.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

File system paths moved from constants to pathutils.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

File system paths moved from constants to pathutils.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

File system paths moved from constants to pathutils.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

File system paths moved from constants to pathutils.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

File system paths moved from constants to pathutils.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is inpreparation for the implementation of virtual clusters. Many
paths will change based on an environment variable and are no longer
constant and should no longer be in “constants.py”. Since “constants.py”
is already huge a number of other paths are also moved in the process.

For now a wildcard import is used to re-export all paths from
“constants.py” (this will change over coming patches). Daemon log paths
have been changed to use a function. “RUN_GANETI_DIR” was moved to
“RUN_DIR” as the latter was only used in “constants.py”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is needed for virtual clusters where a non-root user is used to run
the daemons.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

It was missing in commit 243cdbcc on May 7, 2008.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

Also, the value of always_failover is not hard-coded any more, and it's
possible to disable changing the always_failover parameter, if needed.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

If an administrator sets always_failover, it means that there is no need
for another explicit approval to failover instead of migrating.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The old primary node was being locked to do RPC, but we don't lock nodes
for RPC anymore, so the lock is useless.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The biggest change is going from a command line parameter for specifying
a daemon's directory to using an environment variable. Some other things
are clarified.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This adapts the type of some request input fields to more stricter ones.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This was used to verify the result of instance allocation. However, this
is not logic which belongs to the cmdlib rather than to the iallocator
request itself. So abstract it by its own function. And update the
ValidateResult accordingly.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

objects.Disk.ComputeLDParams: Simplify structure, don't use and
overwrite local variables.

rpc.AnnotateDiskParams: Replace explicit loop.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Also update the copyright header.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

For some reason pylint can't determine, that the members are calculated
upon runtime and complains. This is weird as the same mechanism works on
opcode/objects. I haven't found the issue for it yet, so let's just
ignore the warnings for now.

This is a separate patch for easier revert later on, once it's fixed.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add the new multi-alloc request to the test allocation framework. For
now we just create --count identical instances. This might need further
improvements later.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Now that we've the refactoring done, we can easily add the new mode and
everything works.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

After the refactor, the old calls are not working anymore, so we switch
to the new ones.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

The IAllocator class was handling all the requests on its own, passing
in parameters on top level which works, but is hard to maintain and not
flexible.

With the upcoming change to the IAllocator for MultiAllocate we can't
use the toplevel parameters anymore. Therefore, we refactor the code
into separate classes, which just do their own stuff. It works similiar
to Opcode containers.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

As we will introduce another set of containers using the __slots__ trick
we abstract away as much as possible to separate bases classes. The
child classes then adapt them for their needs. This leads to less code
duplication.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This function will also be needed for the iallocator refactor. Also this
function does not belong to cmdlib but rather to rpc. Lets move it
there.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This makes it possible to better untangle the IAllocator code, which
would otherwise lead to a recursive import structure as we need those
helpers there as well.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Commit f037e9d7 added command line flags to filter jobs, but the
documentation was not updated. Also mention the ability to use
more complex filters.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

We can build the test groups directly in the `testSuite' helper,
instead of doing it (much later) in the test harness.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Since the recent commits improved the speed of the two "slow" test
groups to regular test speed, we can remove this kludge and simplify
significantly our test runner, yay!

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

The Cluster object, as it is defined right now, has many '[String]'
members, which means that in a standard arbitrary generator these will
become very big, which is the reason for the current slowness of the
test 'Config_serialisation'.

By resizing the generator to 8 (arbitrary chosen to limit the list
length and the string sizes), and by reducing a bit the node count, we
can make this test be as fast as the others (about 10x
improvement). This means we can test more cases, for the same cost.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This test has a few deficiencies, which this patch addresses:

- using arbitrary 1 or 2 node count for allocation is obsolete,
  nowadays we need to use a number appropriate for the instance's disk
  template (and we should remove that parameter…)
- generating a random node is sub-optimal, since we could generate an
  offline node, and no instance will fit on a cluster composed of only
  offline nodes
- generating arbitrary instances "such that" they can be allocated is
  an expensive test; let's rather generate instances smaller than our
  template node, and add a check that they indeed can be allocated
- using boolean return type, instead of nicely annotated properties

For the nice annotation and the extra check, we need to change a
helper function's signature, so that we can extract a bit more
information out of it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, this test is very slow. Upon investigation, this is due to
how `tieredAlloc' works:

- tries to allocate one instance
- if failed, shrink the instance by the "most failed" resource
- restart

In this algorithm, if the "most failed" resource is e.g. memory, and
the maximum available memory is much smaller than the current
template, it means we will have to shrink and try to allocate many
many times until we finally get with the to-be-allocated instance
memory size to a reasonable value. In the real world, this is not the
case, but when testing with arbitrary memory/node values, it can be
that we execute the shrink hundreds of thousands of times per test.

So we "improve" the test by directly generating an instance just
slightly bigger than the node, so that we don't have to shrink too
many times. This requires a new export from test/…/Instance.hs.

Additionally, we allow up to 5 instances to be tiered-allocated, and
we cleanup the test checks, making the conditions much, much more
readable (IMHO).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This test expands the "single-alloc-no-rebalance" by allocating a few
instances on a small cluster, and ensuring that after we allocate all
of them, either we can't rebalance or if we rebalance the score
improvement is very small.

The last condition is needed because sometime rounding errors (we're
using double-precision floating point) can accumulate and result in
what is a no real change in the cluster state, but with an
infinitesimal score decrease (e.g. 1e-14).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Currently, this just checks that a cluster cannot be rebalanced after
a single instance allocation. However, we can also test whether the
allocation decision computed a correct new cluster score, by checking
that against the one computed from the actual new node list.

Also, for nicer display, we convert the test from a Boolean to a
Property, with nice annotations.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

* devel-2.6:
  Fix bug in non-mirrored instance allocation
  Fix gnt-debug iallocator

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

* stable-2.6:
  Fix bug in non-mirrored instance allocation
  Fix gnt-debug iallocator

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>