GitLab

* stable-2.10
  Update tag limitations
  Fix typos in doc/design-storagetypes.rst
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

We always supported underscores in tags since \w in pythons
interpretation of regular expressions does include the underscore.
While this might have happened by accident, there is no reason to
change the implementation. The motivation for the restriction
was to avoid accidents with wrong (manual) shell escaping; the
underscore, however, has no special meaning to the shell. So
just make the documentation say what we implemented.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

As the LURenewCrypto is a pain to debug, this patch
adds a lot more logging of events to the method.
Note:
- As renew-crypto is a relatively rarely used operation
  in a normal production cluster, this won't clutter up
  real user's log files.
- Most of the messages are in debug mode, so they would
  anyway just show up in log files of clusters run in
  debug mode (as for example our QA clusters.
- A few log messages are in error mode, which is
  intentional as they log more details about the errors
  than is given in the feedback functions.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Gangbiao Liu <lgb.nwpu@gmail.com>
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

* stable-2.10
  Make getFQDN prefer cluster protocol family
  Add version of getFQDN accepting preferences
  Make getFQDN honor vcluster
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

In getFQDN, if the primary IP family of the cluster is known,
provide it as a hint.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Still the best way to get the "canonical name" of a host
is to look up its host name and reverse look up the IP
address obtained. Obviously, that the result depends
on the protocol family used. So accept hints on which
family to prefer.

Note that getFQDN is exported on higher branches, so we
keep its type stable to avoid semantic merge conflicts.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Make the official (exported) getFQDN function honor the vcluster
setup. In this way, also Haskell daemons can obtain their name
correctly in a vcluster. In particular, voting will work.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Cherry-picked-form: c29501f6

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

As soon as we have networks bigger than a /22, the lower
index of a line in the usage map can have 4 digits. So
prepare the formating for this.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

"gnt-network info" also provides information about the mapping of used
IPs in that network. This mapping is obtained from the query as a long
string without any spaces. It seems that textwrap.wrap does not perform
well on those strings. Therefore, manually wrap that string, thus bringing
down the time "gnt-network info" takes for a /16 from 1.5 minutes to half
a second. Fixes issue 1016.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This adds a comment about the recent improvements of
'gnt-cluster renew-crypto --new-node-certificates'
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* stable-2.10
  fix typos in design-file-based-storage.rst doc
  Switch to our osminor
  Provide an alternative for os.minor working around its bug
  Fix typo
  CanTieredAlloc test: make instances big enough
  After master-failover verify reachability of master IP
  Report failure to deactivate old master IP in exit code
  Expose warnings during master-failover
  Fix manpage for gnt-cluster copyfile
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Fix typos in doc/design-file-based-storage.rst.
Signed-off-by: Gangbiao Liu <lgb.nwpu@gmail.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

...and thus work around the bug in os.minor
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Python's os.minor still contains an old definition, whereas the
current one has changed. So we add our own definition working around
this bug.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Lisa Velden <velden@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Fix a subtle bug in the CanTieredAlloc test. The property we were
testing for was actually not valid with the addition of an ipolicy.
The lower policy bound for disk was 4 times the unit for disks (256M).
So, a valid node have free disk only slightly larger than the ipolicy
lower bound. When shrinking disks, however, we go in steps of full
units---and thus might miss the window of opportunity if that is smaller
that a full unit. So, only use nodes that have at least 5 units worth
of resources for every resource.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

...and warn if it is not. Note that the master activates
the master IP in an asynchronous task and will continue
even if that fails.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

If we failed to disable the old master IP, the master failover
did not fully succeed, hence that should be reported in the
exit code. Nevertheless, the best is to proceed, as it is
better to have working cluster, albeit only reachable via
the primary IP of the new master node (and not the cluster
master IP), than not have a cluster. Also note, that for
this reason master will start up even if it cannot set
the cluster master IP. So, while there, fix the warning
message as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

During master failover, there are some situations where problems
occur but the best thing to do is to carry on. These problems
are logged using the usual mechanism. However, a user usually
does not look into the log file unless the command executed
returns some hints that something might have gone wrong.
So also return the warnings as an additional return value,
allowing the CLI to report properly.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Change "copyfile" position in example
Signed-off-by: Lisa Velden <velden@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

* stable-2.10
  Fixed typos
  Pass correct params in move-instance
  In CanTieredAlloc test set IPolicy
  Make genInstanceMaybeBiggerThanNode honor policy lower bound
  Also export a null ISpec
  Support instance generation within ranges
  Add a function to leave the list monad

Conflicts:
	src/Ganeti/BasicTypes.hs (take all the imports)
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

move-instance incorrectly passes backend params as the OS params when
moving an instance. This patch fixes this.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Changing the test to allocate on nodes with a non-trivial
instance policy has two advantages.

- We test in a more realistic (and also more challenging)
  environment.

- Once the lower limit of the ipolicy is reached no more
  allocations are possible, thus having fewer shrinking
  rounds---and hence speeding up the test by a factor of 4.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Note that in all current calls to this function, the node has the
nullIPolicy, to the semantics does not change here.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

This is an ISpec for an instance with 0 resources. It can serve as a trivial
lower bound where we have to provide one but do not actually care.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

This will be used to generate instances big enough to fall within
a given instance policy.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

The list monad provides convenient syntax for non-deterministic
algorithms. Add a function leaving that monad with this intuition
in mind.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Cherry-picked-from: a1da8a50

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

When looking up configuration data of instances which don't
exist, the code so far fails with a cryptic error messages
about NoneType not having an attribute. Although actually
this situation should not happen, let's at least throw an
exception with a proper description.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This fixes a minor problem in LURenewCrypto, where we
use the exception variable after the try/except block.
By assigning it to a previously initialized one, we make
sure that we can access this information.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

* stable-2.10
  Make QA fail if KVM hotplugging fails
  Always preserve QA command output
  Don't lose stdout/stderr in AssertCommand
  qa_utils: Allow passing fail=None to AssertCommand
  qa_utils: Make AssertCommand return stdout/stderr as well
  Allow plain/DRBD conversions regardless of lack of disks
  Add support for ipolicy modifications to mock config
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Due to hard-coded timeouts used when listing instances, the Xen
unit tests took around 60s to execute. The key offender was a timeout
of five seconds used for an unsuccessful listing of instances. This
patch refactors the code slightly to define the timeout and delays used
in one place, allowing these to be changed during testing to a more
acceptable value. As a result, these tests take around 5s to execute.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Xen's utilities used by Ganeti to report instance state can have
interesting quirks, such as reporting that an instance resides in
different and somewhat contradictory states.

This patch improves the situation by ignoring the paused state, and
encoding some of the more exotic combinations that may appear.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Unlike almost all other modification commands, hotplugging a device and
failing does not result in an error code showing that the operation was
unsuccessful. Because of this, the QA ignored hotplugging failures. To
fix this, this patch makes the QA examine the output of the command for
signs of failure.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

A previous patch suppressed command output if the command succeded,
which reduces the amount of information we have in the QA, especially
warnings or the like. This patch restores the output, while still
ignoring the use cases in which we really do not care whether the
command succeeds or not.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

If a command fails/succeeds that should not do so,
show stdout and stderr. This information can be
relevant for debugging.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Cherry-picked-from: 7f7c9c2c

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This is for cases where we don't care about the exit code.
Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Cherry-picked-from: 226455df

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>