GitLab

Merged from branches/ganeti/ganeti-1.2 r1648
Use static values for new hvm instance flags

Reviewed-by: iustinp

This is a result of the log timestamp changes.

Reviewed-by: imsnah

The 'old-style' info, error, debug logs do not make much sense. This
patch unifies the SetupLogging and SetupDaemon functions. As a result,
all the commands logs to a 'commands.log' file.

The patch also changes the log setup to keep going if there's an error
in setting up the file logging but we're logging to stderr.

Also, burnin now logs to its own file (burnin.log).

Reviewed-by: ultrotter

Currently, the feedback_fn argument to SubmitOpCode is no longer used.
We still need it in burnin, so we re-enable it by making the code call
that function with the msg argument in case feedback_fn is callable. The
patch also modifies burnin to accept the new argument format (msg is not
a triple instead of a string).

The patch also removes the ‘proc’ argument as it's obsolete; instead we
can accept a luxi.Client instance (noone uses this right now).

Reviewed-by: ultrotter

In 2.0 burnin submits job to the master daemon, so it doesn't need to
create an internal Processor anymore. Even if the processor is not used
anywhere in the burnin code it was still initialized as a leftover of
how burnin used to work. Fixing this.

Reviewed-by: iustinp

Currently the iallocator option is ignored by burnin at import/export
time even if passed in. With this patch it becomes used. The log message
used by the importer is also changed to reflect this.

This patch also improves import/export on the non-iallocator case:
  - The secondary node is not passed anymore on non-mirrored templates
  - On mirrored templates the secondary node is logged

Reviewed-by: imsnah

- Shorter code
- Reorder arguments to logger.SetupLogging calls to make more sense

Reviewed-by: iustinp

Currently: the LURemoveInstance.Exec() method uses the ignore_failures
attribute of the OpRemoveInstance opcode, but it doesn't check for its
existence. The patch adds this attribute to _OP_REQP and to all the
places where this opcode was created.

This attributes is always passed by gnt-instance, but burnin didn't pass
it so it can fail if it enters the 'fail to remove disks' branch of the
method (which is why it was not triggered until now).

Reviewed-by: ultrotter, imsnah

This patch removes the lock and the lock options from cli.py and moves
them to the master.

Later during development we can remove it completely, but for now it's
good to protect any other tool that uses the lock directly.

Reviewed-by: imsnah,ultrotter

This patch converts the cli.py SubmitOpCode method to use the unix
protocol and thus execute the opcodes via the master.

The patch allows a partial burnin to work with the master. Currently the
query opcodes, since they are executed via the SubmitOpCode, are
executed inside a job too, which is suboptimal, but they work fine.

The cmd lock has been removed from the master, but the cli.py still
takes the lock. This is ok for this in-progress patch (since the master
still has only one executor thread). This will be fixed in a future
patch.

Reviewed-by: ultrotter

This patch implements secondary replace via the iallocator. The new
opcode parameter 'iallocator' behaves like this: if passed, it will
always compute and assign a new secondary, behaving in effect as if the
secondary node has been passed. It conflicts with actually giving the
secondary too.

[Note: not tested with remote_raid1, but the code should behave the
same, we only touch CheckPrereq and we assign a node.]

The patch also adds burnin support for the replace secondary operation;
with this in place, burnin can fully work with auto-assigned nodes.

Reviewed-by: ultrotter

This simple patch adds memory size for the burnin instances, which helps
testing allocator algorithm response based on the available node memory
and instance memory size values.

Reviewed-by: ultrotter

This patch adds the two attributes file_storage_dir and file_driver
to the CreateInstance opcode in the ImportExport method, since
otherwise it will fail with the file storage patches.

Reviewed-by: iustinp

When the allocator or a non-mirrored template are used the message shown by
burnin at instance creation time was wrong. Fixing it.

Reviewed-by: iustinp

Make burnin remove exports after importing the instance back.

Reviewed-by: iustinp

This patch adds the possibility to execute instance renaming when
doing a cluster burnin. The optional parameter --rename takes
one 'spare' instance name.
Given this option it renames all given instances to the spare one
and back to their original name.

Reviewed-by: iustinp

This patch adds iallocator support to burnin, currently only in instance
creation. This means that, depending on the external algorithm, not all
nodes might get instances.

The patch also restricts node selection and iallocator options together.

Reviewed-by: ultrotter

This patch does two things:
- adjusts burnin to work with the file storage backend patches
- adds the possibilty to do a burnin for file based instances

Reviewed-by: iustinp

Currently, the opcode DiagnoseOS is the only opcode that return a
structure of objects.OS (which is a custom class, and not a simple
python object) and furthermore all the processing of OS validity across
nodes is left to the clients of this opcode.

It would be more logical to have this opcode be similar to list
instances/nodes, in the sense that:
  - it should return a table of results
  - the fields in the table should be selectable

This patch does the above. The possible fields are:
  - name (os name)
  - valid (bool representing validity across all nodes)
  - node_status, which is a complicated structure required for ‘gnt-os
    diagnose’

With this patch, gnt-os list becomes a very simple iteration over the
list of results, filtering out non-valid ones. gnt-os diagnose is still
complicated, but no more than before.

The burnin tool has also been modified to work with the modified
results, and is simpler because of this (it only needs to know if an OS
is valid or not, not the per-node details).

Reviewed-by: imsnah

This patch removes the options that allow to create local_raid1 or
remote_raid1 instances. It also modifies the documentation and removes
these disk templates from burnin and from qa.

Reviewed-by: imsnah

This tiny patch fixes the verbose option to actually work, and also when
creating instances it logs the secondary node too (even if this doesn't
apply for plain templates, it doesn't create an error).

Reviewed-by: imsnah

Even if this doesn't have any practical use for actually creating
instances, it can be used for very fast burnin and testing just the
add/start/stop/remove functionality.

This has also revealed a bug in export/import related to diskless
instances, so it's educational value is proved.

Reviewed-by: ultrotter

Even though burnin was born just to do that test it now contains a lot more
things one might try, so it makes sense to make instance start/stop optional
too.

This creates a burnin that at the bare minimum tests instance create and
remove, if all the --no options are specified, but usually does a lot more.

Reviewed-by: iustinp

Instances get exported to a remote node, then removed and imported back to
their original nodes. This should be an idempotent option from the instance
point of view, and help making sure ImportExport is kept up to date.

It will also help making burnin take a lot longer, which is nice to take a nap.
"...but I'm doing a cluster burnin...". Unfortunately this subfeature is a bit
jeopardized by the fact that the new code can be skipped with the
--no-importexport option, but nobody needs to know that, do they?

Reviewed-by: iustinp

The burnin code deals with "-t plain", but the command line parser
doesn't allow that as an option. This patch fixes this issue.

Reviewed-by: ultrotter

This patch adds support for specifying and changing the boot device order for
HVM instances. The boot device order specification is ignored for non HVM
instances.

Reviewed-by: iustinp

Reviewed-by: schreiberal

Reviewed-by: iustinp

Reviewed-by: iustinp

Reviewed-by: iustinp

Currently, the burnin output is very verbose (we log the entire output
from LU feedback) and duplicate in some cases (as we enable debugging).

This patch disables debugging and only logs the burnin output when an
error occurs.

Reviewed-by: imsnah

This patch changes the burnin:
  - split the code into functions
  - add replace2 (replace the secondary) for remote_raid1
  - add replace1/2 for drbd8

Reviewed-by: imsnah

This patch adds a '-n' option to burnin that takes a comma-separated
list of nodes to perform the burnin on.

Reviewed-by: ultrotter

Allow burnin to use the new drbd8 template (for which case one needs to
disable replacement of disks, burnin does yet support that with drbd8).

The patch also changes do-replace[12] to no-replace[12] as that is what
they actually do.

Reviewed-by: imsnah

Reviewed-by: iustinp

This patch moves the passing of the feedback_fn argument from the
(Exec|Chain)OpCode to the initialization of the Processor instance.

Reviewed-by: imsnah

Since we added a new, required parameter to OpCreateInstance, we need to
update burnin to build the correct opcode.

Reviewed-by: schreiberal

Change this to have the exact same parameters as OpQueryInstances.

Also fix burnin which is broken since r146.

Reviewed-by: imsnah

This changes the raising of exceptions from:
  raise Exception, value
to
  raise Exception(value)

as the first form will be removed in python-3000 and the second form is
preferred now.

The changes also involve a few cases of changing from raising standard
exceptions and use our own ones.

The new version also fixes many pylint-generated warnings, especially in
ganeti-noded where I changed many methods to @staticmethod.

There is no functionality changed (barring any bugs).

Also improve the burnin by doing an optional replace-disks.