GitLab

Prevent starting a given instance when that instance has been shutdown
by the user, given that the instance must be first properly shutdown.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Update Xen backend to properly recognize when a domain has been
shutdown by the user and to properly cleanup a shutdown domain when
Ganeti requests Xen to stop this domain.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add instance state 'USER_down' which is a state used in reporting only
and it represents the situation in which the user has shutdown the
instance but Ganeti's configuration still has this instance marked as
'ADMIN_up'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Change Xen domain configuration to preserve the domain state after
shutdown, as opposed to automatic reboot, which is the default
behaviour.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add 'InstanceState' datatype which is the Haskell counterpart of the
Python type 'HvInstanceState'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Add 'HvInstanceState' which represents the state of an instance
(either 'running' or 'shutdown') and fix the remaining hypervisor
backends to return a value of this type.  Before this patch, each
backend was returning their own value, some were returning strings,
others integers.  However, the value that represents the state of an
instance was never actually used in Ganeti, until instance shutdown
detection became necessary.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Despite the fact that the '--network' parameter of
'gnt-network add' is mandatory, it is listed in the man
page in parenthesis, which suggests that it is optional.
Fixing this to avoid user confusion.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Disks and nics can not only be addressed by indices, but also
by name. Hence add a constructor for this case as well, to be
faithful to the python world.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

A previous patch [229da00f

] added an annotation for ignoring the warning,
but to the middle of a function, which doesn't compile. This patch moves
the annotation to the end of the function to correct the problem.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This warning appears only in newer versions of hlint (mine was v1.8.43),
and in this case it's reported incorrectly. The arguments to "showJSON"
have different types, therefore it's not possible to move "showJSON" in
front of "if".
Signed-off-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

With the change to luxid handling job queue requests, we
also inherited the parsing semantics of luxid, i.e., a
request is only considered syntactically correct, if all
parts are. Document that we decided to consider this change
of semantics desirable.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Historically, Ganeti would partially handle syntactically incorrect
luxi requests, if the outer structure was that of SubmitManyJobs
request. Moreover, this behavior was verified by a test. We changed to
completely reject syntactically incorrect luxi requests and decided
that this semantics was more desirable. Hence adapt the test
accordingly.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Now that mandatory opcode fields are mandatory also on disk,
document that, once we use soft drains during upgrades, the disk
representation of the job queue needs to be updated by cfgupgrade
as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

In order to guarantee all mandatory slots to be present, add
a custom constructor to BaseOpCode adding those fields with
default value, instead of inheriting the constructor from
outils.ValidatedSlots.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

While it's generally better to beg for forgiveness than ask for
permission in Python, it's not a good idea to rely on exceptions to
model control flow, especially if the resulting code is simpler.
Signed-off-by: Santi Raffa <rsanti@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

DTS_FILEBASED is a constant that exists and this commit makes sure
that it is used whenever sensible, rather than resorting to hardcoding
the pair of templates in very many files.
Signed-off-by: Santi Raffa <rsanti@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

As the responsibility for writing the job queue changed
to luxid, make masterd call to luxid when it is necessary
to write a job.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

As the base parameters also includes the reason trail,
set it, if not set already, together with the other
base parameters in the case of an opcode expanding to
a list of opcodes.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Handle this request by writing the jobs to the
queue and inform masterd; masterd will then also
distribute the jobs to all master candidates.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

As luxid is to take over responsibility for the job queue,
handle this request by writing the job to the queue and then
informing masterd; masterd will also distribute the job to
all master candidates.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Adding jobs to the queue is only allowed if the queue
is not drained.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Since luxid is going to write to the job queue,
it needs to honor drains of the queue as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

During the transition to the new daemon layout, from step 2
onwards, luxid will write to the queue but masterd will trigger
the execution. Therefore, add a new luxi request to tell masterd
to pick up a job that has already been written to the queue.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

The current restart procedure for masterd includes functionality
to pick up a job from the queue and restart it, if it hasn't been
started before. Move this functionality into a separate function
to be able to have the enqueuing be done by luxid.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Two avoid two processes simultaneously accessing the same
on-file structure, like the job queue, file locks are used.
Therefore, provide this functionality in Haskell as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

To avoid several processes accessing the queue at the same time,
Ganeti locks the queue via a lock file on disk. Provide the path
to this file.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This function writes a (non-archived) job to disk. The file
name can be computed from the job id, which is part of the job.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Add utility functions to allocate new job ids by increasing
the value stored in the serial file. As this function is
used in a multi-threaded program, synchronize access over
an internal lock.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

With luxi daemon taking over part of the job queue management, it will
also be responsible for replicating the queue to all master candidates.
Therefore, add a function to extract the list of master candidates from
the configuration.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

To be able to replicate the job queue, in particular the serial,
luxid needs to be able to send the jobqueue_update RPC. So add
its definition.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This version of 'the' properly lives in the 'Result' monad,
as opposed to traditional one calling 'error'. The reason
why it is 'Bad' that not precisely one element is returned
is given as argument.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This methods allows reading the maximal job serial number from
disk.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This function handles the pure part of generating a job,
i.e., assuming the job id already assigned and not setting
time stamps.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

When queueing many jobs, the dependencies between them need to
be resolved with the knowledge of their respective job id.
Lift the computation of the absolute dependency to the level
of MetaOpCodes.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

SubmitManyJobs also accepts jobs with dependencies given as
relative ids. Together with the absolute id of the job, once
assigned, the dependency can be resolved. Add a function doing
this computation.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

When generating jobs from sequences of op-codes, it is necessary
to wrap op-codes into queued form.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

This function allows to use 'IO a' objects in a safe
way, using the 'try' function; the outcome is reported
as a 'Result'. IOErrors are logged and the result is
'Bad', while in the case of no errors, a result-yielding
function is applied to the value.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

To keep our on-file data consistent at any moment, we change
file contents by atomically replacing the file with a new one.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

See issue 235. It turned out the feature is a bit more complex than expected,
hence this design doc.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>