GitLab

Since the current changes in the SSL handling have
been partially disabled, remove their announcement
in the NEWS file until there is something actually
user-facing happening.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Pep8 complained about bracket indentation. This fixes it to satisfy Pep8
as well as our style guide.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
(cherry-picked from ccded668)
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

When adding a node, Ganeti checks whether the node is already
part of the cluster by querying for the node name. However,
as queries are meant to return all nodes with the given name,
it might well return the empty list when a new node is to be
added. Handle that case correctly.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

As reading jobs from disk is an expensive operation, when querying
for jobs, we optimize by considering which values the job-id is asked
for in the filter. As any reasonable person would not add the same
clause twice in an Or-clause, the implicit assumption was that the
job ids obtained by inspecting the filter would be unique; hence the
information for those jobs could be returned directly.

That all did hold true and everyone was happy till with commit 13ef1fa5
a query for a single job suddenly returned the job twice violating
WaitForJobChange's expectation. Fix this by taking the nub of the
jids asked for in the filter.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

* stable-2.10
  Fix 'JobIdListOnly' type from 'List' to 'Map'
  Remove NEWS entry for 2.9.4
  Workaround for monitor bug related to greeting msg
  hotplug: Verify if a command succeeded or not
  hotplug: Call each qemu commmand with an own socat
  upgrade: start daemons after ensure-dirs
  upgrade design: ensure-dirs before starting daemons
  Fix network management section in admin.rst
  Adapt release date for 2.10.0-rc2
  Revision bump for 2.10.0-rc2
  Update NEWS file in preparation of 2.10.0rc2
  Add Network Management section in admin.rst

* stable-2.9
  Revision bump for 2.9.4
  Set release date for 2.9.4
  Note UUID identification change in NEWS file
  Allow classic queries to use either names or UUIDs
  Document the change of noded's group in NEWS
  Make the LUInstanceCreate return node names, not UUIDs
  Document new handling of degraded instances in NEWS
  Gracefully handle degraded instances in verification
  Be aware of the degraded case when cleaning up an instance
  Document changes to file-based disks in NEW
  Preserve disk basename on instance rename
  Update NEWS file
  Modify test to reflect RAPI operation changes
  Add QA tests for RAPI multi-instance allocatio
  Fix multi-allocation RAPI method
  Assign unique filenames to filebased disks

* stable-2.8
  Fix execution group of NodeD

Conflicts:
	NEWS: take both additions
	configure.ac: ignore suffix bump
	lib/cmdlib/instance_utils.py: manually apply
	    845b7ed1 to stable-2.10 hunk
	src/Ganeti/Query/Query.hs: take all exports
	test/py/cmdlib/instance_unittest.py: take assertions
	    from both branches for the respective storage
	    types
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

* stable-2.9
  Revision bump for 2.9.4
  Set release date for 2.9.4
  Note UUID identification change in NEWS file
  Allow classic queries to use either names or UUIDs
  Document the change of noded's group in NEWS

Conflicts:
	NEWS: take all stable-2.9 additions, effectively
	    restoring the 2.9.4 entry, hence effectively
	    reverting 78cb4606
	configure.ac: ignore revision bump
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

* This patch fixes a type discrepancy between the Haskell type
  'Ganeti.OpCodes.JobIdListOnly' and the Python type predicate
  'ganeti.ht.TJobIdListOnly', the former being a list and the latter a
  dictionary.

* Remove some whitespace
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

When UUIDs are used in CLI commands, such addressing of objects fails
or succeeds inconsistently across object types. Worse yet, some calls
do not fail, but simply return no result. This is due to the way the
classical queries are handled in the Haskell code - a filter is built,
checking if the names match. If the names do not match, but the object
can still be retrieved through UUID, the query is successful, but there
is nothing to return.

This patch remedies the situation by extending the filter into an or
clause checking for either name or uuid. For objects that have no uuid
field, the name is reused, making the filter equivalent to the old one.

Works towards resolving issue 703.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Remove the NEWS entry for 2.9.4 temporarily until it actually gets
released. Otherwise check-news would complain about the unreleased
version below released ones.

The NEWS entry will be added back once 2.9.4 is released and the
corresponding commits are merged up to 2.10 again.
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

* stable-2.9
  Make the LUInstanceCreate return node names, not UUIDs
  Document new handling of degraded instances in NEWS
  Gracefully handle degraded instances in verification
  Be aware of the degraded case when cleaning up an instance
  Document changes to file-based disks in NEWS
  Preserve disk basename on instance rename
  Update NEWS file
  Modify test to reflect RAPI operation changes
  Add QA tests for RAPI multi-instance allocation
  Fix multi-allocation RAPI method
  Assign unique filenames to filebased disks

* stable-2.8
  Fix execution group of NodeD

Conflicts:
	NEWS (took both changes)
	lib/config.py (trivial)
	test/py/ganeti.cmdlib_unittest.py
          (manually reapplied changes to
           test/py/cmdlib/instance_unittest.py)
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

For jobs still queued, we ask the queue to change the priority,
and replicate the changed job. For jobs that have already been
started, we have to contact the job directly, which, at the moment,
means forwarding the request to masterd.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

There is a separation of responsibilities here. For jobs still
in the queue, it is the responsibility of the queue (scheduler),
for started jobs, the job itself has to take care of it. To avoid
the job transitioning inbetween, it is temporarily dequeued during
the operation. The operation changes the file on master, while
leaving the replication to the caller.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

When adding new jobs, don't add them at the end, but at a
position that fits with their job id. In this way, we can
build operations that require fully dequeing a job an adding
it later after some modifications.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

...to also provide the job itself. In this way, the function can
also be used for tasks that require temporarily removing a job
from the queue.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

...by changing the priority of the non-finished opcodes.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This pure function follows the semantic that an opcode, including
its priority, may only be changed if the opcode is not finalized.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

There is often need to manipulate these errors, for example to convert a String
from Result into an exception. These functions make this easier.

Function 'toErrorStr' lifts 'Result' to any 'MonadError'. This is useful
for converting 'Result' into 'ResultT' or any other similar monad stack.

Functions 'catchErrorT' and 'handleErrorT' catch errors just as 'catchError'
does, but also allow to change the error type.

Functions `withError` and `withErrorT` allow to modify an error within
GenericResult or ResultT. This is convenient when combining functions
with different error types, for example to convert between strings and
exceptions.

Furthemore, 'failError' generalize 'Bad'. It can be used anywhere 'Bad' is, and
in any 'MonadError' instance.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Cherry-pick of 565821d1Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

They have the very same functionality, and using our own FromString only
causes unnecessary code duplication.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Cherry-pick of a87a017bSigned-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This allows to use Alternative specific combinators, namely `optional`.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Cherry-pick of 78209a84Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

QMP may return multiple greeting messages upon connection.
This is reported on qemu-devel. The fix is one-liner but
until it get's released this is a quick and dirty workaround
that flushes the client's buffer after getting the first
greeting message.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Just after issuing _CallHoplugCommands() we invoke
_VerifyHotplugCommand() which parses `info pci` result
and searches for given PCI slot and device id.

If we previously had removed a device but it is still there
abort. Do the same if we had add a device but it is not found.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

Previously we issued one socat command with two "\n" separated
actions (e.g. netdev_add ...\ndevice_add...)

After having observed a strange monitor behavior [1] splitting
those commands and introducing a sleep time in between, may reduce
any race possibilities. Additionally it will be a step forward
parsing monitor output of each command and decide whether to
modify runtime files or raise a HotplugError.

This patch simply changes _CallHotplugCommand() to take a
list of commands and issue them separately. Introduces one second
sleep after each command and removes sleep from HotModDevice().

[1] https://groups.google.com/d/msg/ganeti-devel/hCQnmuqKtU0/UWGnw251cskJSigned-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

* stable-2.8
  Fix execution group of NodeD
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose A. Lopes <jabolopes@google.com>

The Node deamon was executed with the wrong gid (gnt-daemons) instead of the one
assigned to it by configure.ac.

Fixes Issue 707.
Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

In this way, we cann pass through the opaque parameters
required for disk creation and modification in the case of
external storage.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

The field catching the remaining fields will always be of the same
shape, so add a function for this to make usage simple.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

So that objects using AndRestArguments are available for testing.
As the AndRestArguments are intended for passing through additional
parameters passed on the command line, we restrict them to the
values that are allowed at this position: Strings of ASCII characters
and Integers.

Also, another property guaranteed by the specification is that the
keys in the map are different from all the other fields. To obtain
this property without adding additional complexity to the test
infrastructure, we choose a length for the keys that is longer than
all the valid fields.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

A field of this type will capture all the remaining fields
of an object as JSValues. Obviously, the intended use is
to have precisely one such field. This mechanism will allow
to pass opaque values trough, as it is, e.g., required for
the disk parameters for external storage.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Commit 580b1fdd incorrectly assumes that disk parameters are
just the standard ones, whereas the man page explicitly states
that additional parameters can be passed as well, if they make
sense for the chosen storage type. Fix this.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

In commit 702c3270 two new parameters were added to the
Create function of BlockDev. Make subclasses also adhere
this specification.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

The LUInstanceCreate returned names instead of UUIDs in 2.6. Along the
way, the names were internally replaced with UUIDs, and the abstraction
leaked. This patch fixes the issue.
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

In commit 702c3270 two new parameters were added to the
constructor of BlockDev. Make the subclassess accept these
additional parameters as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Until now Disk name and uuid was not available on bdev level.
In case of ExtStorage, this info is useful, and may be for other
templates in the future too.

This patch treats the name and uuid object slots just like the size
one and passes them to BlockDev.__init__() and to BlockDev.Create().

The ExtStrorage interface exports these options to scripts via the
VOL_CNAME and VOL_UUID environment variables.

This fixes Issue 696.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

On upgrading a cluster, we only can rely on daemons starting
up cleanly, if all needed directories are generated first. So
ensure-dirs needs to be run first.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Align the automatic upgrade procedure with the manual upgrade
instructions and run ensure-dirs before starting the new daemons.
This is necessary, as only after ensuring that all needed directories
are present we can rely on the new daemons starting up cleanly.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>