GitLab

This includes NEWS and INSTALL
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. for both Debian distribution. Also check that the library is present
during the configuration step.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Rename some functions related to instance communication not to collide
with the naming convention used in the opcodes.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Factor out the opcodes 'OpNetworkAdd' and 'OpNetworkConnect' used in
'LUClusterSetParams' and 'LUGroupAdd' in order to reduce code
duplication and keep the configuration of the instance communication
in one place.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Extend QA with tests for the instance communication mechanism.  These
tests cover only the cluster creating the instance communication
network and connecting it to the existing node group, and also if new
groups are added they are also connected to the instance communication
network.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

When a new group is added, if the instance communication network is
enabled, then this group must also be connected to this network.

* 'LUGroupAdd._AddInstanceCommunicationNetwork' connects the newly
  added group to the instance communication network by resorting to
  the 'OpNetworkConnect' opcode.

* As a result of the previous change, the type of 'OpGroupAdd' changed
  and may optionally return a list of jobs.  Therefore, it is
  necessary to update the type of the opcode (in 'Ganeti.OpCodes') and
  the CLI for 'gnt-group add' must also be changed.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Extend CLI 'gnt-cluster modify' with
'--instance-communication-network'.  Given that the return type for
'OpClusterSetParams' changed to optionally return a list of jobs, it
is also necessary to handle the result of this opcode accordingly.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Extend 'LUClusterSetParams' to create the user-supplied instance
communication network in case this network does not exist.  Note that
if the user-supplied network already exists, nothing needs to be done
as 'CheckPrereq' already checks this network's configuration.

With this patch, Ganeti will be able to create the instance
communication network automatically.  The following example shows the
command line options that instruct Ganeti to enable the instance
communication network and, if this network does not exist, then Ganeti
creates it.

  gnt-cluster modify --instance-communication-network=mynetwork

The instance communication network can be disabled by setting the
parameter to the empty string, for example:

  gnt-cluster modify --instance-communication-network=

This will not destroy the network itself or modify any running
instances.  This will have to be done manually.

In this patch:

* 'LUClusterSetParams._EnsureInstanceCommunicationNetwork' creates and
  connects to all groups the instance communication network by
  resorting the 'OpNetworkAdd' and 'OpNetworkConnect'.

* 'LUClusterSetParams.ModifyInstanceCommunicationNetwork' compares the
  previous configured 'instance_communication_network' with the
  user-supplied one and determines whether the opcodes is changing the
  configuration, which might require creating the instance
  communication network.

* Extend 'LUClusterSetParams.Exec' to call
  'ModifyInstanceCommunicationNetwork' and handle the user-supplied
  'instance_communication_network'.

* Because 'ClusterSetParams' now may return a list of jobs, the type
  of the opcode must be updated in 'Ganeti.OpCodes'.

* Add network related constants to 'Ganeti.Constants'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Later, the logical unit for 'OpClusterSetParams' will be responsible
for creating the instance communication network in case it does not
exist.  For now, it is important to check whether the network the user
is requesting to become assigned to instance communication has the
right configuration and otherwise warn about potential security risks.

* Add a new static method
  'LUClusterSetParams._CheckInstanceCommunicationNetwork' which checks
  if the user-supplied network is configured correctly and issues
  security warnings otherwise.  A correctly configured instance
  communication network is link-local (i.e., 169.254.0.0/16 for IPv4
  and fe80::/64 for IPv6), has no gateways, and has a specific MAC
  prefix.

* Extend 'LUClusterSetParams.CheckPrereq' to call the above method to
  check whether the user supplied network matches the intended
  configuration.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

... and update the unit tests.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

This helper function detects whether an opcode returned a list of jobs
(i.e., a result of the type ht.TJobIdListOnly) and in this case it
uses 'ganeti.cli.JobExecutor' to wait for the jobs and determine the
return code which is based on the return codes of all jobs.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

* Add 'ganeti.objects.Cluster.instance_communication_network' to the
  config so the logical unit for 'OpInstanceCreate' can access the
  instance communication network which is stored in the cluster
  configuration.

* Change logical unit for 'OpInstanceCreate' to use the instance
  communication network parameter defined in the cluster configuration
  instead of using a predefined one.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

* Add parameter 'instance_communication_parameter' to the Python
  'ganeti.objects.Cluster' and the Haskell 'Ganeti.Objects.Cluster'.

* Update Haskell 'QueryClusterInfo' to return also the
  'instance_communication_network' parameter.

* Update Python 'LUClusterQuery' to return also the
  'instance_communication_network' parameter.

* Update Python 'ShowClusterConfig' to include information about the
  'instance_commuication_network' parameter

* Update 'ganeti.objects.Cluster.UpgradeConfig' to ugprade also
  'instance_communication_network' parameter to the empty string, if
  unspecified.

* Update the configuration upgrade tool (i.e., 'tools/cfgupgrade') to
  handle upgrading of the 'instance_communication_network' parameter
  as well as downgrading.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Since python interprets an empty dictionary as 'False',
testing for the existance of the 'instances' field in
the configuration has to be done by explicitely testing
for 'None'.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

The instance parameter "osparams_private" was newly introduced
in 2.12. So remove it when downgrading the configuration to 2.11.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

The new cluster parameter "osparams_private_cluster" was
newly introduced in 2.12. So remove it when downgrading
to 2.11.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

* stable-2.11
  Update the list dependencies in INSTALL
  Make luxid's Haskell dependencies mandatory
  Fix start/stop scripts to run luxid unconditionally
  Remove the HTOOLS configuration variable
  Add a config. conditional for running Haskell coverage
  Make htools and luxid mandatory in the Makefile
  Temporarily remove SSL changes from NEWS file
  Fix bracket style in rapi-workload
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

In particular, move luxid's dependencies to the section with mandatory
dependencies.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

in the build configuration files.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

and update the unit check the list of daemons.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

.. and update the code that uses it.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Similarly to PY_UNIT, add HS_UNIT to run Haskell coverage only if we
have the required modules.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Since Ganeti now uses luxid for core operations, it needs to be always
present.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Since the current changes in the SSL handling have
been partially disabled, remove their announcement
in the NEWS file until there is something actually
user-facing happening.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Pep8 complained about bracket indentation. This fixes it to satisfy Pep8
as well as our style guide.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
(cherry-picked from ccded668

)
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Jose Lopes <jabolopes@google.com>

* stable-2.11
  Gracefully handle queries for non-existing nodes
  Consider job-IDs queried for twice only once

Conflicts:
	lib/client/gnt_node.py: trivial
	src/Ganeti/Query/Query.hs: import ALL the functions
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

When adding a node, Ganeti checks whether the node is already
part of the cluster by querying for the node name. However,
as queries are meant to return all nodes with the given name,
it might well return the empty list when a new node is to be
added. Handle that case correctly.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

As reading jobs from disk is an expensive operation, when querying
for jobs, we optimize by considering which values the job-id is asked
for in the filter. As any reasonable person would not add the same
clause twice in an Or-clause, the implicit assumption was that the
job ids obtained by inspecting the filter would be unique; hence the
information for those jobs could be returned directly.

That all did hold true and everyone was happy till with commit 13ef1fa5


a query for a single job suddenly returned the job twice violating
WaitForJobChange's expectation. Fix this by taking the nub of the
jids asked for in the filter.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

* stable-2.11
  (no changes)

* stable-2.10
  Fix 'JobIdListOnly' type from 'List' to 'Map'
  Remove NEWS entry for 2.9.4
  Workaround for monitor bug related to greeting msg
  hotplug: Verify if a command succeeded or not
  hotplug: Call each qemu commmand with an own socat
  upgrade: start daemons after ensure-dirs
  upgrade design: ensure-dirs before starting daemons
  Fix network management section in admin.rst
  Adapt release date for 2.10.0-rc2
  Revision bump for 2.10.0-rc2
  Update NEWS file in preparation of 2.10.0rc2
  Add Network Management section in admin.rst

* stable-2.9
  Revision bump for 2.9.4
  Set release date for 2.9.4
  Note UUID identification change in NEWS file
  Allow classic queries to use either names or UUIDs
  Document the change of noded's group in NEWS
  Make the LUInstanceCreate return node names, not UUIDs
  Document new handling of degraded instances in NEWS
  Gracefully handle degraded instances in verification
  Be aware of the degraded case when cleaning up an instance
  Document changes to file-based disks in NEW
  Preserve disk basename on instance rename
  Update NEWS file
  Modify test to reflect RAPI operation changes
  Add QA tests for RAPI multi-instance allocatio
  Fix multi-allocation RAPI method
  Assign unique filenames to filebased disks

* stable-2.8
  Fix execution group of NodeD
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

* stable-2.10
  Fix 'JobIdListOnly' type from 'List' to 'Map'
  Remove NEWS entry for 2.9.4
  Workaround for monitor bug related to greeting msg
  hotplug: Verify if a command succeeded or not
  hotplug: Call each qemu commmand with an own socat
  upgrade: start daemons after ensure-dirs
  upgrade design: ensure-dirs before starting daemons
  Fix network management section in admin.rst
  Adapt release date for 2.10.0-rc2
  Revision bump for 2.10.0-rc2
  Update NEWS file in preparation of 2.10.0rc2
  Add Network Management section in admin.rst

* stable-2.9
  Revision bump for 2.9.4
  Set release date for 2.9.4
  Note UUID identification change in NEWS file
  Allow classic queries to use either names or UUIDs
  Document the change of noded's group in NEWS
  Make the LUInstanceCreate return node names, not UUIDs
  Document new handling of degraded instances in NEWS
  Gracefully handle degraded instances in verification
  Be aware of the degraded case when cleaning up an instance
  Document changes to file-based disks in NEW
  Preserve disk basename on instance rename
  Update NEWS file
  Modify test to reflect RAPI operation changes
  Add QA tests for RAPI multi-instance allocatio
  Fix multi-allocation RAPI method
  Assign unique filenames to filebased disks

* stable-2.8
  Fix execution group of NodeD

Conflicts:
	NEWS: take both additions
	configure.ac: ignore suffix bump
	lib/cmdlib/instance_utils.py: manually apply
	    845b7ed1

 to stable-2.10 hunk
	src/Ganeti/Query/Query.hs: take all exports
	test/py/cmdlib/instance_unittest.py: take assertions
	    from both branches for the respective storage
	    types
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

As masterd is going away, set default for all clients to
luxid's socket.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

* stable-2.9
  Revision bump for 2.9.4
  Set release date for 2.9.4
  Note UUID identification change in NEWS file
  Allow classic queries to use either names or UUIDs
  Document the change of noded's group in NEWS

Conflicts:
	NEWS: take all stable-2.9 additions, effectively
	    restoring the 2.9.4 entry, hence effectively
	    reverting 78cb4606
	configure.ac: ignore revision bump
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

... but do not try to check the copyrights for local branches which do
not have a proper remote branch, which would fail at a later step
anyway.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

As all RAPI requests now go to luxid, and masterd is going away,
remove option from RAPI client to chose a different socket.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

As all luxi clients talk to luxid now, and masterd will
go away, remove the option to use socket different from
luxid's.
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>