GitLab

* stable-2.11
  Update design doc wrt to improved SSL design
  Test node certificate renewal in QA
  Use node UUID as client certificate serial number
  Revert "Temporarily remove SSL changes from NEWS file"
  Revert "Disabling client certificate usage"
  Fix watcher tampering with instance userdown QA

Conflicts:
	qa/qa_instance.py (trivial)
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

This patch updates the design document of Ganeti's node
security to make it consistent with the implementation
and to extend it with a couple of suggestions to improve
the SSL security even more.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This extends the QA by explicitely testing the renewal
of SSL client certificates.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

It turns out, that some implementations of OpenSSL are more
pedantic in checking the certficates than others. In this
particular case, the SSL connection could not be
established when the serial number of the certificates
was not unique.

To avoid this problem, this patch extends Ganeti's X509
infrastructure to set the certificate's serial
number. In case of client certificates, we now use the
node's UUID as serial number, because the UUIDs are
assumed to be unique in a cluster. This is however still
not complying to how SSL was designed to be used, but at
least it is a lot better than setting every serial number
to 1, which was used before and is still used for other
certificates than the client certificate.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This reverts commit 89671df1, which temporarily removed
the announcement of improved SSL security in the NEWS file.
As this patch series fixes the SSL problem that caused
the disabling of the features, we are adding back the
announcement.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This reverts commit 45f75526, which was introduced to
temporarily disable the implementation of SSL client
certificates. As this patch series fixes the reason for
the disabling, we are rolling back the patch.
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Function 'atomicUpdateFile' extends 'atomicWriteFile' that allows any
action to be run on a temporary file.

Function 'atomicUpdateLockedFile' additionally locks the original file
using flock and checks if its state conforms to the last one.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

It's used on the Python side, it's been missing on the Haskell side.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This makes usage of ReaderT's liftIO more convenient.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

All stubs are generated from Haskell during compilation.
Also ignore the executable of the new daemon.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This ambiguity was introduced by adding the WConfd client.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The client combines the abstract client class and the WConfD
stub to provide a Python interface to WConfD.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The generation is included in hs2py.hs, together with other
Haskell to Python code generators.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The client provides _GenericInvoke(...) for a stub and uses its
_GetSocketPath() for opening a Transport.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The names of the arguments of generated Python methods are derived from
Haskell types to be as descriptive as possible.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This involves adding a new sets of variables for the directory.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Directory "lib/rpc/stubs" will contains RPC stubs generated from
Haskell.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. instead of AbstractClient itself. Also let every client call
_InitTransport() as needed. This allows to determine socket addresses
later than during the initialization of a class.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The daemon exposes the declared functions in Ganeti.WConfd.Core to RPC
clients (currently just 'echo').
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Also list it in the Haskell datatype, constants, Python constants and
test configuration.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

There are no exported functions yet, except for `echo`, which just
returns its argument back to a client.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This allows to log in monad stacks containing RWS.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This monad encapsulates working with the daemon and client state, as
well as failures and IO operations.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The main function in this module takes a list of names of functions and
uses TH to create an RPC server, represented as a value of type Handler
(from Ganeti.UDSServer). This can be readily used to create a daemon
that dispatches RPC requests to the given functions.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

In particular, test (de)serialization of "OneTuple".
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

It tests if a deserialization method fails properly when an invalid
input is given.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

In particular, functions for extracting types of arguments from a
function type, and for uncurrying functions with an arbitrary number of
arguments.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This error appears after adding the dependency to lifted-base.
See also #683 and ee7caf27.

I surmise that something like this is happening: Some code, that uses
the library, is only referenced through Template Haskell. A probable bug
in GHC linker fails to notice it and doesn't include the library when
linking, causing the error. Referencing the library explicitly in THH.hs
solves the problem.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This allows to use lifted operations like 'fork' or 'bracket' inside
ResultT.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. as long as they're instances of "MonadBaseControl IO" and "MonadLog".
This allows the UDSServer to call functions like "fork" within monads
such as "ResultT e IO" or "ReaderT IO".
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

This includes NEWS and INSTALL
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

.. for both Debian distribution. Also check that the library is present
during the configuration step.
Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

The watcher was tampering with the instance user down QA because, when
the watcher sees an instance marked an 'USER_down', it will cleanup
the instance, that is it will stop the instance, thus marking it as
'ADMIN_down'.  The change in instance status was causing the instance
user down QA to fail sporadically.  This patch pauses the watcher
during this test.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Reviewed-by: Helga Velroyen <helgav@google.com>

Rename some functions related to instance communication not to collide
with the naming convention used in the opcodes.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Factor out the opcodes 'OpNetworkAdd' and 'OpNetworkConnect' used in
'LUClusterSetParams' and 'LUGroupAdd' in order to reduce code
duplication and keep the configuration of the instance communication
in one place.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Extend QA with tests for the instance communication mechanism.  These
tests cover only the cluster creating the instance communication
network and connecting it to the existing node group, and also if new
groups are added they are also connected to the instance communication
network.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

When a new group is added, if the instance communication network is
enabled, then this group must also be connected to this network.

* 'LUGroupAdd._AddInstanceCommunicationNetwork' connects the newly
  added group to the instance communication network by resorting to
  the 'OpNetworkConnect' opcode.

* As a result of the previous change, the type of 'OpGroupAdd' changed
  and may optionally return a list of jobs.  Therefore, it is
  necessary to update the type of the opcode (in 'Ganeti.OpCodes') and
  the CLI for 'gnt-group add' must also be changed.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Extend CLI 'gnt-cluster modify' with
'--instance-communication-network'.  Given that the return type for
'OpClusterSetParams' changed to optionally return a list of jobs, it
is also necessary to handle the result of this opcode accordingly.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Extend 'LUClusterSetParams' to create the user-supplied instance
communication network in case this network does not exist.  Note that
if the user-supplied network already exists, nothing needs to be done
as 'CheckPrereq' already checks this network's configuration.

With this patch, Ganeti will be able to create the instance
communication network automatically.  The following example shows the
command line options that instruct Ganeti to enable the instance
communication network and, if this network does not exist, then Ganeti
creates it.

  gnt-cluster modify --instance-communication-network=mynetwork

The instance communication network can be disabled by setting the
parameter to the empty string, for example:

  gnt-cluster modify --instance-communication-network=

This will not destroy the network itself or modify any running
instances.  This will have to be done manually.

In this patch:

* 'LUClusterSetParams._EnsureInstanceCommunicationNetwork' creates and
  connects to all groups the instance communication network by
  resorting the 'OpNetworkAdd' and 'OpNetworkConnect'.

* 'LUClusterSetParams.ModifyInstanceCommunicationNetwork' compares the
  previous configured 'instance_communication_network' with the
  user-supplied one and determines whether the opcodes is changing the
  configuration, which might require creating the instance
  communication network.

* Extend 'LUClusterSetParams.Exec' to call
  'ModifyInstanceCommunicationNetwork' and handle the user-supplied
  'instance_communication_network'.

* Because 'ClusterSetParams' now may return a list of jobs, the type
  of the opcode must be updated in 'Ganeti.OpCodes'.

* Add network related constants to 'Ganeti.Constants'.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Later, the logical unit for 'OpClusterSetParams' will be responsible
for creating the instance communication network in case it does not
exist.  For now, it is important to check whether the network the user
is requesting to become assigned to instance communication has the
right configuration and otherwise warn about potential security risks.

* Add a new static method
  'LUClusterSetParams._CheckInstanceCommunicationNetwork' which checks
  if the user-supplied network is configured correctly and issues
  security warnings otherwise.  A correctly configured instance
  communication network is link-local (i.e., 169.254.0.0/16 for IPv4
  and fe80::/64 for IPv6), has no gateways, and has a specific MAC
  prefix.

* Extend 'LUClusterSetParams.CheckPrereq' to call the above method to
  check whether the user supplied network matches the intended
  configuration.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>