security.rst 9.94 KB
Newer Older
1
Security in Ganeti
Iustin Pop's avatar
Iustin Pop committed
2
==================
3

Guido Trotter's avatar
Guido Trotter committed
4
Documents Ganeti version 2.7
5

6
7
8
Ganeti was developed to run on internal, trusted systems. As such, the
security model is all-or-nothing.

9
Up to version 2.3 all Ganeti code ran as root. Since version 2.4 it is
10
11
12
13
14
15
16
possible to run all daemons except the node daemon and the monitoring daemon
as non-root users by specifying user names and groups at build time.
The node daemon continues to require root privileges to create logical volumes,
DRBD devices, start instances, etc. Cluster commands can be run as root or by
users in a group specified at build time. The monitoring daemon requires root
privileges in order to be able to access and present information that are only
avilable to root (such as the output of the ``xm`` command of Xen).
17
18
19
20

Host issues
-----------

21
22
For a host on which the Ganeti software has been installed, but not
joined to a cluster, there are no changes to the system.
23
24
25

For a host that has been joined to the cluster, there are very important
changes:
Iustin Pop's avatar
Iustin Pop committed
26
27
28
29

- The host will have its SSH host key replaced with the one of the
  cluster (which is the one the initial node had at the cluster
  creation)
30
- A new public key will be added to root's ``authorized_keys`` file,
Iustin Pop's avatar
Iustin Pop committed
31
32
33
34
35
36
37
38
39
  granting root access to all nodes of the cluster. The private part of
  the key is also distributed to all nodes. Old files are renamed.
- Communication between nodes is encrypted using SSL/TLS. A common key
  and certificate combo is shared between all nodes of the cluster.  At
  this time, no CA is used.
- The Ganeti node daemon will accept RPC requests from any host within
  the cluster with the correct certificate, and the operations it will
  do as a result of these requests are:

40
  - running commands under the ``/etc/ganeti/hooks`` directory
Iustin Pop's avatar
Iustin Pop committed
41
42
  - creating DRBD disks between it and the IP it has been told
  - overwrite a defined list of files on the host
43
44
45
46
47

As you can see, as soon as a node is joined, it becomes equal to all
other nodes in the cluster, and the security of the cluster is
determined by the weakest node.

48
49
Note that only the SSH key will allow other machines to run any command
on this node; the RPC method will run only:
Iustin Pop's avatar
Iustin Pop committed
50
51
52

- well defined commands to create, remove, activate logical volumes,
  drbd devices, start/stop instances, etc;
53
54
- run well-defined SSH commands on other nodes in the cluster
- scripts under the ``/etc/ganeti/hooks`` directory
55
56
- scripts under the ``/etc/ganeti/restricted-commands`` directory, if
  this feature has been enabled at build time (see below)
57
58

It is therefore important to make sure that the contents of the
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
``/etc/ganeti/hooks`` and ``/etc/ganeti/restricted-commands``
directories are supervised and only trusted sources can populate them.

Restricted commands
~~~~~~~~~~~~~~~~~~~

The restricted commands feature is new in Ganeti 2.7. It enables the
administrator to run any commands in the
``/etc/ganeti/restricted-commands`` directory, if the feature has been
enabled at build time, subject to the following restrictions:

- No parameters may be passed
- No absolute or relative path may be passed, only a filename
- The ``/etc/ganeti/restricted-commands`` directory must
  be owned by root:root and have mode 0755 or stricter
- Executables must be regular files or symlinks, and must be executable
  by root:root

Note that it's not possible to list the contents of the directory, and
there is an intentional delay when trying to execute a non-existing
command (to slow-down dictionary attacks).

Since for Ganeti itself this functionality is not needed, and is only
provided as a way to help administrate or recover nodes, it is a local
site decision whether to enable or not the restricted commands feature.

By default, this feature is disabled.

87
88
89
90

Cluster issues
--------------

91
92
As mentioned above, there are multiple ways of communication between
cluster nodes:
Iustin Pop's avatar
Iustin Pop committed
93
94
95
96
97

- SSH-based, for high-volume traffic like image dumps or for low-level
  command, e.g. restarting the Ganeti node daemon
- RPC communication between master and nodes
- DRBD real-time disk replication traffic
98

99
100
The SSH traffic is protected (after the initial login to a new node) by
the cluster-wide shared SSH key.
101

102
103
104
105
106
RPC communication between the master and nodes is protected using
SSL/TLS encryption. Both the client and the server must have the
cluster-wide shared SSL/TLS certificate and verify it when establishing
the connection by comparing fingerprints. We decided not to use a CA to
simplify the key handling.
107

Iustin Pop's avatar
Iustin Pop committed
108
109
The DRBD traffic is not protected by encryption, as DRBD does not
support this. It's therefore recommended to implement host-level
110
firewalling or to use a separate range of IP addresses for the DRBD
111
112
113
114
115
116
traffic (this is supported in Ganeti through the use of a secondary
interface) which is not routed outside the cluster. DRBD connections are
protected from erroneous connections to other machines (as may happen
due to software issues), and from accepting connections from other
machines, by using a shared secret, exchanged via RPC requests from the
master to the nodes when configuring the device.
Iustin Pop's avatar
Iustin Pop committed
117
118
119
120

Master daemon
-------------

121
122
The command-line tools to master daemon communication is done via a
UNIX socket, whose permissions are reset to ``0660`` after listening but
123
124
125
before serving requests. This permission-based protection is documented
and works on Linux, but is not-portable; however, Ganeti doesn't work on
non-Linux system at the moment.
126

127
128
129
Conf daemon
-----------

130
In Ganeti 2.8, the ``confd`` daemon (if enabled at build time), serves
131
132
133
134
135
136
137
138
139
140
both network-originated queries (about the static configuration) and
local (UNIX socket) queries (about the run-time configuration; answering
these means talking to other cluster nodes, which makes use of the
internal RPC SSL certificate). This makes it a bit more sensitive to
bugs (a remote attacker could get direct access to the intra-cluster
RPC), so to harden security it's recommended to:

- disable confd at build time if it's not needed in your setup
- otherwise, configure Ganeti (at build time) to use separate users, so
  that the confd daemon doesn't also have access to the server SSL/TLS
141
  certificates.
142

143
144
145
146
147
NB: the second suggestion is not valid since Ganeti 2.8.0~beta1, because confd
needs access to the certificate in order to communicate on the network.
This will be fixed when the planned split of the two functionalities
(local/remote querying) of confd into two separate daemons will take place,
in a future Ganeti version.
148

149
150
151
152
153
154
155
156
157
158
159
160
161
162
Monitoring daemon
-----------------

The monitoring daemon provides information about the status and the
performance of the cluster over HTTP.
It is currently unencrypted and non-authenticated, therefore it is strongly
advised to set proper firewalling rules to prevent unwanted access.

The monitoring daemon runs as root, because it needs to be able to access
privileged information (such as the state of the instances as provided by
the Xen hypervisor). Nevertheless, the security implications are mitigated
by the fact that the agent only provides reporting functionalities,
without the ability to actually modify the state of the cluster.

163
164
165
Remote API
----------

166
Starting with Ganeti 2.0, Remote API traffic is encrypted using SSL/TLS
167
168
169
by default. It supports Basic authentication as per :rfc:`2617`. Users
can be granted different capabilities. Details can be found in the
:ref:`RAPI documentation <rapi-users>`.
170

171
172
173
Paths for certificate, private key and CA files required for SSL/TLS
will be set at source configure time. Symlinks or command line
parameters may be used to use different files.
174

175
176
177
178
179
180
181
182
183
184
Inter-cluster instance moves
----------------------------

To move instances between clusters, different clusters must be able to
communicate with each other over a secure channel. Up to and including
Ganeti 2.1, clusters were self-contained entities and had no knowledge
of other clusters. With Ganeti 2.2, clusters can exchange data if tokens
(an encryption certificate) was exchanged by a trusted third party
before.

185
186
187
188
KVM Security
------------

When running KVM instances under Ganeti three security models ara
189
available: "none", "user" and "pool".
190

191
Under security model "none" instances run by default as root. This means
192
193
194
195
that, if an instance gets jail broken, it will be able to own the host
node, and thus the ganeti cluster. This is the default model, and the
only one available before Ganeti 2.1.2.

196
197
Under security model "user" an instance is run as the user specified by
the hypervisor parameter "security_domain". This makes it easy to run
Iustin Pop's avatar
Iustin Pop committed
198
199
200
201
202
all instances as non privileged users, and allows one to manually
allocate specific users to specific instances or sets of instances. If
the specified user doesn't have permissions a jail broken instance will
need some local privilege escalation before being able to take over the
node and the cluster. It's possible though for a jail broken instance to
203
204
affect other ones running under the same user.

205
Under security model "pool" a global cluster-level uid pool is used to
206
207
208
209
210
211
212
start each instance on the same node under a different user. The uids in
the cluster pool can be set with ``gnt-cluster init`` and ``gnt-cluster
modify``, and must correspond to existing users on all nodes. Ganeti
will then allocate one to each instance, as needed. This way a jail
broken instance won't be able to affect any other. Since the users are
handed out by ganeti in a per-node randomized way, in this mode there is
no way to make sure a particular instance is always run as a certain
213
user. Use mode "user" for that.
214
215
216
217
218
219
220
221
222
223
224
225
226

In addition to these precautions, if you want to avoid instances sending
traffic on your node network, you can use an iptables rule such as::

  iptables -A OUTPUT -m owner --uid-owner <uid>[-<uid>] -j LOG \
    --log-prefix "ganeti uid pool user network traffic"
  iptables -A OUTPUT -m owner --uid-owner <uid>[-<uid>] -j DROP

This won't affect regular instance traffic (that comes out of the tapX
allocated to the instance, and can be filtered or subject to appropriate
policy routes) but will stop any user generated traffic that might come
from a jailbroken instance.

227
.. vim: set textwidth=72 :
Iustin Pop's avatar
Iustin Pop committed
228
229
230
231
.. Local Variables:
.. mode: rst
.. fill-column: 72
.. End: