design-impexp2.rst

Design for import/export version 2

Contents

• Current state and shortcomings
• Proposed changes
  • Software requirements
  • Transport
  • SSL certificates
  • Source
    • Option 1: CGI
    • Option 2: FastCGI
  • Destination
• The big picture for instance moves
• HTTP resources on source
• Code samples
• Backwards compatibility
  • Version 1
  • EXP_SIZE_FD
  • Compression
• Notes
• Performance considerations
• Other considered solutions

Current state and shortcomings

Ganeti 2.2 introduced :doc:`inter-cluster instance moves <design-2.2>` and replaced the import/export mechanism with the same technology. It's since shown that the chosen implementation was too complicated and and can be difficult to debug.

The old implementation is henceforth called "version 1". It used socat in combination with a rather complex tree of bash and Python utilities to move instances between clusters and import/export them inside the cluster. Due to protocol limitations, the master daemon starts a daemon on the involved nodes and then keeps polling a status file for updates. A non-trivial number of timeouts ensures that jobs don't freeze.

In version 1, the destination node would start a daemon listening on a random TCP port. Upon receiving the destination information, the source node would temporarily stop the instance, create snapshots, and start exporting the data by connecting to the destination. The random TCP port is chosen by the operating system by binding the socket to port 0. While this is a somewhat elegant solution, it causes problems in setups with restricted connectivity (e.g. iptables).

Another issue encountered was with dual-stack IPv6 setups. socat can only listen on one protocol, IPv4 or IPv6, at a time. The connecting node can not simply resolve the DNS name, but it must be told the exact IP address.

Instance OS definitions can provide custom import/export scripts. They were working well in the early days when a filesystem was usually created directly on the block device. Around Ganeti 2.0 there was a transition to using partitions on the block devices. Import/export scripts could no longer use simple dump and restore commands, but usually ended up doing raw data dumps.