This adds basic infrastructure for filtering (fully functional except,
as usual, for runtime data), and then uses it for node queries.

Since the filtering exports regex matching as an external
functionality, we have to use a regex library. There are many flavours
of these in Haskell (see
http://www.haskell.org/haskellwiki/Regular_expressions

), but since we
want to be as compatible as we can with Python's, we use the
regex-pcre one, which is a wrapper to the widely used 'pcre' library.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

I'm doing this change for future performance optimisations. Currently
we use the Luxi interface just as a client, so not in the hot path,
but when we'll use this as a server interface, we're interested to
both reduce the space and time consumption of the interface.

We have to simultaneous changes here:

- switch from using socket-related function (sendto, recv, etc.) to
  handle-based functions, since the standard network library doesn't
  work with sockets
- switch from using Strings for the internal buffer to strict
  ByteStrings; the only downside is that we now have the issue of
  decoding/encoding from binary to UTF-8 strings, a fact which brings
  its own issues into the mix (we have to check for failed decodings,
  etc.); but this is similar to what we'll have to handle on the
  Python side when moving to Python 3.x

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

It seems that 'explicitely' is wrong, and that the right form is
'explicitly'. This is just fixing the typo plus adjusting affected
paragraphs.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This patch aims at improving documentation for installing ganeti - both for users and developers.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Hinotify was not marked as a dependency for hconfd.

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Added the new Haskell library requirements, for both normal and
developer usage.

Furthermore, all commands are now converted to the shell-example
lexer.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add documentation and modify manpages for the RBD disk template.

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Stratos Psomadakis <psomas@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

While testing with ghc 7.2, I saw that some imports we are using are
very old (from ghc 6.8 time), even though current libraries are using
different names.

We fix this and bump minimum documented version to ghc 6.12, as I
don't have 6.10 to test anymore (possibly still works with that
version, but better safe - both Ubuntu Lucid and Debian Squeeze ship
with 6.12 nowadays).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add a shell script that will be the default replacement for the body of
backend.ActivateMasterIp and backend.DeactivateMasterIp.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Also added optional affinity Python package for CPU pinning under KVM
to the INSTALL doc.

Signed-off-by: Tsachy Shacham <tsachy@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

These were forgot when the supported library versions were changed.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

With this parser, command line utilities will be able to provide filters
through query2 in a simplistic language. Example filters:

  name == "node3.example.com"
  master or (name == "node4.example.com")
  be/memory == 128 and name =~ /^web/i
  "inst1.example.com" in sinst_list
  status != "up"
  not master

Parts of the syntax came from Python, others from Perl. Documentation
will be added in follow-up patches.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Was not sure about the bit in admin.rst, hope it's fine.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

This documents the needed libraries for Haskell development. It also
fixes a tiny typo in devnotes.rst.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

This uses an option only available in patched socat versions. More
information is available from the INSTALL update included in this
patch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This uses an option only available in patched socat versions. More
information is available from the INSTALL update included in this
patch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Signed-off-by: Manuel Franceschini <livewire@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Currently the RAPI client uses the urllib2 and httplib modules from
Python's standard library. They're used with pyOpenSSL in a very fragile
way, and there are known issues when receiving large responses from a RAPI
server.

By switching to PycURL we leverage the power and stability of the
widely-used curl library (libcurl). This brings us much more flexibility
than before, and timeouts were easily implemented (something that would
have involved a lot of work with the built-in modules).

There's one small drawback: Programs using libcurl have to call
curl_global_init(3) (available as pycurl.global_init) while exactly one
thread is running (e.g. before other threads) and are supposed to call
curl_global_cleanup(3) (available as pycurl.global_cleanup) upon exiting.
See the manpages for details. A decorator is provided to simplify this.

Unittests for the new code are provided, increasing the test coverage of
the RAPI client from 74% to 89%.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

- Update NEWS file
- Remove dependency on OpenSSL (pyOpenSSL remains)
- Update manpages, fix typos and other things

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Xen: Just specify 3.0 or above as a requirement, as versions above 3.1
have been tested and work fine with Ganeti. Also note that the Xen
dependency is optional, if one wants to work with KVM.

KVM: list as an alternative, specify that 72 (the one shipped with
lenny) is the bare minimum, but newer qemu-kvm is recommended.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

This is in preparation to including it into the large
documentation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

This big patch converts the documentation build system to sphinx
(http://sphinx.pocoo.org/

). Since that uses reStructuredText sources
too, there is no change (yet) in the documents themselves, just in the
build system.

As before, the docs are pre built by the maintainer, and the end-user
doesn't need sphinx or other rst tools to build the docs. Note that we
are not distributing PDFs, so building that will require the tools.

The docs will be stored under doc/html and the build system also need an
extra directory doc/build. These are considered (by automake)
maintainer-related objects and are removed at maintainer-clean time.

The patch also fixes some small issues: add a docpng variable, add
doc/api (also generated by maintainer) in maintainer-clean-local, etc.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Reviewed-by: iustinp

I forgot to remove these when converting the QA configuration from YAML
to JSON.

Reviewed-by: ultrotter

Since we have removed support for local and remote raid1, update the man
pages and guides to reflect the new situation.

Reviewed-by: imsnah

Since 8.0.6 and lower have issues that break Ganeti's usage of drbd,
this patch notes the required version of DRBD.

Reviewed-by: ultrotter

This patch does:
  - add drbd8 information to the docs (the new disk template, examples,
    etc.)
  - fixes a section header to make it more clear
  - update the NEWS file by changing format (format taken by looking at
    /usr/sharea/doc/*/NEWS.gz) and adding more information on changes
    since beta1

Reviewed-by: ultrotter

This duplicates some code from the DRBDev class, but not very much, and
it will be expanded with the new functionality available for the 8.x
version. Currently the code is not accessible outside the module.

This patch introduces a dependency on the pyparsing module.

Reviewed-by: imsnah

This patch completely  gets rid of fping
 - replace all fping invocations with TcpPing calls
 - update documentation accordingly.
 - associated cleanups (use constant for localhost IP, use more sensible
   defaults for TcpPing and _use_ those)

Reviewed-by: iustinp

- Add NEWS file with major changes between versions.
- Bump RPC version number
- No longer serialize in RPC, but just convert to dict

Old Pickle based configuration files can be converted using the cfgupgrade
utility.

Reviewed-by: iustinp, ultrotter

Reviewed-by: ultrotter

- Add Makefile in doc/examples/

Reviewed-by: schreiberal

This is more in line with the other document names.

Also, change the title to simply "Ganeti installation tutorial".

Reviewed-by: ultrotter

In case one installs from source, make, tar and gzip are also needed. tar and
gzip are almost always installed, but make not, so let's add a note about that.

Also remove a '>' in the apt-get install example so copy-paste works.

Reviewed-by: imsnah