Commit 84534f2b authored by Ilias Tsitsimpis's avatar Ilias Tsitsimpis
Browse files

docs: Improve snf-burnin documenation

parent 8dfe6b85
......@@ -4,100 +4,108 @@ snf-burnin
^^^^^^^^^^
:ref:`snf-burnin <snf-burnin>` is an integration testing tool for a running
Synnefo deployment. It uses the Synnefo REST APIs to run test scenarios for the
following categories:
Synnefo deployment. Using the Synnefo REST APIs, it simulates a real user and
tries to identify any bugs or performance issues by running a series of a tests.
The tests are devided into the following categories:
* :ref:`Authentication <unauthorizedtestcase>`
* :ref:`Images <imagestestcase>`
* :ref:`Flavors <flavorstestcase>`
* :ref:`Servers <serverstestcase>`
* :ref:`Network <networktestcase>`
* :ref:`Storage <pithostestcase>`
* :ref:`Authentication Tests <unauthorizedtestcase>`
* :ref:`Image Tests <imagestestcase>`
* :ref:`Flavor Tests <flavorstestcase>`
* :ref:`Server Tests <serverstestcase>`
* :ref:`Network Tests <networktestcase>`
* :ref:`Storage Tests <pithostestcase>`
Usage
=====
**Example:**
:ref:`snf-burnin <snf-burnin>` is a command line tool written in python. It
supports a number of command line options though which the user can change the
behaviour of the tests.
A typical usage of snf-burnin is:
::
snf-burnin --token=TOKEN \
--api=CYCLADES_BASE_URL
--pithos=PITHOS_BASE_URL \
--astakos=ASTAKOS_BASE_URL \
--plankton=PLANKTON_URL \
--plankton-user=PLANKTON_SYSTEM_USER \
snf-burnin --token=USERS_SECRET_TOKEN \
--auth-url="https://accounts.synnefo.org/identity/v2.0/" \
--system-images-user=SYSTEM_IMAGES_USER_ID \
--image-id=IMAGE_ID \
--log-folder=LOG_FOLDER
For more info
The above options are the minimal ones (mandatory) that one has to speficy in
order for snf-burnin to properly function. The first two are the credentials
needed to access Synnefo's REST API and can be found in the user's dashboard.
The third is needed by some :ref:`Image Tests <imagestestcase>` as we will see
later. The forth tells snf-burnin which image to use for creating our test
servers and the last one specifies the log folder where any results should be
saves.
For more information about snf-burnin and it's command line options, run
snf-burnin with help.
::
snf-burnin --help
Usage: snf-burnin [options]
::
snf-burnin runs a number of test scenarios on a Synnefo deployment.
Options:
-h, --help show this help message and exit
--api=API The API URI to use to reach the Synnefo API
--plankton=PLANKTON The API URI to use to reach the Plankton API
--plankton-user=PLANKTON_USER
Owner of system images
--pithos=PITHOS The API URI to use to reach the Pithos API
--astakos=ASTAKOS The API URI to use to reach the Astakos API
--token=TOKEN The token to use for authentication to the API
--nofailfast Do not fail immediately if one of the tests fails
(EXPERIMENTAL)
--no-ipv6 Disables ipv6 related tests
--action-timeout=TIMEOUT
Wait SECONDS seconds for a server action to complete,
then the test is considered failed
--build-warning=TIMEOUT
Warn if TIMEOUT seconds have passed and a build
operation is still pending
--build-fail=BUILD_TIMEOUT
Fail the test if TIMEOUT seconds have passed and a
build operation is still incomplete
--query-interval=INTERVAL
Query server status when requests are pending every
INTERVAL seconds
--fanout=COUNT Spawn up to COUNT child processes to execute in
parallel, essentially have up to COUNT server build
requests outstanding (EXPERIMENTAL)
--force-flavor=FLAVOR ID
Force all server creations to use the specified FLAVOR
ID instead of a randomly chosen one, useful if disk
space is scarce
--image-id=IMAGE ID Test the specified image id, use 'all' to test all
available images (mandatory argument)
--show-stale Show stale servers from previous runs, whose name
starts with `snf-test-'
--delete-stale Delete stale servers from previous runs, whose name
starts with `snf-test-'
--force-personality=PERSONALITY_PATH
Force a personality file injection.
File path required.
--log-folder=LOG_FOLDER
Define the absolute path where the output
log is stored.
-V, --verbose Print detailed output about multiple processes
spawning
--set-tests=TESTS Set comma seperated tests for this run.
Available tests: auth, images, flavors,
servers, server_spawn,
network_spawn, pithos.
Default = all
-h, --help show this help message and exit
--auth-url=AUTH_URL The AUTH URI to use to reach the Synnefo API
--system-images-user=SYSTEM_IMAGES_USER
Owner of system images
--token=TOKEN The token to use for authentication to the API
--nofailfast Do not fail immediately if one of the tests fails
(EXPERIMENTAL)
--no-ipv6 Disables ipv6 related tests
--action-timeout=TIMEOUT
Wait SECONDS seconds for a server action to complete,
then the test is considered failed
--build-warning=TIMEOUT
Warn if TIMEOUT seconds have passed and a build
operation is still pending
--build-fail=BUILD_TIMEOUT
Fail the test if TIMEOUT seconds have passed and a
build operation is still incomplete
--query-interval=INTERVAL
Query server status when requests are pending every
INTERVAL seconds
--fanout=COUNT Spawn up to COUNT child processes to execute in
parallel, essentially have up to COUNT server build
requests outstanding (EXPERIMENTAL)
--force-flavor=FLAVOR ID
Force all server creations to use the specified FLAVOR
ID instead of a randomly chosen one, useful if disk
space is scarce
--image-id=IMAGE ID Test the specified image id, use 'all' to test all
available images (mandatory argument)
--show-stale Show stale servers from previous runs, whose name
starts with `snf-test-'
--delete-stale Delete stale servers from previous runs, whose name
starts with `snf-test-'
--force-personality=PERSONALITY_PATH
Force a personality file injection.
File path required.
--log-folder=LOG_FOLDER
Define the absolute path where the output
log is stored.
-V, --verbose Print detailed output about multiple processes
spawning
--set-tests=TESTS Set comma seperated tests for this run.
Available tests: auth, images, flavors,
servers, server_spawn,
network_spawn, pithos.
Default = all
Log files
=========
In each run, snf-burnin stores log files in the folder defined in the
--log-foler parameter, under the folder with the timestamp of the
snf-burnin-run and the image used for it. The name prefixes of the log
files are:
In each run, snf-burnin stores log files under the folder defined in the
--log-folder parameter. For every run, it creates a new subfolder using a
timestamp and the image-id as unique names. The name prefixes of the log files
are:
* details: Showing the complete log of snf-burnin run.
* error: Showing the testcases that encountered a runtime error.
......@@ -107,114 +115,173 @@ files are:
Detailed description of testcases
=================================
Here we have a complete list of all the tests snf-burnin performs, each listed
under the category in which it belongs. The user can choose to run some or all
of the categories listed below using the "--set-tests" command line flag.
.. _unauthorizedtestcase:
UnauthorizedTestCase
--------------------
* Test that trying to access without a valid token, fails
* Use a random token and try to authenticate to Astakos service. The expected
responce should be "401 Unauthorized".
.. _imagestestcase:
ImagesTestCase
--------------
* Test image list actually returns images
* Test detailed image list has the same length as list
* Test detailed and simple image list contain the same names
* Test system images have unique names
* Test every image has specific metadata defined
* Download image from Pithos
* Upload and register image
* Request from Cyclades the list of all registered images and check that its
length is greater than 0 (ie test that there are registered images for the
users to use).
* Request from Cyclades the list of all registered images with details and check
that is length is greater than 0.
* Test that the two lists retrieved earlier contain exactly the same images.
* Using the SYSTEM_IMAGES_USER_ID choose only the images that belong to the
system user and check that their names are unique. This test can not be
applied for all images as the users can name their images whatever they want.
* Again for the images that belong to the system user check that the "osfamily"
and the "root_partition" metadata values have been defined. These metadata
values are mandatory for an image to be used.
* Download from Pithos+ the image specified with the "--image-id" parameter and
save it locally.
* Create a new container to Pithos+ named "images".
* Upload the download image to Pithos+ under the "images" container.
* Use Plankton service to register the above image. Set the "osfamily" and
"root_partition" metadata values which are mandatory.
* Request from Cyclades the list of all registered images and check that our
newly registered image is among them.
* Delete image from Pithos+ and also the local copy on our disk.
.. _flavorstestcase:
FlavorsTestCase
---------------
* Test flavor list actually returns flavors
* Test detailed flavor list has the same length as list
* Test detailed and simple flavor list contain the same names
* Test flavors have unique names
* Test flavor names have correct format
* Request from Cyclades the list of all flavors and check that its length is
greater than 0 (ie test that there are flavors for the users to use).
* Request from Cyclades the list of all flavors with details and check that its
length is greater than 0.
* Test that the two lists retrived earlier contain exactly the same flavors.
* Test that all flavors have unique names.
* Test that all flavors have a name of the form CxxRyyDzz where xx is the vCPU
count, yy is the RAM in MiB, and zz is the Disk in GiB.
.. _serverstestcase:
ServersTestCase
---------------
* Test simple and detailed server list have the same length
* Test simple and detailed servers have the same names
* Request from Cyclades the list of all servers with and without details and
check that the two lists have the same length.
* Test that simple and detailed servers lists have the same names.
SpawnServerTestCase
-------------------
* Submit create server
* Test server is in BUILD state in server list
* Test server is in BUILD state in server details
* Change server metadata
* Verify the changed metadata are correct
* Verify server metadata are set based on image metadata
* Wait until server changes state to ACTIVE, and verify state
* Test if OOB server console works
* Test if server has IPv4
* Test if server has IPv6
* Test if server responds to ping on IPv4 address
* Test if server responds to ping on IPv6 address
* Submit shutdown request
* Verify server status is STOPPED
* Submit start request
* Test server status is ACTIVE
* Test if server responds to ping on IPv4 address (verify up and running)
* Test if server responds to ping on IPv6 address (verify up and running)
* Test SSH to server and verify hostname (IPv4)
* Test SSH to server and verify hostname (IPv6)
* Test RDP connection to server (only for Window Images) (IPv4)
* Test RDP connection to server (only for Window Images) (IPv6)
* Test file injection for personality enforcement
* Submit server delete request
* Test server becomes DELETED
* Test server is no longer in server list
* Submit a create server request to Cyclades service. Use the IMAGE_ID specified
from the command line. If FLAVOR_ID was specified as well use that one, else
choose one randomly. The name of the new server will start with "snf-test-"
followed by a timestamp so we can know which servers have been created from
snf-burnin and when. Also check that the response from Cyclades service
contains the correct server_name, server_flavor_id, server_image_id and the
status of the server is currenlty "BUILD". Finally from the above response,
extract the server's id and password.
* Request from Cyclades the list of all servers with details and check that our
newly created server has correct server_name, server_flavor_id,
server_image_id and the status is "BUILD".
* Request from Cyclades the details from the image we used to build our server.
Extract the "os" and "users" metadata values. Using the first one update the
server's metadata and setup the "os" metadata value to be the same with the
one from the image's metadata. Using the second one determine the username to
use for future connections to this host.
* Retrieve the server's metadata from Cyclades and verify that server's metadata
"os" key is set based on image's metadata.
* Wait until server changes state to ACTIVE. This is done by querying the
service for the server's state every QUERY_INTERVAL period of time until
BUILD_TIMEOUT has been reached. Both QUERY_INTERVAL and BUILD_TIMEOUT values
can be changed from the command line.
* Request from Cyclades service a VNC console to our server. In order to verify
that the returned connection is indeed a VNC one, snf-burnin implements the
first basic steps of the RFB protocol:
* Step 1. Send the ProtocolVersion message (par. 6.1.1)
* Step 2. Check that only VNC Authentication is supported (par 6.1.2)
* Step 3. Request VNC Authentication (par 6.1.2)
* Step 4. Receive Challenge (par 6.2.2)
* Step 5. DES-Encrypt challenge, using the password as key (par 6.2.2)
* Step 6. Check that the SecurityResult is correct (par 6.1.3)
* Request from Cyclades the server's details and check that our server's has
been assigned with an IPv4 address.
* Check that our server has been assigned with an IPv6 address. This test can be
skipped if for some reason the targeted Synnefo deployment doesn't support
IPv6.
* Test that our server responds to ping requests on IPv4 address.
* Test that our server responds to ping requests on IPv6 address. This test can
also be skipped.
* Submit a shutdown request for our server.
* Wait and verify that the status of our server became "STOPPED".
* Submit a start request for our server.
* Wait and verify that the status of our server became "ACTIVE" again.
* Test if server responds to ping on IPv4 address (verify up and running).
* Test if server responds to ping on IPv6 address (verify up and running).
* If the server is a Linux machine, SSH to it using its IPv4 address and verify
that it has a valid hostname.
* If the server is a Linux machine, SSH to it using its IPv6 address and verify
that it has a valid hostname.
* If the server is a Windows machine, try to connect to its RDP port using both
its IPv4 and IPv6 addresses.
* If during the creation of the server, the user chose a personality file to be
used, check that this file is been presented in the server and its contents
are correct.
* Submit server delete request.
* Wait and verify that the status of our server became "DELETED".
* Request from Cyclades the list of all servers and verify that our newly
deleted server is not in the list.
.. _networktestcase:
NetworkTestCase
---------------
* Submit create server A request
* Test server A becomes ACTIVE
* Submit create server B request
* Test server B becomes ACTIVE
* Submit create private network request
* Submit connect VMs to private network
* Test if VMs are connected to network
* Submit reboot request to server A
* Test server A responds to ping on IPv4 address (verify up and running)
* Submit reboot request to server B
* Test server B responds to ping on IPv4 address (verify up and running)
* Connect via SSH and setup the new network interface in server A
* Connect via SSH and setup the new network interface in server B
* Connect via SSH to server A and test if server B responds to ping on IPv4 address
* Disconnect servers from network and verify the network details
* Send delete network request and verify that the network is deleted from the list
* Send request to delete servers and wait until they are actually deleted
* Submit create server A request.
* Wait and verify that the status of our A server became "ACTIVE".
* Submit create server B request.
* Wait and verify that the status of our B server became "ACTIVE".
* Submit create private network request. Wait and verify that the status of the
network became "ACTIVE".
* Connect the two servers (A and B) into the newly created network. Wait and
verify that both machines got an extra nic, hence have been connected to the
network.
* Reboot server A.
* Test if server A responds to ping on IPv4 address (verify up and running)
* Reboot server B.
* Test if server B responds to ping on IPv4 address (verify up and running)
* Connect via SSH and setup the new network interface in server A.
* Connect via SSH and setup the new network interface in server B.
* Connect via SSH to server A and test if server B responds to ping via their
new interface.
* Disconnect both servers from network. Check network details and verify that
both servers have been successfully disconnected.
* Send delete network request. Verify that the network has been actually
deleted.
* Send request to delete servers and wait until they are actually deleted.
.. _pithostestcase:
PithosTestCase
--------------
* Test container list is not empty
* Test containers have unique names
* Create a new container
* Upload simple file to newly created container
* Download file from Pithos and test it is the same with the one uploaded
* Remove created file and container from Pithos
Example scripts
===============
Under /snf-tools/conf you can find example scripts for automating snf-burnin
testing using cron.
* **snf-burnin-run.sh** runs snf-burnin with the given parameters, deletes
stale instances (servers, networks) from old runs and delete logs older
than a week. It aborts if snf-burnin runs for longer than expected.
* **snf-burnin-output.sh** checks for failed snf-burnin tests the last 30
minutes in a given log folder. Exit status is 0 if no failures were
encountered, else exit status is 1.
* Request from Pithos+ the list of containers and check that its length is
greater than 0 (ie test that there are containers).
* Test that the containers have unique names.
* Create a new container. Choose a random name for our container and then check
that it has been successfully created.
* Upload a file to Pithos+ under our newly created container.
* Download the file from Pithos+ and test it is the same with the one uploaded.
* Remove created file and container from Pithos+ and verify that they have been
successfully deleted.
Burnin as alert tool
========================
Burnin can be used to verify that a Synnefo deployment is working as expected
and verify the admins in case of an error. For this there is a script under the
/snf-tools/conf directory named **snf-burnin-run.sh** which is intended to be
used from cron to periodically run burnin. It runs simultaneous many instances
of burnin for a number of different users and report errors though email.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment