Commit 8193c499 authored by Georgios Ouzounis's avatar Georgios Ouzounis
Browse files

Changes on Lambda Application guides' text.

parent bec1b505
......@@ -136,13 +136,13 @@
<div class="box-header with-border">
<h3 class="box-title">
<span class="label label-warning"><i class="fa fa-th"></i></span>
How to export my immutable data</h3>
How to export my master dataset</h3>
</div><!-- /.box-header -->
<div class="box-body">
<div class="row">
<div class="col-md-12">
<!-- Conversations are loaded here -->
Follow the steps described in this section to export your immutable data.
Follow the steps described in this section to export your master dataset.
</div><!--col-->
</div><!--row-->
<div class="row">
......@@ -162,14 +162,14 @@
<div class="box-header with-border">
<h3 class="box-title">
<span class="label label-warning"><i class="fa fa-th"></i></span>
How to check what is running</h3>
How to check if my application is running</h3>
</div><!-- /.box-header -->
<div class="box-body">
<div class="row">
<div class="col-md-12">
<!-- Conversations are loaded here -->
Follow the steps described in this section to check what is running.
Follow the steps described in this section to check if your application is running.
</div><!--col-->
</div><!--row-->
<div class="row">
......
<h3>Lambda Applications</h3>
<p>One may find an excellent guide on writing Lambda Applications on the landing page of the LoD service. The source code exhibited through that example is available on <a href="https://github.com/grnet/okeanos-LoD/tree/master/example">github</a>. For easy reference, the aforementioned guide is also include in this page.</p>
<p>One may find a guide on writing Lambda Applications on the landing page of the LoD service. The source code exhibited through that example is available on <a href="https://github.com/grnet/okeanos-LoD/tree/master/example">github</a>. For easy reference, the aforementioned guide is also include in this page.</p>
<p>In order to upload, deploy and start running a Lambda Application on top of a Lambda Instance the user of the service will go through the workflow shown below, again using the Web UI of the LoD service VM.</p>
......@@ -139,18 +139,18 @@ In order to showcase the full capabilities of your Lambda Instance this example
<br><br>
<h4>Application</h4>
An application is program written in the Java programming language. Each application should be compiled and assembled in a single .jar file before being uploded on Pithos+ through the web pages of a LoD Service VM. Inside your Java code, you can use the <a href="https://ci.apache.org/projects/flink/flink-docs-release-0.10/quickstart/java_api_quickstart.html">Apache Flink Java API</a> to utilize the full capabilities of your Lambda Instance.
An application is program written in the Java programming language. Each application should be compiled and assembled in a single .jar file before being uploaded on Pithos+ through the LoD Service. Inside your Java code, you can use the <a href="https://ci.apache.org/projects/flink/flink-docs-release-0.10/quickstart/java_api_quickstart.html">Apache Flink Java API</a> to utilize the full capabilities of your Lambda Instance.
<br><br>
<h4>Apache Kafka Topic</h4>
Apache Kafka is the data ingestion layer used by a Lambda Instance. It maintains feeds of messages in categories called topics. Each topic has its own name and can be used either to write or read data from it. Upon creation of a Lambda Instance, you can specify the names of the Kafka topics you want to be created. If no name is provided, then three topics will be created, named "input", "stream-output" and "batch-output". The topics created on a Lambda Instance are categorized to "input" and "output" topics. All the data
sent to an input topic, will be automatically saved on the immutable dataset described below.<br>
sent to an input topic, will be automatically appended to the master dataset described below.<br>
To get more information about Apache Kafka, you can visit <a href="kafka.apache.org">kafka.apache.org</a>
<br><br>
<h4>Immutable Dataset</h4>
The immutable dataset is the place on a Lambda Instance where all input data will be saved and is implemented using Apache HDFS. As stated in the previous paragraph, any data sent on a topic categorized as input topic, will be saved on the immutable dataset automatically. This is done using Apache Flume.<br>
<h4>Master Dataset</h4>
The master dataset is an immutable, append-only set of raw data that is stored in the Apache HDFS cluster of your Lambda Instance. As stated in the previous paragraph, any data sent on a topic categorized as input topic, will be appended to the master dataset automatically via Apache Flume.<br>
For more information regarding Apache HDFS and FLume you can visit <a href="hadoop.apache.org">hadoop.apache.org</a> and <a href="flume.apache.org">flume.apache.org</a> respectively.
<br><br>
......@@ -232,7 +232,7 @@ In order to use Apache Flink in your application, you need to create an executio
env.execute("Stream Word Count");
</pre>
Note that, when calling the "execute" method of the environment, a name is provided in the form of a String. This name will be requested when uploading your application on a LoD Service VM, so you need to make sure this name is unique for each of your applications.
<b><i>Note that, when calling the "execute" method of the environment, a name is provided in the form of a String. This name will be requested when uploading your application on a LoD Service VM, so you need to make sure this name is unique for each of your applications.</i></b>
<br><br>
......
......@@ -3,7 +3,7 @@
<div class="col-xs-12">
<div class="box box-primary">
<div class="box-header with-border">
<h3 class="box-title">How to check what is running</h3>
<h3 class="box-title">How to check if my application is running</h3>
<div class="box-tools pull-right">
<button class="btn btn-box-tool" data-widget="collapse"><i class="fa fa-minus"></i></button>
<button class="btn btn-box-tool" data-widget="remove"><i class="fa fa-times"></i></button>
......@@ -35,7 +35,7 @@
<h1>Check Running Applications</h1>
<p>
To view your Lambda Applications you can follow the guide show below.
To view your Lambda Applications you can follow the steps below.
</p>
<div class="row margin-bottom">
......@@ -84,7 +84,7 @@
<ol>
<li>
Log in the LoD Service VM using your ~okeanos token.
Log into the LoD Service VM using your <a href="https://accounts.okeanos.grnet.gr/ui/api_access">~okeanos token</a>.
</li>
<li>
......@@ -92,7 +92,7 @@
</li>
<li>
On the list of Lambda Applications that appear, choose the button "Details" to get more details about a specific Application.
From the list of Lambda Applications that appear, click "Details" to see more information about a specific Application.
</li>
</ol>
</div><!--col-->
......
......@@ -3,7 +3,7 @@
<div class="col-xs-12">
<div class="box box-primary">
<div class="box-header with-border">
<h3 class="box-title">How to export my immutable data</h3>
<h3 class="box-title">How to export my master dataset</h3>
<div class="box-tools pull-right">
<button class="btn btn-box-tool" data-widget="collapse"><i class="fa fa-minus"></i></button>
<button class="btn btn-box-tool" data-widget="remove"><i class="fa fa-times"></i></button>
......@@ -12,7 +12,7 @@
<div class="box-body no-padding">
<div class="row">
<div class="col-md-12">
<div class="pad"> Follow the steps described in this section to export your immutable data.
<div class="pad"> Follow the steps described in this section to export your master dataset.
</div>
</div><!-- /.col -->
......@@ -28,22 +28,21 @@
<!-- DIRECT CHAT DANGER -->
<div class="box box-success ">
<div class="box-header with-border">
<h3 class="box-title">Short guide on how to export the immutable data from a Lambda Instance</h3>
<h3 class="box-title">Short guide on how to export the master dataset from a Lambda Instance</h3>
</div><!-- /.box-header -->
<div class="box-body">
<div class="row">
<div class="col-md-12">
<h1>Export Immutable Data</h1>
<h1>Export Master Dataset</h1>
<p>
<!-- TODO Explain with pictures where to find the hostname of the Master Node -->
The immutable data of each Lambda Instance, are saved on its <a href="https://hadoop.apache.org">Hadoop Distributed File System(HDFS)</a>.
In order to export them, you will have to connect over ssh on the Master Node of your Lambda Instance and create a snapshot of the HDFS. After
The master dataset of each Lambda Instance, is stored on its <a href="https://hadoop.apache.org">Hadoop Distributed File System(HDFS)</a>.
In order to export them, you will have to connect over ssh to the Master Node of your Lambda Instance and create a snapshot of the HDFS. After
that, you can move the snapshot on the file system of your VM and then, secure copy it on your local machine, upload it on the cloud etc...
</p>
<p>
The step-by-step directives your should execute are the following:
The step-by-step directives you should execute are the following:
<ol>
<li>
......@@ -85,12 +84,12 @@ ssh -l root snf-XXXXXX.vm.okeanos.grnet.gr
</li><br>
<li>
Copy the snapshot on the local file system of your Laptop:
Copy the snapshot on the local file system of your computer:
<pre>
scp -r root@snf-XXXXXX.vm.okeanos.grnet.gr:/root/s0 .
</pre>
In the above directive, you should again replace the XXXXXX with the id of your VM. Also, the path "/root/s0" indicates the place of the SnapShot directory on the
VM and "." indicates the place where the SnapShot will be copied on your Laptop. You can change these paths to fit your needs.
VM and "." indicates the local directory in which the SnapShot will be copied to. You can change these paths to fit your needs. The format of the data is raw text.
</li>
</ol>
</p>
......
......@@ -54,11 +54,11 @@
ssh -l root snf-XXXXXX.vm.okeanos.grnet.gr
</pre>
where XXXXXX should be replaced with the id of the VM(e.g. snf-543786.vm.okeanos.grnet.gr). To find the id of the Master Node
of your Lambda Instance you can use {{#link-to 'faqs.lambda-instance.find-master-node-hostname'}}this{{/link-to}} guide.
of your Lambda Instance follow the instructions from {{#link-to 'faqs.lambda-instance.find-master-node-hostname'}}this{{/link-to}} guide.
</li><br>
<li>
Move to the directory where the log files are places:
Move to the directory with the log files:
<pre>
cd /home/flink
</pre>
......
......@@ -33,25 +33,10 @@
<div class="box-body">
<div class="row">
<div class="col-md-12">
<h1>Getting Results</h1>
<h1>Getting the Results</h1>
<p>
Lambda on Demand service is used to process data. This means that, to get results, you first have to provide some data.
To do that, you should first {{#link-to 'index'}}create a LoD service VM{{/link-to}}, a {{#link-to 'faqs.lambda-instance.create'}}
Lambda Instance{{/link-to}} and a {{#link-to 'faqs.lambda-application.create'}}Lambda Application{{/link-to}}.
</p>
<p>
After you have created these three elements, follow {{#link-to 'faqs.lambda-application.run'}}this{{/link-to}} guide to find out how
to upload your Lambda Application on your Lambda Instance using the web interface of your LoD service VM. You can follow
{{#link-to 'faqs.lambda-application.observe-running'}}this{{/link-to}} guide to observe your runnning applications on a LoD service VM.
</p>
<p>
At this point, you have successfully started a Lambda Application on a Lambda Instance. As stated before, in order to get results, you
first have to provide some data. To send data on a Lambda Instance you can use Apache Kafka which is installed by default. Follow the
{{#link-to 'faqs.lambda-instance.kafka-topics'}}Kafka topics guide{{/link-to}} to get information on how to send data to an Apache Kafka
topic and also how to receive the results of your Lambda Application's processing.
To get the results, you should configure your application to send them to an Apache Kafka topic. After that, you can read {{#link-to 'faqs.lambda-instance.kafka-topics'}}this{{/link-to}} guide to find out how to get the results from the topic you sent them.
</p>
</div><!--col-->
</div><!--row-->
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment