Commit c687afff authored by Themis Zamani's avatar Themis Zamani
Browse files

Merge pull request #14 from gouzouni625/LAM-22

LAM-22 Kafka stream input consumer.
parents 528baae6 e4085d4b
#!/bin/bash
# This script should be executed on the machine on which zookeeper is running. To execute it on another machine, change the
# "localhost" on the following command with the IP address of the machine that runs zookeeper.
/usr/local/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic input;
#!/bin/bash
# Change "zookeeper-host" with the IP address of the machine that runs Apache Zookeeper.
# Change "application-master-host" with the IP address of the machine tha runs the Apache Flink Application Master.
# This command will create a live feed of data, from the topic "input" to the port 9999 of the Application Master host.
# The stream job will then read the data from the port 9999.
# The Application Master is responsible for distributing the data to the machines that run the Apache Flink job.
/usr/local/kafka/bin/kafka-console-consumer.sh --consumer.config /usr/local/kafka/config/consumer-stream.properties --zookeeper zookeeper-host:2181 --topic input | nc -lk application-master-host -p 9999;
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# see kafka.consumer.ConsumerConfig for more details
# Zookeeper connection string
# comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
zookeeper.connect=127.0.0.1:2181
# timeout in ms for connecting to zookeeper
zookeeper.connection.timeout.ms=6000
#consumer group id
group.id=consumer-stream
#consumer timeout
consumer.timeout.ms=5000
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment