Apache Flume – Ingesting log data into Hadoop and Kafka
Free Download Udemy Apache Flume – Ingesting log data into Hadoop and Kafka. With the help of this course you can Detailed workshop about using Flume to ingest web server logs into live Hadoop and Kafka Cluster..
This course was created by Durga Viswanatha Raju Gadiraju. It was rated 4.7 out of 5 by approx 4027 ratings. There are approx 49011 users enrolled with this course, so don’t wait to download yours now. This course also includes 3 hours on-demand video, Full lifetime access, Access on mobile and TV & Certificate of Completion.
What Will You Learn?
-
Understand basics of Flume
-
Implement simple flume agent
-
Understand multiple flume agent flows
-
Setup multiple agents using Avro as connector
-
Setup agent to get log data into HDFS
-
Understand sources like netcat, avro and exec in detail
-
Understand channels like memory and file in detail
-
Understand sinks like logger, avro and HDFS in detail
As part of this session we will understand how we can use Apache Flume to ingest streaming real time data in detail.
-
Overview of Flume
-
Setting up gen_logs
-
Develop first Flume Agent
-
Understand Source, Sink and Channel
-
Flume Multi Agent Flows
-
Get data into HDFS using Flume
-
Limitations and Conclusion
For this demo we will be using our Big Data developer labs. You need to have access to existing big data cluster or sign up to our labs.
Hands On demos:
-
Developing simple Flume agent to get data from netcat to agent logs
-
Develop multi agent flow where data from web server logs go to avro sink and then from avro source to logger
-
Develop multiplex flow where data from web server logs is written to HDFS and Kafka
Buy Udemy’s Apache Flume – Ingesting log data into Hadoop and Kafka
Free Download Udemy’s Apache Flume – Ingesting log data into Hadoop and Kafka
4.5