Primjena MapReduce algoritma na analizu nizova tekstualnih podataka

master's thesis

• Author: Karolina Volarić
• Mentor: Luka Grubišić (mentor)
• Committee member: Luka Grubišić (predsjednik povjerenstva)
• Committee member: Ivica Nakić (član povjerenstva)
• Committee member: Vedran Krčadinac (član povjerenstva)
• Committee member: Dragutin Svrtan (član povjerenstva)
• Granter: University of Zagreb; Faculty of Science; (Department of Mathematics); Zagreb
• Defense date and country: 2014-07-14, Croatia
• Scientific / art field, discipline and subdiscipline: NATURAL SCIENCES; Mathematics

Abstract

U ovom radu je opisan način rada i primjena Apache Hadoop-a i njegovih komponenti. Najvažnija komponenta je MapReduce koja ima sve veću primjenu. Da bismo mogli koristiti MapReduce algoritam potrebno je razumjeti njegov način rada, te naučiti neka pravila za pisanje samog algoritma kao što je korištenje combiner funkcije. Da bi u potpunosti razumjeli koncept Hadoop-a, objašnjeni su pojmovi Flume, Hive, HDFS, te Oozie. Primjena Hadoop-a i MapReduce-a je pokazana na analizi društvene mreže Twitter. Podaci su prikupljani prema određenim uvjetima pomoću Apache Flume-a, zatim su obrađeni s Oozie-em, a upiti nad njima su izvršeni pomoću Hive-a.

Abstract (english)

This thesis describes the operation and use of Apache Hadoop and its components. The most important component is MapReduce. To use MapReduce algorithm it is necessary to understand its mode of operation, and learn some of the rules for writing the algorithm as well as the use of the combiner function. In order to fully understand the concept of Hadoop, the following concepts are explained: Flume, Hive, HDFS and Oozie. Use of Hadoop and MapReduce is shown in the analysis of social network Twitter. Data were collected according to certain conditions using Apache Flume, then they were processed with Oozie-operation and queried using Hive.

• Keywords: primjena Apache Hadoop-a; MapReduce algoritam; Flume; Hive; HDFS; Oozie; Twitter
• Keywords (english): use of Apache Hadoop; MapReduce algorithm; Flume; Hive; HDFS; Oozie; Twitter
• Language: croatian
• URN:NBN: urn:nbn:hr:217:795312
• Study programme: Title: Computer Science and Mathematics; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra računarstva i matematike (magistar/magistra računarstva i matematike)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2019-02-19 15:03:33