MapReduce basics

MapReduce

dsl5.example.com: invalid  user ‘bart’

dsl5.example.com: invalid  user ‘bart’

dsl5.example.com: invalid  user ‘lisa’

dsl5.example.com: invalid  user ‘dave’

dsl5.example.com: user ‘admin’ logs in successfully

—-> map

dsl5.example.com: 1

dsl5.example.com: 1

dsl5.example.com: 1

—-> reduce

dsl5.example.com: 3

Used in large sets of data, like analysing server logs, spam filter, etc.

Haloop,  a Java framework.

cd /tmp/hadoop-1.0.4 bin/hadoop jar \

contrib/streaming/hadoop-streaming-1.0.4.jar \ -mapper /tmp/map.py -reducer /tmp/reduce.py \ -input /tmp/log.txt -output /tmp/output

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s