EMR 事始め

とりあえずここをすすめる。

http://docs.amazonwebservices.com/ElasticMapReduce/latest/GettingStartedGuide/Welcome.html

まずは、Amazon Linux上で、Ruby CLIをインストール

http://aws.amazon.com/developertools/2264

$ mkdir -p ~/aws/emr
$ cd ~/work/emr
$ wget http://elasticmapreduce.s3.amazonaws.com/elastic-mapreduce-ruby.zip
$ unzip elastic-mapreduce-ruby.zip -d /path/to/emrclient/
$ cd /path/to/emrclient/
$ vi credentials.json
$ cat credentials.json
{
"access_id": "AKIAILTPC3JEDW2YUEYQ",
"private_key": "xxxxxxxxx", 
"key-pair": "id_rsa",
"key-pair-file": "/home/ec2-user/.ssh/id_rsa",
"log_uri": "s3://yasu-emr-log/logs",
"region": "ap-northeast-1"
}

 CLIの動作確認

$ elastic-mapreduce --list
j-2P9MU1B0RWWW2     TERMINATED     ec2-176-34-10-240.ap-northeast-1.compute.amazonaws.comyasuflow1
   COMPLETED      Setup Hadoop Debugging        
   COMPLETED      Setup Hive                    
$ elastic-mapreduce --list --active
$ elastic-mapreduce --list --all
j-2P9MU1B0RWWW2     TERMINATED     ec2-176-34-10-240.ap-northeast-1.compute.amazonaws.comyasuflow1
   COMPLETED      Setup Hadoop Debugging        
   COMPLETED      Setup Hive