FLASHC

Assessment: Hortonworks Certified Apache Hadoop Administrator 1.x
Date Completed: 2013/12/24
Result: Pass

聖誕禮物 Got

Hadoop 1.0 Administrator Certification
References for Certification Candidates
Intended Audience
The Certified Apache Hadoop Administrator certification is intended for IT administrators and operators who deploy, manage and monitor Hadoop-based solutions, consultants who create Hadoop project proposals and Hadoop administration instructors. Those certified are recognized as having high level of skill in Apache Hadoop administration

 
Hbase 啟動流程 
開啟一個 Hbase cluster 可以按造以下流程開啟

HDFS
DataNodes
HBase HMaster (active)
HBase HMaster (backup)
HBase Region Servers

note一下，

0.20 前使用 org.apache.hadoop.mapred 舊介面
0.20 版本開始引入org.apache.hadoop.mapreduce 的新API

MapFile 是 排序且帶索引的 hadoop SequenceFile 。
一個 MapFile 在 HDFS上是一個資料夾，包含兩個file組成，一個是index，也就是key的索引，另外一個就是 data，排序好的原始資料。
在查找時，只需要把index載入，memory中，使用binary search的方式，就可以很快查找到要找的key。
index
內含 
# hadoop fs -text numbers.map/index

IPC port
JobTracker:8021
namenode:8020

JobTracker WebUI : 50030

Tutorial - Apache Hive - Apache Software Foundation
https://cwiki.apache.org/confluence/display/Hive/Tutorial#Tutorial-Joins
Hive 的說明內只出了Joins的幾種組合用法，
其中有個是 left semi join
In order check the existence of a key in another table, the user can use LEFT SEMI JOIN as illustrated by the following example.
INSERT OVERWRITE TABLE pv_users
SELECT u.*
FROM user u LEFT SEMI JOIN page_view pv ON (pv.userid = u.id)
WHERE pv.date = '2008-03-03';

[hadoop] commission and decommission Step

[hadoop] job scheduler

[hadoop] intermediate Sort
keyword:spill index , spill files, MapReduce

Goal: Sort by key