《2022年伪分布式模式安装hadoop单节 .pdf》由会员分享，可在线阅读，更多相关《2022年伪分布式模式安装hadoop单节 .pdf（5页珍藏版）》请在taowenge.com淘文阁网|工程机械CAD图纸|机械工程制图|CAD装配图下载|SolidWorks_CaTia_CAD_UG_PROE_设计图分享下载上搜索。
1、伪分布式模式安装hadoop单节点 - 伪分布式模式安装hadoop 单节点 -2.6.0 1、系统信息rootlsn-linux # cat /etc/issue CentOS release 6.3 (Final) Kernel r on an m rootlsn-linux # uname -a Linux lsn-linux 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux rootlsn hadoop-2.6.0# hostname lsn-linux ro
2、otlsn-linux # cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 :1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.160.3 lsn-linux rootlsn-linux # ll /lib64/libc.so.6 lrwxrwxrwx. 1 root root 12 Nov 11 22:35 /lib64/libc.so.6 -> l
3、ibc-2.12.so 2、准备工作2.1、jdk7.0 rootlsn hadoop-2.6.0# yum remove java rootlsn # rpm -ivh jdk-7u21-linux-x64.rpm Preparing. # 100% 1:jdk # 100% Unpacking JAR files. rt.jar. jsse.jar. charsets.jar. tools.jar. localedata.jar. 名师资料总结 - - -精品资料欢迎下载 - - - - - - - - - - - - - - - - - - 名师精心整理 - - - - - - - 第
4、1 页，共 5 页 - - - - - - - - - rootlsn # java -version java version "1.7.0_21" Java(TM) SE Runtime Environment (build 1.7.0_21-b11) Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode) 2.2、 /etc/profile配置，增加：export JAVA_HOME=/usr/java/jdk1.7.0_21 export HADOOP_PREFIX=/hadoop/hadoop
5、-2.6.0 export CLASSPATH=".:$JAVA_HOME/lib:$CLASSPATH" export PATH="$JAVA_HOME/bin:$PATH:$HADOOP_PREFIX/bin:$PATH" export HADOOP_PREFIX PATH CLASSPATH 2.3、ssh 免密码登陆设置（在root 目录完成）ssh-keygen -t rsa -P "" 然后一直回车即可cat /root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys
6、 2.4、tar zxf hadoop-2.6.0.tar.gz 文件位置/hadoop/hadoop-2.6.0 3、配置/hadoop/hadoop-2.6.0/etc/hadoop 3.1、vi hadoop-env.sh 增加export JAVA_HOME=/usr/java/jdk1.7.0_21 3.2、vi core-site.xml <configuration> <property> <name>hadoop.tmp.dir</name> <value>/hadoop/tmp</value> </
7、property> <property> <name>fs.default.name</name> <value>hdfs:/lsn-linux:9000</value> </property> 名师资料总结 - - -精品资料欢迎下载 - - - - - - - - - - - - - - - - - - 名师精心整理 - - - - - - - 第 2 页，共 5 页 - - - - - - - - - </configuration> 3.3、vi mapred-site.xml <confi
8、guration> <property> <name>mapred.job.tracker</name> <value>lsn-linux:9001</value> </property> </configuration> 3.4、vi yarn-site.xml <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> <p
9、roperty> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> 3.5、 vi hdfs-site.xml <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.namenode.n
10、ame.dir</name> <value>/hadoop/hadoop-2.6.0/dfs/name</value> </property> <property> <name>dfs.datannode.data.dir</name> <value>/hadoop/hadoop-2.6.0/dfs/data</value> </property> 4、启动bin/hadoop namenode -format sbin/start-all.s 名师资料总结 - - -精品资
11、料欢迎下载 - - - - - - - - - - - - - - - - - - 名师精心整理 - - - - - - - 第 3 页，共 5 页 - - - - - - - - - h 5、查看jps、bin/hadoop dfsadmin -report rootlsn hadoop-2.6.0# jps 6215 Jps 6037 ResourceManager 5623 NameNode 5743 DataNode 5898 SecondaryNameNode 6132 NodeManager rootlsn hadoop-2.6.0# bin/hadoop dfsadmin -re
12、port DEPRECATED: Use of this script to execute hdfs command is deprecated. Instead use the hdfs command for it. 14/12/04 19:40:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform. using builtin-java classes where applicable Configured Capacity: 29025366016 (27.03 GB
13、) Present Capacity: 24784756736 (23.08 GB) DFS Remaining: 24784707584 (23.08 GB) DFS Used: 49152 (48 KB) DFS Used%: 0.00% Under replicated blocks: 0 Blocks with corrupt replicas: 0 Missing blocks: 0 - Live datanodes (1): Name: 182.168.8.131:50010 (localhost) Hostname: localhost Decommission Status :
14、 Normal Configured Capacity: 29025366016 (27.03 GB) DFS Used: 49152 (48 KB) Non DFS Used: 4240609280 (3.95 GB) DFS Remaining: 24784707584 (23.08 GB) DFS Used%: 0.00% DFS Remaining%: 85.39% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) 名师资料总结 - - -精品资料欢迎下载 - - - - -
15、- - - - - - - - - - - - - 名师精心整理 - - - - - - - 第 4 页，共 5 页 - - - - - - - - - Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 1 Last contact: Thu Dec 04 19:40:21 CST 2014 问题：rootlsn-linux hadoop-2.6.0# hadoop fs -ls / 14/12/09 19:43:06 WARN util.NativeCodeLoader: Unable to load native-hadoop l
16、ibrary for your platform. using builtin-java classes where applicable rootlsn-linux hadoop-2.6.0# 原因是 hadoop-2.6.0.tar.gz 安装包是在32 位机器上编译的，64 位的机器加载本地库.so 文件时出错，不影响使用。解决：1、重新编译源码后将新的lib/native替换到集群中原来的lib/native 2、修改 hadoop-env.sh ，增加export HADOOP_OPTS="-Djava.library.path=$HADOOP_PREFIX/lib:$HADOOP_PREFIX/lib/native" 名师资料总结 - - -精品资料欢迎下载 - - - - - - - - - - - - - - - - - - 名师精心整理 - - - - - - - 第 5 页，共 5 页 - - - - - - - - -