Hadoop2.7实战v1.0之HDFS HA-低调大师

Hadoop2.7实战v1.0之HDFS HA

2016-03-05 779

HDFS HA实战v1.0

当前环境: hadoop+zookeeper(namenode,resourcemanager HA)

namenode	serviceId	Init status
sht-sgmhadoopnn-01	nn1	active
sht-sgmhadoopnn-02	nn2	standby

参考: http://blog.csdn.net/u011414200/article/details/50336735

一.查看namenode是active还是standby

1.打开网页

2.查看zkfc日志

点击(此处)折叠或打开

[root@sht-sgmhadoopnn-01 logs]# more hadoop-root-zkfc-sht-sgmhadoopnn-01.telenav.cn.log
…………………..
2016-02-28 00:24:00,692 INFO org.apache.hadoop.ha.ZKFailoverController: Trying to make NameNode at sht-sgmhadoopnn-01/172.16.101.55:8020 active...
2016-02-28 00:24:01,762 INFO org.apache.hadoop.ha.ZKFailoverController: Successfully transitioned NameNode at sht-sgmhadoopnn-01/172.16.101.55:8020 to active state
[root@sht-sgmhadoopnn-02 logs]# more hadoop-root-zkfc-sht-sgmhadoopnn-01.telenav.cn.log
…………………..
2016-02-28 00:24:01,186 INFO org.apache.hadoop.ha.ZKFailoverController: ZK Election indicated that NameNode at sht-sgmhadoopnn-02/172.16.101.56:8020 should become standby
2016-02-28 00:24:01,209 INFO org.apache.hadoop.ha.ZKFailoverController: Successfully transitioned NameNode at sht-sgmhadoopnn-02/172.16.101.56:8020 to standby state

3. 通过命令hdfs haadmin –getServiceState

###$HADOOP_HOME/etc/hadoop/hdfs-site.xml, dfs.ha.namenodes.[dfs.nameservices]

dfs.ha.namenodes.mycluster

nn1,nn2

点击(此处)折叠或打开

[root@sht-sgmhadoopnn-02 logs]# hdfs haadmin -getServiceState nn1
active
[root@sht-sgmhadoopnn-02 logs]# hdfs haadmin -getServiceState nn2
standby

二.基本命令

点击(此处)折叠或打开

[root@sht-sgmhadoopnn-02 logs]# hdfs --help

Usage: hdfs [--config confdir] [--loglevel loglevel] COMMAND

       where COMMAND is one of:

  dfs run a filesystem command on the file systems supported in Hadoop.

  classpath prints the classpath

  namenode -format format the DFS filesystem

  secondarynamenode run the DFS secondary namenode

  namenode run the DFS namenode

  journalnode run the DFS journalnode

  zkfc run the ZK Failover Controller daemon

  datanode run a DFS datanode

  dfsadmin run a DFS admin client

  haadmin run a DFS HA admin client

  fsck run a DFS filesystem checking utility

  balancer run a cluster balancing utility

  jmxget get JMX exported values from NameNode or DataNode.

  mover run a utility to move block replicas across

                               storage types

  oiv apply the offline fsimage viewer to an fsimage

  oiv_legacy apply the offline fsimage viewer to an legacy fsimage

  oev apply the offline edits viewer to an edits file

  fetchdt fetch a delegation token from the NameNode

  getconf get config values from configuration

  groups get the groups which users belong to

  snapshotDiff diff two snapshots of a directory or diff the

                       current directory contents with a snapshot

  lsSnapshottableDir list all snapshottable dirs owned by the current user

                                                Use -help to see options

  portmap run a portmap service

  nfs3 run an NFS version 3 gateway

  cacheadmin configure the HDFS cache

  crypto configure HDFS encryption zones

  storagepolicies list/get/set block storage policies

  version print the version

###########################################################################

[root@sht-sgmhadoopnn-02 logs]# hdfs namenode --help

Usage: java NameNode [-backup] |

        [-checkpoint] |

        [-format [-clusterid cid ] [-force] [-nonInteractive] ] |

        [-upgrade [-clusterid cid] [-renameReserved<k-v pairs>] ] |

        [-upgradeOnly [-clusterid cid] [-renameReserved<k-v pairs>] ] |

        [-rollback] |

        [-rollingUpgrade <rollback|downgrade|started> ] |

        [-finalize] |

        [-importCheckpoint] |

        [-initializeSharedEdits] |

        [-bootstrapStandby] |

        [-recover [ -force] ] |

        [-metadataVersion ] ]

###########################################################################

[root@sht-sgmhadoopnn-02 logs]# hdfs haadmin --help

-help: Unknown command

Usage: haadmin

    [-transitionToActive [--forceactive] <serviceId>]

    [-transitionToStandby <serviceId>]

    [-failover [--forcefence] [--forceactive] <serviceId> <serviceId>]

    [-getServiceState <serviceId>]

    [-checkHealth <serviceId>]

    [-help <command>]

transitionToActive 和

微信关注我们

原文链接：https://yq.aliyun.com/articles/578889

转载内容版权归作者及来源网站所有！

低调大师中文资讯倾力打造互联网数据资讯、行业资源、电子商务、移动互联网、网络营销平台。持续更新报道IT业界、互联网、市场资讯、驱动更新,是最及时权威的产业资讯及硬件资讯报道平台。

Hadoop2.7实战v1.0之添加DataNode节点后,更改文件复制策略dfs.replication

1.查看当前系统的复制策略dfs.replication为3,表示文件会备份成3份 a.通过查看hdfs-site.xml 文件点击(此处)折叠或打开 [root@sht-sgmhadoopnn-01 ~]# cd /hadoop/hadoop-2.7.2/etc/hadoop [root@sht-sgmhadoopnn-01 hadoop]# more hdfs-site.xml <property> <name>dfs.replication</name> <value>3</value> </property> b.通过查看当前hdfs文件的复制值是多少点击(此处)折叠或打开 [root@sht-sgmhadoopnn-01 hadoop]# hdfs dfs -ls /testdir Found 7 items -rw-r--r-- 3 root supergroup 37322672 2016-03-05 17:59 /testdir/012_HDFS.avi -rw-r--r-- 3 root su...

2016-03-06

693

YARN HA实战v1.0 当前环境:hadoop+zookeeper(namenode,resourcemanager HA) resourcemanager serviceId init status sht-sgmhadoopnn-01 rm1 active sht-sgmhadoopnn-02 rm2 standby 参考: http://blog.csdn.net/u011414200/article/details/50336735 http://blog.csdn.net/u011414200/article/details/50276257 一.查看resourcemanager是active还是standby 1.打开网页 http://172.16.101.55:8088/cluster/cluster http://172.16.101.56:8088/cluster/cluster 2.查看resourcemanager日志点击(此处)折叠或打开 [root@sht-sgmhadoopnn-01 logs]# more yarn-root-resource...

2016-03-06

651

资源下载

更多资源

Mario

马里奥是站在游戏界顶峰的超人气多面角色。马里奥靠吃蘑菇成长，特征是大鼻子、头戴帽子、身穿背带裤，还留着胡子。与他的双胞胎兄弟路易基一起，长年担任任天堂的招牌角色。

腾讯云软件源

为解决软件依赖安装时官方源访问速度慢的问题，腾讯云为一些软件搭建了缓存服务。您可以通过使用腾讯云软件源站来提升依赖包的安装速度。为了方便用户自由搭建服务架构，目前腾讯云软件源站支持公网访问和内网访问。

Nacos

Nacos /nɑ:kəʊs/ 是 Dynamic Naming and Configuration Service 的首字母简称，一个易于构建 AI Agent 应用的动态服务发现、配置管理和AI智能体管理平台。Nacos 致力于帮助您发现、配置和管理微服务及AI智能体应用。Nacos 提供了一组简单易用的特性集，帮助您快速实现动态服务发现、服务配置、服务元数据、流量管理。Nacos 帮助您更敏捷和容易地构建、交付和管理微服务平台。

Spring

Spring框架（Spring Framework）是由Rod Johnson于2002年提出的开源Java企业级应用框架，旨在通过使用JavaBean替代传统EJB实现方式降低企业级编程开发的复杂性。该框架基于简单性、可测试性和松耦合性设计理念，提供核心容器、应用上下文、数据访问集成等模块，支持整合Hibernate、Struts等第三方框架，其适用范围不仅限于服务器端开发，绝大多数Java应用均可从中受益。