搜索[ATH事业群]结果-低调大师优秀个人博客

帆软FineBI5.1集群模式踩坑指南

帆软FineBI5.1集群模式踩坑指南安装之前，貌似没做任何准备，centos7.7 [hadoop@master ~]$ lsb_release -a LSB Version: :core-4.1-

2019-10-23

阿里开发者技术交流钉钉群汇总【2019】

垂直技术领域开发者 Java开发者技术群入群方式钉钉扫描下方二维码进群：云计算开发者技术群入群方式钉钉扫描下方二维码进群： Kubernetes开发者技术群入群方式钉钉扫描下方二维码进群： serverless

2019-09-04

3月21日，在2019阿里云峰会·北京上，阿里巴巴旗下的阿里云、支付宝、淘宝、钉钉、高德等联合发布“阿里巴巴小程序繁星计划”：提供20亿元补贴，扶持200万+小程序开发者、100万+商家。凡入选“超星”的小程序，入驻支付宝、淘宝、钉钉、高德后还能得到流量重点支持。阿里云同时发布小程序云，提供一站式云服务，为开发者提供稳定和便捷的后端云服务，包括serverless开发套件、应用托管服务、函数计算等。支付宝、高德、淘宝、钉钉及众多第三方上亿流量的小程序后端获统一标准。开发者可在这些小程序端上进行统一的应用发布、资源管理、数据管理。未来，阿里巴巴商业操作系统的业务能力也将在小程序云上逐步开放，帮助电商、金融、生活服务等各行业开发者和企业客户快速链接阿里巴巴经济体生态。阿里巴巴小程序云应用已经正式启动，已获得广大开发者的热烈关注，为了和

2019-03-31

Java后端技术交流群！学习技术期待你的加入！

专注Java技术,Spring全家桶、微服务、MySQL、MyCat、集群、分布式、中间件、Linux、多线程，群内有大牛解答。

2018-07-24

开源软件为中国软件提供了发展机遇--陆首群

陆首群与OSS领袖Kim-O"Reilly在一起社区和企业两种互相补充的开发机制为了加快中国Linux和开源软件的发展，我们认为，在开发机制方面，要形成社区开发和企业开发互为补充的两种机制

2017-11-15

项目管理经验谈——来自项目管理群的讨论

项目管理经验谈——来自项目管理群的讨论 max-pm-青岛说: 我今天讲的大纲项目管理 1、项目经理定位 2、能力定位及发展路线 3、项目经理管控范围及项目经理类型 4、如何才能培养出相对合格的项目经理

2017-11-07

Centos7.3部署Elasticsearch5.4.0集群

一、安装JDK 1.1下载安装 cd /usr/local/src tar -zxvf jdk-8u131-linux-x64.tar.gz mv jdk1.8.0_131 /usr/local/java 1.2修改配置文件 vim /etc/profile //最后面添加 export JAVA_HOME=/usr/local/java CLASSPATH=/usr/local/java/lib/dt.jar/usr/local/java/lib/tools.jar PATH=/usr/local/java/bin:$PATH export PATH JAVA_HOMECLASSPATH source /etc/profile //导入配置文件 java -version //查看java版本二、安装Elasticsearch 官网：https://www.elastic.co/downloads 参考：http://blog.csdn.net/u013365215/article/details/70159159 2.1下载安装 cd /usr/local/src/ curl -L -Ohttps://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.4.0.tar.gz tar -xvf elasticsearch-5.4.0.tar.gz mv elasticsearch-5.4.0 /usr/local/ cd /usr/local/elasticsearch-5.4.0/ 2.2系统调优 1）配置系统最大打开文件描述符数 vim /etc/sysctl.conf vm.max_map_count = 262144 执行以下命令生效 sysctl -p 2）配置进程最大打开文件描述符 vim /etc/security/limits.conf //文件最后 * soft nofile 65536 * hard nofile 65536 2.3编写ES Master节点配置文件 vim config/elasticsearch.yml cluster.name: my-es node.name:node-1 network.host:192.168.19.141 http.port: 9200 transport.tcp.port: 9300 discovery.zen.ping.unicast.hosts: ["192.168.19.141","192.168.19.142","192.168.19.143"] discovery.zen.minimum_master_nodes: 2 #避免出现跨域问题 http.cors.enabled: true http.cors.allow-origin: "*" 第二个、第三个节点的配置只需修改成对应的即可。 2.4启动ES 用root账号启动会报错：java.lang.RuntimeException: can not runelasticsearch as root 因为Elasticsearch5.0之后，不能使用root账户启动，我们先创建一个elasticsearch组和账户： groupadd elsearch useradd elsearch -g elsearch -p elasticsearch chown -R elsearch.elsearch /usr/local/elasticsearch-5.4.0/ 后台启动： su -elsearch -c "/usr/local/elasticsearch-5.4.0/bin/elasticsearch -d" 2.5安装head开源插件参考：http://blog.csdn.net/ronmy/article/details/63685254 只在master上安装插件即可。elasticsearch5不可以直接通过plugin -install mobz/elasticsearch-head安装，并且head需要在node环境下运行，具体步骤如下：第一步：安装node curl -sL -o /etc/yum.repos.d/khara-nodejs.repohttps://copr.fedoraproject.org/coprs/khara/nodejs/repo/epel-7/khara-nodejs-epel-7.repo yum install -y nodejs nodejs-npm 查看安装版本第二步：安装grunt cd /usr/lib/node_modules/npm/ npm install grunt-cli npm install grunt 查看版本： /usr/lib/node_modules/npm/node_modules/.bin/grunt -version 第三步：安装head yum install -y git cd /usr/local/ git clone git://github.com/mobz/elasticsearch-head.git cd elasticsearch-head/ npm install 注意：这步可以会有一些报错信息，可以忽略。 vim _site/app.js #修改『http://localhost:9200』字段到本机ES端口与IP 第四步：启动head并在后台运行 ./node_modules/grunt/bin/gruntserver & 浏览器访问：注意：若不能形成集群，可能是 iptables 或者 selinux 的原因 2.6开机自启 vim /etc/rc.local su - elsearch -c "/usr/local/elasticsearch-5.3.1/bin/elasticsearch -d" /usr/local/elasticsearch-head/node_modules/grunt/bin/grunt server & 保存退出后，给自启文件赋予执行权限（若不加，Centos7不会开机执行） chmod +x /etc/rc.d/rc.local 本文转自M四月天 51CTO博客，原文链接：http://blog.51cto.com/msiyuetian/1926325，如需转载请自行联系原作者

2017-10-30

“十三五”中国打造世界级智慧城市群

另外，长江中游城市群、中原城市群、扬子江城市群等词眼也出现在多位两会代表的议案中。从“一带一路”、京津冀协同发展、长江经济带建设，再到推动粤港澳大湾区城市群发展。

2017-07-04

微信 6.3.32 for Android发布群收款微信群聊可以收钱

近期版本微信群聊中可以按群成员和日期查找聊天内容，群主可启用需群主确认才可邀请朋友进群功能，群聊里也可以收钱了，聊天中视频的进度条，滑动可调整播放进度。全新特性！

2016-12-14

hadoop-2.6.4集群编译搭建-阿里云和腾讯云

腾讯云阿里云 hadoop集群编译搭建环境准备阿里云配置： [hadoop@lizer_ali ~]$ uname -a Linux lizer_ali 2.6.32-573.22.1.el6.x86_64 #1 SMP Wed Mar 23 03:35:39 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux [hadoop@lizer_ali ~]$ head -n 1 /etc/issue CentOS release 6.5 (Final) [hadoop@lizer_ali ~]$ cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c 1 Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz [hadoop@lizer_ali ~]$ getconf LONG_BIT 64 [hadoop@lizer_ali ~]$ cat /proc/meminfo MemTotal: 1018508 kB MemFree: 353912 kB 腾讯云配置： [hadoop@lizer_tx ~]$ uname -a Linux lizer_tx 2.6.32-573.18.1.el6.x86_64 #1 SMP Tue Feb 9 22:46:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux [hadoop@lizer_tx ~]$ head -n 1 /etc/issue CentOS release 6.7 (Final) [hadoop@lizer_tx ~]$ cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c 1 Intel(R) Xeon(R) CPU E5-26xx v3 [hadoop@lizer_tx ~]$ getconf LONG_BIT 64 [hadoop@lizer_tx ~]$ cat /proc/meminfo MemTotal: 1020224 kB MemFree: 688488 kB 创建用户 useradd hadoop passwd haddop jdk1.7安装：下载：http://www.oracle.com/technetwork/java/javase/downloads/java-archive-downloads-javase7-521261.html#jdk-7u80-oth-JPR wget http://download.oracle.com/otn/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz?AuthParam=1469844164_7ce09e1f99570835183215c3510e95e0 mv jdk-7u80-linux-x64.tar.gz\?AuthParam\=1469844164_7ce09e1f99570835183215c3510e95e0 jdk-7u80-linux-x64.tar.gz 配置jdk tar zxf jdk-7u80-linux-x64.tar.gz -C /opt/ 配置环境变量 vim /etc/profile export JAVA_HOME=/opt/jdk1.7.0_80 export JRE_HOME=/opt/jdk1.7.0_80/jre export CLASSPATH=./$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH 生效： source /etc/profile 编译hadoop2.6.4所需软件 yum install gcc cmake gcc-c++ 安装maven wget http://www-eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz 安装maven：http://www.blogjava.net/caojianhua/archive/2011/04/02/347559.html tar zxf apache-maven-3.3.9-bin.tar.gz -C /usr/local/ vim /etc/profile export MAVEN_HOME=/usr/local/apache-maven-3.3.9 export PATH=$PATH:$MAVEN_HOME/bin source /etc/profile [root@lizer_ali hadoop]# mvn -v Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00) Maven home: /usr/local/apache-maven-3.3.9 Java version: 1.7.0_80, vendor: Oracle Corporation Java home: /opt/jdk1.7.0_80/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "2.6.32-573.22.1.el6.x86_64", arch: "amd64", family: "unix" 安装protobuf 要求版本protobuf-2.5.0 wget https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gz cd protobuf-2.5.0/ ./configure -prefix=/usr/local/protobuf-2.5.0 make && make install vim /etc/profile export PROTOBUF=/usr/local/protobuf-2.5.0 export PATH=$PROTOBUF/bin:$PATH protoc --version 安装ant wget http://www-eu.apache.org/dist//ant/binaries/apache-ant-1.9.7-bin.tar.gz tar zxf apache-ant-1.9.7-bin.tar.gz -C /usr/local/ vim /etc/profile export ANT_HOME=/usr/local/apache-ant-1.9.7 export PATH=$PATH:$ANT_HOME/bin source /etc/profile ant -version Apache Ant(TM) version 1.9.7 compiled on April 9 2016 yum install autoconf automake libtool yum install openssl-devel 安装findbugs http://findbugs.sourceforge.net/downloads.html wget http://prdownloads.sourceforge.net/findbugs/findbugs-3.0.1.tar.gz?download mv findbugs-3.0.1.tar.gz\?download findbugs-3.0.1.tar.gz tar zxf findbugs-3.0.1.tar.gz -C /usr/local/ vim /etc/profile export FINDBUGS_HOME=/usr/local/findbugs-3.0.1 export PATH=$FINDBUGS_HOME/bin:$PATH findbugs -version hadoop编译安装：下载hadoop：http://hadoop.apache.org/releases.html wget http://www-eu.apache.org/dist/hadoop/common/hadoop-2.6.4/hadoop-2.6.4-src.tar.gz tar zxf hadoop-2.6.4-src.tar.gz cd hadoop-2.6.4-src more BUILDING.txt 查看如何编译安装 mvn clean package -Pdist,native,docs -DskipTests -Dtar 编译过程中，需要下载很多包，等待时间比较长。当看到hadoop各个项目都编译成功,即出现一系列的SUCCESS之后，即为编译成功。有些包下载卡住，重复执行上面的命令，或可以根据提示到相应的网址(https://repo.maven.apache.org/maven2)下载放到指定位置出现错误1： [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (site) on project hadoop-common: An Ant BuildException has occured: input file /home/hadoop/hadoop-2.6.4-src/hadoop-common-project/hadoop-common/target/findbugsXml.xml does not exist [ERROR] around Ant part ...<xslt style="/usr/local/findbugs-3.0.1/src/xsl/default.xsl" in="/home/hadoop/hadoop-2.6.4-src/hadoop-common-project/hadoop-common/target/findbugsXml.xml" out="/home/hadoop/hadoop-2.6.4-src/hadoop-common-project/hadoop-common/target/site/findbugs.html"/>... @ 44:256 in /home/hadoop/hadoop-2.6.4-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml 解决办法1：参考：http://www.itnose.net/detail/6143808.html 从该命令删除docs参数再运行mvn package -Pdist,native -DskipTests -Dtar 出现错误2： [INFO] Executing tasks main: [mkdir] Created dir: /home/hadoop/hadoop-2.6.4-src/hadoop-common-project/hadoop-kms/downloads [get] Getting: http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.41/bin/apache-tomcat-6.0.41.tar.gz [get] To: /home/hadoop/hadoop-2.6.4-src/hadoop-common-project/hadoop-kms/downloads/apache-tomcat-6.0.41.tar.gz 解决2：卡在这里，应该是不能下载下载上传到指定位置出现错误3： [ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:2.8.1:jar (module-javadocs) on project hadoop-hdfs: MavenReportException: Error while creating archive: [ERROR] ExcludePrivateAnnotationsStandardDoclet [ERROR] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000f31a4000, 130400256, 0) failed; error='Cannot allocate memory' (errno=12) [ERROR] # [ERROR] # There is insufficient memory for the Java Runtime Environment to continue. [ERROR] # Native memory allocation (malloc) failed to allocate 130400256 bytes for committing reserved memory. [ERROR] # An error report file with more information is saved as: [ERROR] # /home/hadoop/hadoop-2.6.4-src/hadoop-hdfs-project/hadoop-hdfs/target/hs_err_pid24729.log [ERROR] [ERROR] Error occurred during initialization of VM, try to reduce the Java heap size for the MAVEN_OPTS environnement variable using -Xms:<size> and -Xmx:<size>. [ERROR] Or, try to reduce the Java heap size for the Javadoc goal using -Dminmemory=<size> and -Dmaxmemory=<size>. 解决3：应该是内存不够，没有分配swap 添加2G swap分区添加或扩大交换分区 dd if=/dev/zero of=/home/swap bs=512 count=4096000 bs 是扇区大小 bs=512 指大小为512B count为扇区数量表示创建一个大小为4G 的文件 /home/swap 用空值填充。of位置可以自己调整。查看当前分区的大小 free -m 格式化并挂载 mkswap /home/swap swapon /home/swap 查看挂载情况 swapon -s 开机自动挂载 vim /etc/fstab /home/swap swap swap defaults 0 0 想写在分区 swapoff /home/swap 出现问题4： main: [mkdir] Created dir: /home/hadoop/hadoop-2.6.4-src/hadoop-hdfs-project/hadoop-hdfs-httpfs/downloads [get] Getting: http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.41/bin/apache-tomcat-6.0.41.tar.gz [get] To: /home/hadoop/hadoop-2.6.4-src/hadoop-hdfs-project/hadoop-hdfs-httpfs/downloads/apache-tomcat-6.0.41.tar.gz 解决4：网络问题，同上 cp /home/hadoop/hadoop-2.6.4-src/hadoop-common-project/hadoop-kms/downloads/apache-tomcat-6.0.41.tar.gz /home/hadoop/hadoop-2.6.4-src/hadoop-hdfs-project/hadoop-hdfs-httpfs/downloads/ 编译安装成功 [INFO] Apache Hadoop Gridmix .............................. SUCCESS [ 6.239 s] [INFO] Apache Hadoop Data Join ............................ SUCCESS [ 4.070 s] [INFO] Apache Hadoop Ant Tasks ............................ SUCCESS [ 3.304 s] [INFO] Apache Hadoop Extras ............................... SUCCESS [ 4.653 s] [INFO] Apache Hadoop Pipes ................................ SUCCESS [ 8.279 s] [INFO] Apache Hadoop OpenStack support .................... SUCCESS [ 7.736 s] [INFO] Apache Hadoop Amazon Web Services support .......... SUCCESS [06:22 min] [INFO] Apache Hadoop Client ............................... SUCCESS [ 9.608 s] [INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [ 0.258 s] [INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [ 6.721 s] [INFO] Apache Hadoop Tools Dist ........................... SUCCESS [ 15.171 s] [INFO] Apache Hadoop Tools ................................ SUCCESS [ 0.022 s] [INFO] Apache Hadoop Distribution ......................... SUCCESS [ 37.343 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 31:45 min [INFO] Finished at: 2016-07-30T23:59:50+08:00 [INFO] Final Memory: 101M/241M [INFO] ------------------------------------------------------------------------ 在hadoop-dist/target/已经生成了可执行文件拷贝到用户目录 cp -r hadoop-2.6.4 ~/ 配置两台机免密互通 ssh-keygen 各自生成的公钥放到另一台机上 cat id_rsa_else.pub >> authorized_keys chmod 600 authorized_keys 网络规划： hadoop1 123.206.33.182 slave hadoop0 114.215.92.77 master 配置hosts vim /etc/hosts 123.206.33.182 hadoop1 tx lizer_tx 114.215.92.77 hadoop0 ali lizer_ali 配置环境变量 vim /etc/profile export HADOOP_HOME=/home/hadoop/hadoop-2.6.4 export PATH=$HADOOP_HOME/bin:$PATH Hadoop配置配置文件放在$HADOOP_HOME/etc/hadoop/下修改一下配置： vim hadoop-env.sh export JAVA_HOME=/opt/jdk1.7.0_80 vim yarn-env.sh export JAVA_HOME=/opt/jdk1.7.0_80 vim slaves (这里没有了master配置文件) hadoop1 vim core-site.xml <configuration> <property> <name>hadoop.tmp.dir</name> <value>/home/hadoop/hadoop/tmp</value> <description>A base for other temporary directories.</description> </property> <property> <name>fs.defaultFS</name> <value>hdfs://hadoop0:9000</value> </property> <property> <name>io.file.buffer.size</name> <value>4096</value> </property> </configuration> vim hdfs-site.xml <configuration> <property> <name>dfs.http.address</name> <value>hadoop0:50070</value> </property> <property> <name>dfs.secondary.http.address</name> <value>hadoop0:50090</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>/home/hadoop/hadoop/name</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>/home/hadoop/hadoop/data</value> </property> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.nameservices</name> <value>hadoop0</value> </property> <property> <name>dfs.namenode.secondary.http-address</name> <value>hadoop0:50090</value> </property> <property> <name>dfs.webhdfs.enabled</name> <value>true</value> </property> <property> <name>dfs.permissions</name> <value>false</value> </property> </configuration> cp mapred-site.xml.template mapred-site.xml vim /etc/mapred-site.xml <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> <final>true</final> </property> <property> <name>mapreduce.jobtracker.http.address</name> <value>hadoop0:50030</value> </property> <property> <name>mapreduce.jobhistory.address</name> <value>hadoop0:10020</value> </property> <property> <name>mapreduce.jobhistory.webapp.address</name> <value>hadoop0:19888</value> </property> <property> <name>mapred.job.tracker</name> <value>hadoop0:9001</value> </property> </configuration> vim yarn-site.xml <configuration>  <property> <name>yarn.resourcemanager.hostname</name> <value>hadoop0</value> </property> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.resourcemanager.address</name> <value>hadoop0:8032</value> </property> <property> <name>yarn.resourcemanager.scheduler.address</name> <value>hadoop0:8030</value> </property> <property> <name>yarn.resourcemanager.resource-tracker.address</name> <value>hadoop0:8031</value> </property> <property> <name>yarn.resourcemanager.admin.address</name> <value>hadoop0:8033</value> </property> <property> <name>yarn.resourcemanager.webapp.address</name> <value>hadoop0:8088</value> </property> </configuration> vim master hadoop0 scp -r /home/hadoop/hadoop-2.6.4/etc/hadoop/* tx:~/hadoop-2.6.4/etc/hadoop/ 启动和关闭hadoop 参考：http://my.oschina.net/penngo/blog/653049 bin/hdfs namenode -format sbin/start-dfs.sh sbin/stop-dfs.sh sbin/start-yarn.sh sbin/stop-yarn.sh sbin/mr-jobhistory-daemon.sh start historyserver sbin/mr-jobhistory-daemon.sh stop historyserver sbin/hadoop-daemon.sh start secondarynamenode sbin/hadoop-daemon.sh stop secondarynamenode [hadoop@lizer_ali hadoop-2.6.4]$ jps 3099 ResourceManager 3430 SecondaryNameNode 2879 NameNode 3470 Jps 3382 JobHistoryServer [hadoop@lizer_tx ~]$ jps 9757 DataNode 9853 NodeManager 10064 Jps 检查节点配置情况 bin/hadoop dfsadmin -report 网页节点管理 http://114.215.92.77:8088/cluster 网页资源管理 http://114.215.92.77:50070/dfshealth.html#tab-overview 新建文件夹 bin/hdfs dfs -mkdir -p input 参考文档： http://hadoop.apache.org/docs/r2.6.4/hadoop-project-dist/hadoop-common/ClusterSetup.html http://hadoop.apache.org/docs/r2.6.4/hadoop-project-dist/hadoop-common/SingleCluster.html

2016-07-30

阿里云解决方案与产品事业部-移动云诚聘架构师/开发/产品人才啦！

你有超强的战斗力吗？你有不屈不挠的精神吗？你有足够的自信吗？如果你有，请看这里！阿里云移动云团队需要你！这里会给你国内顶尖的平台，给你足够的展示空间，给你热情优秀的队友！一起共建移动云，一起赋能企业和开发者！一、业务及平台介绍阿里云ApsaraMobile（移动云）是国内领先的移动开发平台（Mobile App Development Platform），致力于为企业、开发者提供一站式的移动应用开发环境，研发支撑基础设施以及高性能的移动端基础中间件、BaaS服务，帮助开发者削减人力、设备成本，压缩移动应用研发迭代周期，使开发者能够专注于自己核心业务的快速拓展。ApsaraMobile诚邀您携手同行，共同打造移动互联网的坚固基石移动云旗下的EMAS，是一个面向企业服务市场的移动应用研发平台，基于手机淘宝近十年移动互联网研发技术沉淀

2018-02-27