首页 文章 精选 留言 我的

精选列表

搜索[快速入门]，共10000篇文章
优秀的个人博客,低调大师

OpenStack入门修炼之nova服务（控制节点）的部署与测试（10）

1.Nova介绍 Nova是openstack最早的两块模块之一，另一个是对象存储swift。在openstack体系中一个叫做计算节点，一个叫做控制节点。这个主要和nova相关，我们把安装为计算节点成为：nova-compute，把除了nova-compute叫做控制节点。nova-compute是创建虚拟机的，只是创建虚拟机，所有的控制都在另一台上。 OpenStack计算组件请求OpenStack Identity服务进行认证；请求OpenStack Image服务提供磁盘镜像；为OpenStack dashboard提供用户与管理员接口。 nova服务的功能和特点： 实例生命周期的管理 管理计算资源 网络和认证管理 REST风格的API 异步的一致性通信 Hypervisor透明：支持Xen，XenServer/XCP，KVM，UML，VMware vSphere and Hyper-V 2.OpenStack计算服务架构： Nova API：负责接收和响应外部请求。支持Openstack API，EC2 API。外部访问Nova的唯一途径，接受外部请求并通过Message Queue将请求发送给其他的服务组件。 Nova Scheduler：用于云主机调度，决策虚拟机创建应该创建在哪个计算节点上。决策一个虚拟机应该调度到某个物理节点上，需要两步：过滤，计算权值。 Nova Compute：一般运行在计算节点上，通过Message Queue接收并管理KVM的生命周期，Nova compute通过libvirt管理 KVM，通过XenAPI管理Xen。管理虚机的核心服务，通过调用 Hypervisor API 实现虚机生命周期管理。 Hypervisor ：计算节点上跑的虚拟化管理程序，虚机管理最底层的程序。 不同虚拟化技术提供自己的 Hypervisor。常用的 Hypervisor 有 KVM，Xen， VMWare 等。 Nova Conductor：计算节点访问数据库的中间件。nova-compute 经常需要更新数据库，比如更新虚机的状态。 出于安全性和伸缩性的考虑，nova-compute 并不会直接访问数据库，而是将这个任务委托给 nova-conductor。 Nova Consoleauth：用于控制台的授权验证，需要打开vnc需要在Consoleauth进行授权认证。负责对访问虚机控制台请亲提供 Token 认证。 Nova Novncporxy：提供一个代理，用于访问正在运行的实例。通过VNC协议，基于 Web 浏览器的 VNC 访问 。 Nova-spicehtml5proxy：基于 HTML5 浏览器的 SPICE 访问 Nova-xvpnvncproxy：基于 Java 客户端的 VNC 访问 Nova Cert：服务器守护进程向Nova Cert服务提供X509证书。用来为euca-bundle-image生成证书。仅仅是在EC2 API的请求中使用。 消息队列：在守护进程之间传递消息的中心。通常使用RabbitMQ实现，也可以使用另一个AMQP消息队列(如ZeroMQ)来实现。在前面我们了解到 Nova 包含众多的子服务，这些子服务之间需要相互协调和通信。 为解耦各个子服务，Nova 通过 Message Queue 作为子服务的信息中转站。 SQL数据库：Nova 会有一些数据需要存放到数据库中。存储云主机在构建时和运行时的状态，为云基础设施，包括有：可用实例类型、使用中的实例、可用网络、项目 3.计算节点的Nova服务部署与测试 （1）修改配置文件/etc/nova/nova.conf 在[DEFAULT]部分，只启用计算和元数据API： [DEFAULT] ... enabled_apis = osapi_compute,metadata 在[api_database]和[database]部分，配置数据库的连接： [api_database] ... connection = mysql+pymysql://nova:nova@192.168.56.11/nova_api [database] ... connection = mysql+pymysql://nova:nova@192.168.56.11/nova 在[DEFAULT]部分，配置RabbitMQ消息队列访问权限： [DEFAULT] ... transport_url = rabbit://openstack:openstack@192.168.56.11 在 “[DEFAULT]” 和 “[keystone_authtoken]” 部分，配置认证服务访问： [DEFAULT] ... auth_strategy = keystone [keystone_authtoken] ... auth_uri = http://192.168.56.11:5000 auth_url = http://192.168.56.11:35357 memcached_servers = 192.168.56.11:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = nova password = nova 在[DEFAULT]部分，启用网络服务支持： [DEFAULT] ... use_neutron = True firewall_driver = nova.virt.firewall.NoopFirewallDriver -->关闭nova防火墙 tips： 默认情况下，计算服务使用内置的防火墙服务。由于网络服务包含了防火墙服务，你必须使用nova.virt.firewall.NoopFirewallDriver防火墙服务来禁用掉计算服务内置的防火墙服务 在[vnc]部分，配置VNC代理使用控制节点的管理接口IP地址 ： [vnc] ... vncserver_listen = 0.0.0.0 vncserver_proxyclient_address = 192.168.56.11 在 [glance] 区域，配置镜像服务 API 的位置： [glance] ... api_servers = http://192.168.56.11:9292 在 [oslo_concurrency] 部分，配置锁路径： [oslo_concurrency] ... lock_path = /var/lib/nova/tmp 查看所有配置项： [root@linux-node1 ~]# grep "^[a-z]" /etc/nova/nova.conf auth_strategy=keystone #启用keystone进行认证 use_neutron=True #启用网络服务 enabled_apis=osapi_compute,metadata #启用计算和元数据 firewall_driver = nova.virt.firewall.NoopFirewallDriver transport_url=rabbit://openstack:openstack@192.168.56.11 connection=mysql+pymysql://nova:nova@192.168.56.11/nova_api connection=mysql+pymysql://nova:nova@192.168.56.11/nova api_servers=http://192.168.56.11:9292 auth_uri = http://192.168.56.11:5000 auth_url = http://192.168.56.11:35357 memcached_servers = 192.168.56.11:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = nova password = nova lock_path=/var/lib/nova/tmp vncserver_listen=0.0.0.0 vncserver_proxyclient_address=192.168.56.11 （2）同步数据库 [root@linux-node1 ~]# su -s /bin/sh -c "nova-manage api_db sync" nova [root@linux-node1 ~]# su -s /bin/sh -c "nova-manage db sync" nova 同步第二个操作时会有一个WARNING，属于正常。验证查看数据库是否有表 [root@linux-node1 ~]# mysql -h 192.168.56.11 -unova -pnova -e "use nova;show tables;" [root@linux-node1 ~]# mysql -h 192.168.56.11 -unova -pnova -e "use nova_api;show tables;" （3）完成安装，启动compute服务并设置开机自启 [root@linux-node1 ~]# systemctl enable openstack-nova-api.service openstack-nova-consoleauth.service openstack-nova-scheduler.service openstack-nova-conductor.service openstack-nova-novncproxy.service [root@linux-node1 ~]# systemctl start openstack-nova-api.service openstack-nova-consoleauth.service openstack-nova-scheduler.service openstack-nova-conductor.service openstack-nova-novncproxy.service （4）创建服务实体，并在keystone上注册 [root@linux-node1 ~]# openstack service create --name nova --description "OpenStack Compute" compute +-------------+----------------------------------+ | Field | Value | +-------------+----------------------------------+ | description | OpenStack Compute | | enabled | True | | id | 200e9f78a4654e0394eec200c4dab31d | | name | nova | | type | compute | +-------------+----------------------------------+ [root@linux-node1 ~]# openstack endpoint create --region RegionOne compute public http://192.168.56.11:8774/v2.1/%\(tenant_id\)s +--------------+----------------------------------------------+ | Field | Value | +--------------+----------------------------------------------+ | enabled | True | | id | dfb98d75fe7e44da898280d48e331a63 | | interface | public | | region | RegionOne | | region_id | RegionOne | | service_id | 200e9f78a4654e0394eec200c4dab31d | | service_name | nova | | service_type | compute | | url | http://192.168.56.11:8774/v2.1/%(tenant_id)s | +--------------+----------------------------------------------+ [root@linux-node1 ~]# openstack endpoint create --region RegionOne compute internal http://192.168.56.11:8774/v2.1/%\(tenant_id\)s +--------------+----------------------------------------------+ | Field | Value | +--------------+----------------------------------------------+ | enabled | True | | id | a656fdf0dce34db39fdce5c3fd3d3e40 | | interface | public | | region | RegionOne | | region_id | RegionOne | | service_id | 200e9f78a4654e0394eec200c4dab31d | | service_name | nova | | service_type | compute | | url | http://192.168.56.11:8774/v2.1/%(tenant_id)s | +--------------+----------------------------------------------+ [root@linux-node1 ~]# openstack endpoint create --region RegionOne compute admin http://192.168.56.11:8774/v2.1/%\(tenant_id\)s +--------------+----------------------------------------------+ | Field | Value | +--------------+----------------------------------------------+ | enabled | True | | id | 85a0a9b5d4db449cab48b7c033aabac3 | | interface | admin | | region | RegionOne | | region_id | RegionOne | | service_id | 200e9f78a4654e0394eec200c4dab31d | | service_name | nova | | service_type | compute | | url | http://192.168.56.11:8774/v2.1/%(tenant_id)s | +--------------+----------------------------------------------+ [root@linux-node1 ~]# openstack service list [root@linux-node1 ~]# openstack endpoint list （5）验证nova服务是否正常 [root@linux-node1 ~]# openstack host list +-------------+-------------+----------+ | Host Name | Service | Zone | +-------------+-------------+----------+ | linux-node1 | consoleauth | internal | | linux-node1 | conductor | internal | | linux-node1 | scheduler | internal | +-------------+-------------+----------+ [root@linux-node1 ~]# nova service-list +----+------------------+-------------+----------+---------+-------+----------------------------+-----------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | +----+------------------+-------------+----------+---------+-------+----------------------------+-----------------+ | 1 | nova-consoleauth | linux-node1 | internal | enabled | up | 2017-12-12T02:50:09.000000 | - | | 2 | nova-conductor | linux-node1 | internal | enabled | up | 2017-12-12T02:50:06.000000 | - | | 3 | nova-scheduler | linux-node1 | internal | enabled | up | 2017-12-12T02:50:05.000000 | - | +----+------------------+-------------+----------+---------+-------+----------------------------+-----------------+ 4.配置总结： ①数据库连接配置 ②RabbitMQ连接配置 ③Keystone认证配置 ④token缓存配置 ⑤glance-api访问配置 ⑥防火墙关闭、元数据访问、网络启用配置 ⑦锁路径以及VNC代理配置 5.Nova组件如何协调工作？ 对于 Nova，这些服务会部署在两类节点上：计算节点和控制节点。 计算节点上安装了 Hypervisor，上面运行虚拟机。 由此可知： 只有 nova-compute 需要放在计算节点上。 其他子服务则是放在控制节点上的。 通过在计算节点（node2）和控制节点（node1）上运行 ps -elf|grep nova 来查看运行的 nova 子服务 [root@linux-node1 ~]# ps -e |grep nova 1026 ? 00:15:02 nova-scheduler 1027 ? 00:13:52 nova-consoleaut 1028 ? 01:24:20 nova-conductor 1147 ? 00:02:41 nova-novncproxy 1564 ? 00:22:51 nova-compute 3240 ? 01:23:54 nova-api 3250 ? 00:02:20 nova-api 3252 ? 00:00:01 nova-api [root@linux-node2 ~]# ps -e |grep nova 1310 ? 00:28:50 nova-compute 从上面的查询结果上看，我们发现在在控制节点（node1）上也有一个nova-compute，这就意味着node1 既是一个控制节点，同时也是一个计算节点，也可以在上面运行虚机。我们可以通过nova service-list来查看 nova-* 子服务都分布在哪些节点上。 下面我们从虚拟机创建的流程上看nova的组件服务是如何工作的 ①客户向API（nova-api）发送请求：我需要创建一个虚拟机； ②API 对请求进行响应处理，向消息队列（RabbitMQ）发送了一条消息：“让 Scheduler 创建一个虚机”； ③Scheduler（nova-scheduler）监听消息队列获取到 API 发给它的消息，然后执行调度算法，从若干计算节点中选出合适的节点A； ④Scheduler 向 消息队列发送了一条消息：“在计算节点 A 上创建这个虚机”； ⑤计算节点 A 的 Compute（nova-compute）从消息队列中获取到 Scheduler 发给它的消息，然后在本节点的 Hypervisor 上启动虚机。 ⑥在虚机创建的过程中，Compute 如果需要查询或更新数据库信息，会通过消息队列向 Conductor（nova-conductor）发送消息，Conductor 负责数据库访问。 6.openstack设计思路（转）http://blog.csdn.net/cloudman6/article/details/51235388 （1）API 前端服务 OpenStack是一个组件复杂的平台，每个组件都包含这若干的子服务，其中每个服务中必定会有一个API负责接收外部请求。 以 Nova 为例，nova-api 是作为 Nova 组件对外的唯一窗口，向User暴露 Nova 能够提供的功能。 当User需要执行虚机相关的操作，能且只能向 nova-api 发送 REST 请求。 这里的客户包括终端用户、命令行和 OpenStack 其他组件。 设计 API 前端服务的好处在于： 对外提供统一接口，隐藏实现细节 API 提供 REST 标准调用服务，便于与第三方系统集成 可以通过运行多个 API 服务实例轻松实现 API 的高可用，比如运行多个 nova-api 进程 （2）Scheduler 调度服务 对于某项操作，如果有多个实体都能够完成任务，那么通常会有一个 scheduler 负责从这些实体中挑选出一个最合适的来执行操作。 在前面的例子中，Nova 有多个计算节点。 当需要创建虚机时，nova-scheduler 会根据计算节点当时的资源使用情况选择一个最合适的计算节点来运行虚机。 调度服务就好比是一个开发团队中的项目经理，当接到新的开发任务时，项目经理会评估任务的难度，考察团队成员目前的工作负荷和技能水平，然后将任务分配给最合适的开发人员。 （3）Worker 工作服务 调度服务只管分配任务，真正执行任务的是 Worker 工作服务。 在 Nova 中，这个 Worker 就是 nova-compute 了。 将 Scheduler 和 Worker 从职能上进行划分使得 OpenStack 非常容易扩展： 当计算资源不够了无法创建虚机时，可以增加计算节点（增加 Worker） 当客户的请求量太大调度不过来时，可以增加 Scheduler （4）Driver 框架 OpenStack 作为开放的 Infrastracture as a Service 云操作系统，支持业界各种优秀的技术。 这些技术可能是开源免费的，也可能是商业收费的。 这种开放的架构使得 OpenStack 能够在技术上保持先进性，具有很强的竞争力，同时又不会造成厂商锁定（Lock-in）。 那 OpenStack 的这种开放性体现在哪里呢？ 一个重要的方面就是采用基于 Driver 的框架。 以 Nova 为例，OpenStack 的计算节点支持多种 Hypervisor。 包括 KVM, Hyper-V, VMWare, Xen, Docker, LXC 等。 Nova-compute 为这些 Hypervisor 定义了统一的接口，hypervisor 只需要实现这些接口，就可以 driver 的形式即插即用到 OpenStack 中。 下面是 nova driver 的架构示意图 在 nova-compute 的配置文件 /etc/nova/nova.conf 中由 compute_driver 配置项指定该计算节点使用哪种 Hypervisor 的 driver 在我们的环境中因为是 KVM，所以配置的是 Libvirt 的 driver。 不知大家是否还记得我们在学习 Glance 时谈到： OpenStack 支持多种 backend 来存放 image。 可以是本地文件系统，Cinder，Ceph，Swift 等。 其实这也是一个 driver 架构。 只要符合 Glance 定义的规范，新的存储方式可以很方便的加入到 backend 支持列表中。 （5）消息队列服务 在前面创建虚机的流程示意图中，我们看到 nova-子服务之间的调用严重依赖 消息队列服务。 消息队列是 nova-子服务交互的中枢。 以前没接触过分布式系统的同学可能会不太理解为什么不让 API 直接调用Scheduler，或是让Scheuler 直接调用 Compute，而是非要通过 Messaging 进行中转。 这里做一些解释。 程序之间的调用通常分两种：同步调用和异步调用。 同步调用 API 直接调用 Scheduler 的接口就是同步调用。 其特点是 API 发出请求后需要一直等待，直到 Scheduler 完成对 Compute 的调度，将结果返回给 API 后 API 才能够继续做后面的工作。 异步调用 API 通过 Messaging 间接调用 Scheduler 就是异步调用。 其特点是 API 发出请求后不需要等待，直接返回，继续做后面的工作。 Scheduler 从 Messaging 接收到请求后执行调度操作，完成后将结果也通过 Messaging 发送给 API。 在 OpenStack 这类分布式系统中，通常采用异步调用的方式，其好处是： 解耦各子服务 子服务不需要知道其他服务在哪里运行，只需要发送消息给 Messaging 就能完成调用。 提高性能 异步调用使得调用者无需等待结果返回。这样可以继续执行更多的工作，提高系统总的吞吐量。 提高伸缩性 子服务可以根据需要进行扩展，启动更多的实例处理更多的请求，在提高可用性的同时也提高了整个系统的伸缩性。而且这种变化不会影响到其他子服务，也就是说变化对别人是透明的。 （6）Database OpenStack 各组件都需要维护自己的状态信息。 比如 Nova 中有虚机的规格、状态，这些信息都是在数据库中维护的。 每个 OpenStack 组件在 MySQL 中有自己的数据库。 本文转自 IT_外卖小哥 51CTO博客，原文链接:http://blog.51cto.com/jinlong/2049657
2017-11-11
优秀的个人博客,低调大师

OpenStack入门修炼之组件介绍及数据库、RabbitMQ配置（7）

1.OpenStack各组件介绍 Service Project name Description Dashboard Horizon 提供一个基于web的自助服务接口，用来和openstack交互。例如生产实例、分配IP地址和配置接入控制等 Compute Nova 管理计算实例的生命周期。功能主要是按需生成、调度、停止虚拟机 Networking Neutron 提供网络连接服务给其他组件，例如给compute提供网络服务；提供API让用户自己定义网络并使用 Block Storage Cinder 提供永久存储块给在运行中的实例 Identify service Keystone 给其他服务提供认证和授权服务 Image service Glance 存储虚拟机磁盘镜像，生成实例时调用glance中的镜像文件 2.数据库配置 OpenStack需要的基础服务： MySQL：OpenStack各个组件使用，生产环境中需要做集群 RabbitMQ：分布式消息队列，作为OpenStack各个组件的通信枢纽，支持集群 tips：除了Horizon，OpenStack其他组件都需要连接数据库 除了Horizon和Keystone，其他组件都需要连接RabbitMQ （1）数据库配置，并设置开机自启动 [root@linux-node1 ~]# vi /etc/my.cnf.d/openstack.cnf [mysqld] bind-address = 192.168.56.11 default-storage-engine = innodb innodb_file_per_table max_connections = 4096 collation-server = utf8_general_ci character-set-server = utf8 [root@linux-node1 ~]# systemctl enable mariadb.service [root@linux-node1 ~]# systemctl start mariadb.service （2）为了保证数据库服务的安全性，运行mysql_secure_installation脚本。特别需要说明的是，为数据库的root用户设置一个适当的密码。 [root@linux-node1 ~]# mysql_secure_installation NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY! In order to log into MariaDB to secure it, we'll need the current password for the root user. If you've just installed MariaDB, and you haven't set the root password yet, the password will be blank, so you should just press enter here. Enter current password for root (enter for none): OK, successfully used password, moving on... Setting the root password ensures that nobody can log into the MariaDB root user without the proper authorisation. Set root password? [Y/n] Y New password: 123456 Re-enter new password: 123456 Password updated successfully! Reloading privilege tables.. ... Success! 3.RabbitMQ配置 （1）设置rabbitmq开机自启动 [root@linux-node1 ~]# systemctl enable rabbitmq-server.service [root@linux-node1 ~]# systemctl start rabbitmq-server.service （2）添加openstack用户，并给openstack用户配置读写权限 [root@linux-node1 ~]# rabbitmqctl add_user openstack openstack Creating user "openstack" ... [root@linux-node1 ~]# rabbitmqctl set_permissions openstack ".*" ".*" ".*" Setting permissions for user "openstack" in vhost "/" ... （3）查看rabbitmq的插件，增加web页面的浏览插件 [root@linux-node1 ~]# rabbitmq-plugins list Configured: E = explicitly enabled; e = implicitly enabled | Status: * = running on rabbit@linux-node1 |/ [ ] amqp_client 3.6.5 [ ] cowboy 1.0.3 [ ] cowlib 1.0.1 [ ] mochiweb 2.13.1 [ ] rabbitmq_amqp1_0 3.6.5 [ ] rabbitmq_auth_backend_ldap 3.6.5 [ ] rabbitmq_auth_mechanism_ssl 3.6.5 [ ] rabbitmq_consistent_hash_exchange 3.6.5 [ ] rabbitmq_event_exchange 3.6.5 [ ] rabbitmq_federation 3.6.5 [ ] rabbitmq_federation_management 3.6.5 [ ] rabbitmq_jms_topic_exchange 3.6.5 [ ] rabbitmq_management 3.6.5 [ ] rabbitmq_management_agent 3.6.5 [ ] rabbitmq_management_visualiser 3.6.5 [ ] rabbitmq_mqtt 3.6.5 [ ] rabbitmq_recent_history_exchange 1.2.1 [ ] rabbitmq_sharding 0.1.0 [ ] rabbitmq_shovel 3.6.5 [ ] rabbitmq_shovel_management 3.6.5 [ ] rabbitmq_stomp 3.6.5 [ ] rabbitmq_top 3.6.5 [ ] rabbitmq_tracing 3.6.5 [ ] rabbitmq_trust_store 3.6.5 [ ] rabbitmq_web_dispatch 3.6.5 [ ] rabbitmq_web_stomp 3.6.5 [ ] rabbitmq_web_stomp_examples 3.6.5 [ ] sockjs 0.3.4 [ ] webmachine 1.10.3 启用rabbitmq的页面管理插件： [root@linux-node1 ~]# rabbitmq-plugins enable rabbitmq_management The following plugins have been enabled: mochiweb webmachine rabbitmq_web_dispatch amqp_client rabbitmq_management_agent rabbitmq_management Applying plugin configuration to rabbit@linux-node1... started 6 plugins. [root@linux-node1 ~]# lsof -i:15672 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME beam 21833 rabbitmq 50u IPv4 55754 0t0 TCP *:15672 (LISTEN) （4）使用浏览器访问192.168.56.11:15672，用户名密码：guest 本文转自 IT_外卖小哥 51CTO博客，原文链接:http://blog.51cto.com/jinlong/2049462
2017-11-11
优秀的个人博客,低调大师

OpenStack入门修炼之neutron服务（计算节点）的部署与测试（13）

1.Neutron组件安装 [root@linux-node2 ~]# yum install -y openstack-neutron-linuxbridge ebtables ipset 2.配置通用组件 编辑/etc/neutron/neutron.conf文件并完成如下操作： [root@linux-node2 ~]# vim /etc/neutron/neutron.conf 在[DEFAULT]部分，配置RabbitMQ消息队列访问权限： [DEFAULT] ... transport_url = rabbit://openstack:openstack@192.168.56.11 在 “[DEFAULT]” 和 “[keystone_authtoken]” 部分，配置认证服务访问： [DEFAULT] ... auth_strategy = keystone [keystone_authtoken] ... auth_uri = http://192.168.56.11:5000 auth_url = http://192.168.56.11:35357 memcached_servers = 192.168.56.11:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = neutron password = neutron 在 [oslo_concurrency] 部分，配置锁路径： [oslo_concurrency] ... lock_path = /var/lib/neutron/tmp 查看所有配置项： [root@linux-node2 ~]# grep "^[a-z]" /etc/neutron/neutron.conf auth_strategy = keystone transport_url = rabbit://openstack:openstack@192.168.56.11 auth_uri = http://192.168.56.11:5000 auth_url = http://192.168.56.11:35357 memcached_servers = 192.168.56.11:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = neutron password = neutron lock_path = /var/lib/neutron/tmp 3.配置linuxbridge代理 控制节点的/etc/neutron/plugins/ml2/linuxbridge_agent.ini配置文件和计算节点是一样的，可以使用scp拷贝过去 [root@linux-node1 ~]# scp /etc/neutron/plugins/ml2/linuxbridge_agent.ini root@192.168.56.12:/etc/neutron/plugins/ml2/ 4.配置计算服务来使用网络服务 编辑/etc/nova/nova.conf文件并完成下面的操作： 在[neutron]部分，配置访问参数： [neutron] ... url = http://192.168.56.11:9696 auth_url = http://192.168.56.11:35357 auth_type = password project_domain_name = default user_domain_name = default region_name = RegionOne project_name = service username = neutron password = neutron 5.完成安装 重启计算服务： [root@linux-node2 ~]# systemctl restart openstack-nova-compute.service 启动Linuxbridge代理并配置它开机自启动： [root@linux-node2 ~]# systemctl enable neutron-linuxbridge-agent.service [root@linux-node2 ~]# systemctl start neutron-linuxbridge-agent.service 6.在node1上查看是否成功启用 [root@linux-node1 ~]# neutron agent-list [root@linux-node1 ~]# nova service-list 这两个出来的结果正常，才可以正常创建虚拟机 7.底层网络变化 使用brctl show查看此时node2上的虚拟机网络结构如下：linux-node2作为计算节点，对应的tap设备为：tap31d0e21e-bb，并且连接到bridge本文转自 IT_外卖小哥 51CTO博客，原文链接:http://blog.51cto.com/jinlong/2049695
2017-11-11
优秀的个人博客,低调大师

OpenStack入门修炼之nova服务（计算节点）的部署与测试（11）

1.安装服务软件包 [root@linux-node2 ~]# yum install -y centos-release-openstack-newton [root@linux-node2 ~]# yum install -y openstack-nova-compute [root@linux-node2 ~]# yum install -y python-openstackclient [root@linux-node2 ~]# yum install -y openstack-selinux 2.修改配置文件 这里很多配置和node1上面的nova配置文件一样，所以我们先把node1上面的配置文件拷贝过来，然后再修改配置，具体执行过程如下： [root@linux-node1 ~]# scp /etc/nova/nova.conf 192.168.56.12:/etc/nova/ #注意权限，权限错误，服务会起不来 [root@linux-node2 ~]# vim /etc/nova/nova.conf 计算节点不需要使用数据库，此处删除数据库连接 connection=mysql+pymysql://nova:nova@192.168.56.11/nova_api connection=mysql+pymysql://nova:nova@192.168.56.11/nova 编辑/etc/nova/nova.conf文件并完成下面的操作： 在[vnc]部分，启用并配置远程控制台访问： [vnc] ... enabled = True vncserver_listen = 0.0.0.0 vncserver_proxyclient_address = 192.168.56.12 novncproxy_base_url = http://192.168.56.11:6080/vnc_auto.html 3.完成安装 （1）确定您的计算节点是否支持虚拟机的硬件加速。 [root@linux-node2 ~]# egrep -c '(vmx|svm)' /proc/cpuinfo 1 如果不支持，则显示为0 ，可以修改配置文件 1 #virt_type=kvm （2）启动计算服务及其依赖，并将其配置为随系统自动启动： [root@linux-node2 ~]# systemctl enable libvirtd.service openstack-nova-compute.service [root@linux-node2 ~]# systemctl start libvirtd.service openstack-nova-compute.service （3）在控制节点上（node1）查看 [root@linux-node1 ~]# source admin-openstack [root@linux-node1 ~]# nova service-list [root@linux-node1 ~]# openstack compute service list 版权声明：原创作品，谢绝转载。否则将追究法律责任 本文转自 IT_外卖小哥 51CTO博客，原文链接:http://blog.51cto.com/jinlong/2049661
2017-11-11
优秀的个人博客,低调大师

Storm编程入门API系列之Storm的定时任务实现

Storm的定时任务，分为两种实现方式，都是可以达到目的的。 我这里，分为StormTopologyTimer1.java 和 StormTopologyTimer2.java 编写代码StormTopologyTimer1.java 我这里，用的是shuffleGrouping方式。 //设置定时任务 config.put(Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS, 10);//表示storm每隔10秒都会给topology里面的所有bolt发送一个系统级别的tuple String topology_name = StormTopologyTimer1.class.getSimpleName(); package zhouls.bigdata.stormDemo; import java.util.Map; import org.apache.storm.Config; import org.apache.storm.Constants; import org.apache.storm.LocalCluster; import org.apache.storm.StormSubmitter; import org.apache.storm.generated.AlreadyAliveException; import org.apache.storm.generated.AuthorizationException; import org.apache.storm.generated.InvalidTopologyException; import org.apache.storm.spout.SpoutOutputCollector; import org.apache.storm.task.OutputCollector; import org.apache.storm.task.TopologyContext; import org.apache.storm.topology.OutputFieldsDeclarer; import org.apache.storm.topology.TopologyBuilder; import org.apache.storm.topology.base.BaseRichBolt; import org.apache.storm.topology.base.BaseRichSpout; import org.apache.storm.tuple.Fields; import org.apache.storm.tuple.Tuple; import org.apache.storm.tuple.Values; import org.apache.storm.utils.Utils; public class StormTopologyTimer1 { public static class MySpout extends BaseRichSpout{ private Map conf; private TopologyContext context; private SpoutOutputCollector collector; public void open(Map conf, TopologyContext context, SpoutOutputCollector collector) { this.conf = conf; this.collector = collector; this.context = context; } int num = 0; public void nextTuple() { num++; System.out.println("spout:"+num); this.collector.emit(new Values(num)); Utils.sleep(1000); } public void declareOutputFields(OutputFieldsDeclarer declarer) { declarer.declare(new Fields("num")); } } public static class MyBolt extends BaseRichBolt{ private Map stormConf; private TopologyContext context; private OutputCollector collector; public void prepare(Map stormConf, TopologyContext context, OutputCollector collector) { this.stormConf = stormConf; this.context = context; this.collector = collector; } int sum = 0; public void execute(Tuple input) { if(input.getSourceComponent().equals(Constants.SYSTEM_COMPONENT_ID)){ System.out.println("定时时间到了"); }else{ Integer num = input.getIntegerByField("num"); sum += num; System.out.println("sum="+sum); } } public void declareOutputFields(OutputFieldsDeclarer declarer) { } } public static void main(String[] args) { TopologyBuilder topologyBuilder = new TopologyBuilder(); String spout_id = MySpout.class.getSimpleName(); String bolt_id = MyBolt.class.getSimpleName(); topologyBuilder.setSpout(spout_id, new MySpout()); topologyBuilder.setBolt(bolt_id, new MyBolt()).shuffleGrouping(spout_id); Config config = new Config(); //设置定时任务 config.put(Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS, 10);//表示storm每隔10秒都会给topology里面的所有bolt发送一个系统级别的tuple String topology_name = StormTopologyTimer1.class.getSimpleName(); if(args.length==0){ //在本地运行 LocalCluster localCluster = new LocalCluster(); localCluster.submitTopology(topology_name, config, topologyBuilder.createTopology()); }else{ //在集群运行 try { StormSubmitter.submitTopology(topology_name, config, topologyBuilder.createTopology()); } catch (AlreadyAliveException e) { e.printStackTrace(); } catch (InvalidTopologyException e) { e.printStackTrace(); } catch (AuthorizationException e) { e.printStackTrace(); } } } } 停掉，我们复制粘贴来分析分析 64555 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 64578 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:host.name=WIN-BQOBV63OBNM 64578 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:java.version=1.8.0_66 64578 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:java.vendor=Oracle Corporation 64578 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:java.home=C:\Program Files\Java\jre1.8.0_66 64578 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:java.class.path=D:\Code\eclipseMarsCode\stormDemo\target\classes;D:\SoftWare\maven\repository\org\apache\storm\storm-core\1.0.2\storm-core-1.0.2.jar;D:\SoftWare\maven\repository\com\esotericsoftware\kryo\3.0.3\kryo-3.0.3.jar;D:\SoftWare\maven\repository\com\esotericsoftware\reflectasm\1.10.1\reflectasm-1.10.1.jar;D:\SoftWare\maven\repository\org\ow2\asm\asm\5.0.3\asm-5.0.3.jar;D:\SoftWare\maven\repository\com\esotericsoftware\minlog\1.3.0\minlog-1.3.0.jar;D:\SoftWare\maven\repository\org\objenesis\objenesis\2.1\objenesis-2.1.jar;D:\SoftWare\maven\repository\org\clojure\clojure\1.7.0\clojure-1.7.0.jar;D:\SoftWare\maven\repository\com\lmax\disruptor\3.3.2\disruptor-3.3.2.jar;D:\SoftWare\maven\repository\org\apache\logging\log4j\log4j-api\2.1\log4j-api-2.1.jar;D:\SoftWare\maven\repository\org\apache\logging\log4j\log4j-core\2.1\log4j-core-2.1.jar;D:\SoftWare\maven\repository\org\apache\logging\log4j\log4j-slf4j-impl\2.1\log4j-slf4j-impl-2.1.jar;D:\SoftWare\maven\repository\org\slf4j\log4j-over-slf4j\1.6.6\log4j-over-slf4j-1.6.6.jar;D:\SoftWare\maven\repository\javax\servlet\servlet-api\2.5\servlet-api-2.5.jar;D:\SoftWare\maven\repository\org\slf4j\slf4j-api\1.7.7\slf4j-api-1.7.7.jar 64579 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:java.library.path=C:\Program Files\Java\jre1.8.0_66\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:/Program Files/Java/jre1.8.0_66/bin/server;C:/Program Files/Java/jre1.8.0_66/bin;C:/Program Files/Java/jre1.8.0_66/lib/amd64;%WEKA39_HOME%\lib\mysql-connector-java-5.1.21-bin.jar;%WEKA37_HOME%\lib\mysql-connector-java-5.1.21-bin.jar;C:\Program Files\Java\jdk1.8.0_66\jre\lib\ext\mysql-connector-java-5.1.21-bin.jar;C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;D:\SoftWare\MATLAB R2013a\runtime\win64;D:\SoftWare\MATLAB R2013a\bin;C:\Program Files (x86)\IDM Computer Solutions\UltraCompare;C:\Program Files\Java\jdk1.8.0_66\bin;C:\Program Files\Java\jdk1.8.0_66\jre\bin;D:\SoftWare\apache-ant-1.9.0\bin;HADOOP_HOME\bin;D:\SoftWare\apache-maven-3.3.9\bin;D:\SoftWare\Scala\bin;D:\SoftWare\Scala\jre\bin;%MYSQL_HOME\bin;D:\SoftWare\MySQL\mysql-5.7.11-winx64;;D:\SoftWare\apache-tomcat-7.0.69\bin;%C:\Windows\System32;%C:\Windows\SysWOW64;;D:\SoftWare\apache-maven-3.3.9\bin;D:\SoftWare\apache-tomcat-7.0.69\bin;D:\SoftWare\apache-tomcat-7.0.69\bin;D:\SoftWare\Anaconda2;D:\SoftWare\Anaconda2\Scripts;D:\SoftWare\Anaconda2\Library\bin;D:\SoftWare\MySQL Server\MySQL Server 5.0\bin;D:\SoftWare\Python\Python36\Scripts\;D:\SoftWare\Python\Python36\;D:\SoftWare\SSH Secure Shell;D:\SoftWare\eclipse;;. 64579 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:java.io.tmpdir=C:\Users\ADMINI~1\AppData\Local\Temp\ 64579 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:java.compiler=<NA> 64579 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:os.name=Windows 7 64579 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:os.arch=amd64 64580 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:os.version=6.1 64580 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:user.name=Administrator 64581 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:user.home=C:\Users\Administrator 64582 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:user.dir=D:\Code\eclipseMarsCode\stormDemo 64689 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir C:\Users\ADMINI~1\AppData\Local\Temp\32d8c475-84fc-4fb3-baff-b04ab7a8dc5c\version-2 snapdir C:\Users\ADMINI~1\AppData\Local\Temp\32d8c475-84fc-4fb3-baff-b04ab7a8dc5c\version-2 65172 [main] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - binding to port 0.0.0.0/0.0.0.0:2000 65178 [main] INFO o.a.s.zookeeper - Starting inprocess zookeeper at port 2000 and dir C:\Users\ADMINI~1\AppData\Local\Temp\32d8c475-84fc-4fb3-baff-b04ab7a8dc5c 65621 [main] INFO o.a.s.d.nimbus - Starting Nimbus with conf {"topology.builtin.metrics.bucket.size.secs" 60, "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" 1, "logviewer.max.per.worker.logs.size.mb" 2048, "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" 29, "zmq.threads" 1, "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" 1, "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" 60, "drpc.invocations.port" 3773, "supervisor.localizer.cache.target.size.mb" 10240, "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" 1, "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" 5, "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" 0, "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" 1, "pacemaker.host" "localhost", "storm.zookeeper.retry.times" 5, "ui.actions.enabled" true, "zmq.linger.millis" 0, "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" 100, "worker.log.level.reset.poll.secs" 30, "storm.zookeeper.port" 2000, "supervisor.heartbeat.frequency.secs" 5, "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" 64, "supervisor.blobstore.download.thread.count" 5, "drpc.queue.size" 128, "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" 65536, "topology.shellbolt.max.pending" 100, "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" 120, "logviewer.port" 8000, "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" 1024, "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" 2000, "nimbus.inbox.jar.expiration.secs" 3600, "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" 0, "supervisor.localizer.cleanup.interval.ms" 600000, "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" 64, "logviewer.cleanup.age.mins" 10080, "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" 3, "nimbus.credential.renewers.freq.secs" 600, "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" 120, "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" 10, "storm.health.check.timeout.ms" 5000, "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" 500, "topology.workers" 1, "pacemaker.base.threads" 10, "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\fcc1cd09-238a-499f-967b-da2e3564a331", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" 1, "topology.message.timeout.secs" 30, "topology.state.synchronization.timeout.secs" 60, "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" 600, "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" 100, "storm.messaging.netty.max_retries" 300, "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" 20000, "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" 15000, "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" 60, "nimbus.task.timeout.secs" 30, "drpc.port" 3772, "pacemaker.max.threads" 50, "storm.zookeeper.retry.intervalceiling.millis" 30000, "nimbus.thrift.port" 6627, "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" 5, "supervisor.worker.start.timeout.secs" 120, "storm.zookeeper.retry.interval" 1000, "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" 1048576, "pacemaker.thread.timeout" 10, "task.credentials.poll.secs" 30, "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" 1, "topology.state.checkpoint.interval.ms" 1000, "supervisor.slots.ports" [6700 6701 6702 6703], "topology.transfer.buffer.size" 1024, "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" 4, "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" 600, "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" 1024, "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" 120, "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" 5242880, "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" 1, "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -1, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" 3, "task.refresh.poll.secs" 10, "storm.exhibitor.port" 8080, "task.heartbeat.frequency.secs" 3, "pacemaker.port" 6699, "storm.messaging.netty.max_wait_ms" 1000, "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" 3774, "topology.error.throttle.interval.secs" 10, "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" 100000, "nimbus.cleanup.inbox.freq.secs" 600, "storm.blobstore.replication.factor" 3, "worker.heap.memory.mb" 768, "logviewer.max.sum.worker.logs.size.mb" 4096, "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" 30, "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" 4096, "topology.min.replication.count" 1, "topology.disruptor.wait.timeout.millis" 1000, "storm.nimbus.retry.intervalceiling.millis" 60000, "topology.trident.batch.emit.interval.millis" 50, "storm.auth.simple-acl.users" [], "drpc.invocations.threads" 64, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" 8080, "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" 262144, "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" 1048576, "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" 600} 66892 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 66958 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 66959 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:host.name=WIN-BQOBV63OBNM 66959 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:java.version=1.8.0_66 66964 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:java.vendor=Oracle Corporation 66965 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:java.home=C:\Program Files\Java\jre1.8.0_66 66966 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:java.class.path=D:\Code\eclipseMarsCode\stormDemo\target\classes;D:\SoftWare\maven\repository\org\apache\storm\storm-core\1.0.2\storm-core-1.0.2.jar;D:\SoftWare\maven\repository\com\esotericsoftware\kryo\3.0.3\kryo-3.0.3.jar;D:\SoftWare\maven\repository\com\esotericsoftware\reflectasm\1.10.1\reflectasm-1.10.1.jar;D:\SoftWare\maven\repository\org\ow2\asm\asm\5.0.3\asm-5.0.3.jar;D:\SoftWare\maven\repository\com\esotericsoftware\minlog\1.3.0\minlog-1.3.0.jar;D:\SoftWare\maven\repository\org\objenesis\objenesis\2.1\objenesis-2.1.jar;D:\SoftWare\maven\repository\org\clojure\clojure\1.7.0\clojure-1.7.0.jar;D:\SoftWare\maven\repository\com\lmax\disruptor\3.3.2\disruptor-3.3.2.jar;D:\SoftWare\maven\repository\org\apache\logging\log4j\log4j-api\2.1\log4j-api-2.1.jar;D:\SoftWare\maven\repository\org\apache\logging\log4j\log4j-core\2.1\log4j-core-2.1.jar;D:\SoftWare\maven\repository\org\apache\logging\log4j\log4j-slf4j-impl\2.1\log4j-slf4j-impl-2.1.jar;D:\SoftWare\maven\repository\org\slf4j\log4j-over-slf4j\1.6.6\log4j-over-slf4j-1.6.6.jar;D:\SoftWare\maven\repository\javax\servlet\servlet-api\2.5\servlet-api-2.5.jar;D:\SoftWare\maven\repository\org\slf4j\slf4j-api\1.7.7\slf4j-api-1.7.7.jar 66966 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:java.library.path=C:\Program Files\Java\jre1.8.0_66\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:/Program Files/Java/jre1.8.0_66/bin/server;C:/Program Files/Java/jre1.8.0_66/bin;C:/Program Files/Java/jre1.8.0_66/lib/amd64;%WEKA39_HOME%\lib\mysql-connector-java-5.1.21-bin.jar;%WEKA37_HOME%\lib\mysql-connector-java-5.1.21-bin.jar;C:\Program Files\Java\jdk1.8.0_66\jre\lib\ext\mysql-connector-java-5.1.21-bin.jar;C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;D:\SoftWare\MATLAB R2013a\runtime\win64;D:\SoftWare\MATLAB R2013a\bin;C:\Program Files (x86)\IDM Computer Solutions\UltraCompare;C:\Program Files\Java\jdk1.8.0_66\bin;C:\Program Files\Java\jdk1.8.0_66\jre\bin;D:\SoftWare\apache-ant-1.9.0\bin;HADOOP_HOME\bin;D:\SoftWare\apache-maven-3.3.9\bin;D:\SoftWare\Scala\bin;D:\SoftWare\Scala\jre\bin;%MYSQL_HOME\bin;D:\SoftWare\MySQL\mysql-5.7.11-winx64;;D:\SoftWare\apache-tomcat-7.0.69\bin;%C:\Windows\System32;%C:\Windows\SysWOW64;;D:\SoftWare\apache-maven-3.3.9\bin;D:\SoftWare\apache-tomcat-7.0.69\bin;D:\SoftWare\apache-tomcat-7.0.69\bin;D:\SoftWare\Anaconda2;D:\SoftWare\Anaconda2\Scripts;D:\SoftWare\Anaconda2\Library\bin;D:\SoftWare\MySQL Server\MySQL Server 5.0\bin;D:\SoftWare\Python\Python36\Scripts\;D:\SoftWare\Python\Python36\;D:\SoftWare\SSH Secure Shell;D:\SoftWare\eclipse;;. 66966 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:java.io.tmpdir=C:\Users\ADMINI~1\AppData\Local\Temp\ 66966 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:java.compiler=<NA> 66967 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:os.name=Windows 7 66967 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:os.arch=amd64 66967 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:os.version=6.1 66967 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:user.name=Administrator 66967 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:user.home=C:\Users\Administrator 66968 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:user.dir=D:\Code\eclipseMarsCode\stormDemo 66971 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@3330f3ad 67148 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 67155 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52829 67162 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 67277 [main] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in C:\Users\ADMINI~1\AppData\Local\Temp\fcc1cd09-238a-499f-967b-da2e3564a331\blobs 67287 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52829 67367 [main] INFO o.a.s.d.nimbus - Using default scheduler 67377 [SyncThread:0] INFO o.a.s.s.o.a.z.s.p.FileTxnLog - Creating new log file: log.1 67390 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 67398 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@32f96bba 67461 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 67465 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 67465 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52832 67466 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52832 67690 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 67693 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@3a2e9f5b 67745 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 67747 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52835 67749 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 67750 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52835 67755 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f52580000 with negotiated timeout 20000 for client /127.0.0.1:52829 67757 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f52580000, negotiated timeout = 20000 67761 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f52580001 with negotiated timeout 20000 for client /127.0.0.1:52832 67762 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f52580001, negotiated timeout = 20000 67829 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f52580002 with negotiated timeout 20000 for client /127.0.0.1:52835 67830 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f52580002, negotiated timeout = 20000 67903 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 67903 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 67903 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 67925 [main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none 67926 [main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none 68122 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting 68147 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d880f52580002 68190 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d880f52580002 closed 68190 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down 68232 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:52835 which had sessionid 0x15d880f52580002 68255 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 68256 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@3e1fd62b 68265 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 68268 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 68268 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52838 68269 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52838 68269 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 68271 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@40de8f93 68281 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 68283 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 68284 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52841 68284 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52841 68313 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f52580003, negotiated timeout = 20000 68313 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f52580003 with negotiated timeout 20000 for client /127.0.0.1:52838 68317 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 68364 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f52580004, negotiated timeout = 20000 68368 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f52580004 with negotiated timeout 20000 for client /127.0.0.1:52841 68372 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 69287 [main] INFO o.a.s.zookeeper - Queued up for leader lock. 69495 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x15d880f52580001 type:create cxid:0x1 zxid:0x12 txntype:-1 reqpath:n/a Error Path:/storm/leader-lock Error:KeeperErrorCode = NoNode for /storm/leader-lock 69563 [Curator-Framework-0] WARN o.a.s.s.o.a.c.u.ZKPaths - The version of ZooKeeper being used doesn't support Container nodes. CreateMode.PERSISTENT will be used instead. 69743 [main] INFO o.a.s.d.m.MetricsUtils - Using statistics reporter plugin:org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter 69746 [main] INFO o.a.s.d.m.r.JmxPreparableReporter - Preparing... 69951 [main] INFO o.a.s.d.common - Started statistics report plugin... 69976 [main-EventThread] INFO o.a.s.zookeeper - WIN-BQOBV63OBNM gained leadership, checking if it has all the topology code locally. 70045 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 70046 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@889a8a8 70064 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 70066 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 70066 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52844 70072 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52844 70123 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f52580005 with negotiated timeout 20000 for client /127.0.0.1:52844 70124 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f52580005, negotiated timeout = 20000 70124 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 70125 [main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none 70131 [main-EventThread] INFO o.a.s.zookeeper - active-topology-ids [] local-topology-ids [] diff-topology [] 70132 [main-EventThread] INFO o.a.s.zookeeper - Accepting leadership, all active topology found localy. 70134 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting 70137 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d880f52580005 70197 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d880f52580005 closed 70198 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down 70197 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:52844 which had sessionid 0x15d880f52580005 70202 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 70207 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@d919544 70217 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 70218 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@4eea94a4 70226 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 70231 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 70232 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52849 70232 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52849 70239 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 70241 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 70241 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52850 70242 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52850 70275 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f52580006 with negotiated timeout 20000 for client /127.0.0.1:52849 70275 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f52580006, negotiated timeout = 20000 70277 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 70301 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f52580007 with negotiated timeout 20000 for client /127.0.0.1:52850 70302 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f52580007, negotiated timeout = 20000 70303 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 70304 [main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none 70308 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting 70311 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d880f52580007 70329 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d880f52580007 closed 70329 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:52850 which had sessionid 0x15d880f52580007 70329 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down 70331 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 70335 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@f8a6243 70345 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 70347 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52853 70348 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 70349 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52853 70373 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f52580008 with negotiated timeout 20000 for client /127.0.0.1:52853 70373 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f52580008, negotiated timeout = 20000 70374 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 70427 [main] INFO o.a.s.d.supervisor - Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" 60, "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" 1, "logviewer.max.per.worker.logs.size.mb" 2048, "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" 29, "zmq.threads" 1, "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" 1, "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" 60, "drpc.invocations.port" 3773, "supervisor.localizer.cache.target.size.mb" 10240, "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" 1, "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" 5, "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" 0, "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" 1, "pacemaker.host" "localhost", "storm.zookeeper.retry.times" 5, "ui.actions.enabled" true, "zmq.linger.millis" 0, "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" 100, "worker.log.level.reset.poll.secs" 30, "storm.zookeeper.port" 2000, "supervisor.heartbeat.frequency.secs" 5, "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" 64, "supervisor.blobstore.download.thread.count" 5, "drpc.queue.size" 128, "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" 65536, "topology.shellbolt.max.pending" 100, "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" 120, "logviewer.port" 8000, "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" 1024, "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" 2000, "nimbus.inbox.jar.expiration.secs" 3600, "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" 0, "supervisor.localizer.cleanup.interval.ms" 600000, "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" 64, "logviewer.cleanup.age.mins" 10080, "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" 3, "nimbus.credential.renewers.freq.secs" 600, "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" 120, "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" 10, "storm.health.check.timeout.ms" 5000, "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" 500, "topology.workers" 1, "pacemaker.base.threads" 10, "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\ee27d0b2-acb3-4e6d-bc2b-fccd2bd03438", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" 1, "topology.message.timeout.secs" 30, "topology.state.synchronization.timeout.secs" 60, "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" 600, "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" 100, "storm.messaging.netty.max_retries" 300, "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" 20000, "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" 15000, "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" 60, "nimbus.task.timeout.secs" 30, "drpc.port" 3772, "pacemaker.max.threads" 50, "storm.zookeeper.retry.intervalceiling.millis" 30000, "nimbus.thrift.port" 6627, "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" 5, "supervisor.worker.start.timeout.secs" 120, "storm.zookeeper.retry.interval" 1000, "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" 1048576, "pacemaker.thread.timeout" 10, "task.credentials.poll.secs" 30, "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" 1, "topology.state.checkpoint.interval.ms" 1000, "supervisor.slots.ports" (1024 1025 1026), "topology.transfer.buffer.size" 1024, "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" 4, "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" 600, "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" 1024, "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" 120, "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" 5242880, "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" 1, "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -1, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" 3, "task.refresh.poll.secs" 10, "storm.exhibitor.port" 8080, "task.heartbeat.frequency.secs" 3, "pacemaker.port" 6699, "storm.messaging.netty.max_wait_ms" 1000, "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" 3774, "topology.error.throttle.interval.secs" 10, "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" 100000, "nimbus.cleanup.inbox.freq.secs" 600, "storm.blobstore.replication.factor" 3, "worker.heap.memory.mb" 768, "logviewer.max.sum.worker.logs.size.mb" 4096, "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" 30, "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" 4096, "topology.min.replication.count" 1, "topology.disruptor.wait.timeout.millis" 1000, "storm.nimbus.retry.intervalceiling.millis" 60000, "topology.trident.batch.emit.interval.millis" 50, "storm.auth.simple-acl.users" [], "drpc.invocations.threads" 64, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" 8080, "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" 262144, "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" 1048576, "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" 600} 70708 [main] INFO o.a.s.l.Localizer - Reconstruct localized resource: C:\Users\ADMINI~1\AppData\Local\Temp\ee27d0b2-acb3-4e6d-bc2b-fccd2bd03438\supervisor\usercache 70709 [main] WARN o.a.s.l.Localizer - No left over resources found for any user during reconstructing of local resources at: C:\Users\ADMINI~1\AppData\Local\Temp\ee27d0b2-acb3-4e6d-bc2b-fccd2bd03438\supervisor\usercache 70728 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 70730 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@78ec89a6 70754 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 70756 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 70758 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52856 70758 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52856 70806 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f52580009 with negotiated timeout 20000 for client /127.0.0.1:52856 70806 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f52580009, negotiated timeout = 20000 70807 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 70807 [main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none 70811 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting 70815 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d880f52580009 70846 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d880f52580009 closed 70847 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down 70849 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 71207 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@642ee49c 71216 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 71217 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 70848 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x15d880f52580009, likely client has closed socket at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [storm-core-1.0.2.jar:1.0.2] at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [storm-core-1.0.2.jar:1.0.2] at java.lang.Thread.run(Unknown Source) [?:1.8.0_66] 71246 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:52856 which had sessionid 0x15d880f52580009 71247 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52859 71248 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52859 71272 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f5258000a with negotiated timeout 20000 for client /127.0.0.1:52859 71272 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f5258000a, negotiated timeout = 20000 71273 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 71429 [main] INFO o.a.s.d.supervisor - Starting supervisor with id 69e4ec2d-f43f-458c-b65f-563a0a1a3094 at host WIN-BQOBV63OBNM 71438 [main] INFO o.a.s.d.supervisor - Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" 60, "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" 1, "logviewer.max.per.worker.logs.size.mb" 2048, "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" 29, "zmq.threads" 1, "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" 1, "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" 60, "drpc.invocations.port" 3773, "supervisor.localizer.cache.target.size.mb" 10240, "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" 1, "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" 5, "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" 0, "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" 1, "pacemaker.host" "localhost", "storm.zookeeper.retry.times" 5, "ui.actions.enabled" true, "zmq.linger.millis" 0, "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" 100, "worker.log.level.reset.poll.secs" 30, "storm.zookeeper.port" 2000, "supervisor.heartbeat.frequency.secs" 5, "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" 64, "supervisor.blobstore.download.thread.count" 5, "drpc.queue.size" 128, "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" 65536, "topology.shellbolt.max.pending" 100, "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" 120, "logviewer.port" 8000, "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" 1024, "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" 2000, "nimbus.inbox.jar.expiration.secs" 3600, "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" 0, "supervisor.localizer.cleanup.interval.ms" 600000, "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" 64, "logviewer.cleanup.age.mins" 10080, "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" 3, "nimbus.credential.renewers.freq.secs" 600, "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" 120, "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" 10, "storm.health.check.timeout.ms" 5000, "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" 500, "topology.workers" 1, "pacemaker.base.threads" 10, "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\4a59ac17-0114-497f-8254-9ae1ae28a531", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" 1, "topology.message.timeout.secs" 30, "topology.state.synchronization.timeout.secs" 60, "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" 600, "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" 100, "storm.messaging.netty.max_retries" 300, "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" 20000, "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" 15000, "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" 60, "nimbus.task.timeout.secs" 30, "drpc.port" 3772, "pacemaker.max.threads" 50, "storm.zookeeper.retry.intervalceiling.millis" 30000, "nimbus.thrift.port" 6627, "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" 5, "supervisor.worker.start.timeout.secs" 120, "storm.zookeeper.retry.interval" 1000, "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" 1048576, "pacemaker.thread.timeout" 10, "task.credentials.poll.secs" 30, "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" 1, "topology.state.checkpoint.interval.ms" 1000, "supervisor.slots.ports" (1027 1028 1029), "topology.transfer.buffer.size" 1024, "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" 4, "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" 600, "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" 1024, "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" 120, "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" 5242880, "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" 1, "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -1, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" 3, "task.refresh.poll.secs" 10, "storm.exhibitor.port" 8080, "task.heartbeat.frequency.secs" 3, "pacemaker.port" 6699, "storm.messaging.netty.max_wait_ms" 1000, "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" 3774, "topology.error.throttle.interval.secs" 10, "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" 100000, "nimbus.cleanup.inbox.freq.secs" 600, "storm.blobstore.replication.factor" 3, "worker.heap.memory.mb" 768, "logviewer.max.sum.worker.logs.size.mb" 4096, "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" 30, "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" 4096, "topology.min.replication.count" 1, "topology.disruptor.wait.timeout.millis" 1000, "storm.nimbus.retry.intervalceiling.millis" 60000, "topology.trident.batch.emit.interval.millis" 50, "storm.auth.simple-acl.users" [], "drpc.invocations.threads" 64, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" 8080, "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" 262144, "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" 1048576, "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" 600} 71461 [main] INFO o.a.s.l.Localizer - Reconstruct localized resource: C:\Users\ADMINI~1\AppData\Local\Temp\4a59ac17-0114-497f-8254-9ae1ae28a531\supervisor\usercache 71461 [main] WARN o.a.s.l.Localizer - No left over resources found for any user during reconstructing of local resources at: C:\Users\ADMINI~1\AppData\Local\Temp\4a59ac17-0114-497f-8254-9ae1ae28a531\supervisor\usercache 71464 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 71465 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@1290ed28 71474 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 71475 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 71475 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52862 71475 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52862 71506 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f5258000b with negotiated timeout 20000 for client /127.0.0.1:52862 71506 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f5258000b, negotiated timeout = 20000 71506 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 71508 [main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none 71511 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting 71515 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d880f5258000b 71550 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d880f5258000b closed 71551 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 71552 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down 71553 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@29050de5 71555 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x15d880f5258000b, likely client has closed socket at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [storm-core-1.0.2.jar:1.0.2] at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [storm-core-1.0.2.jar:1.0.2] at java.lang.Thread.run(Unknown Source) [?:1.8.0_66] 71556 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:52862 which had sessionid 0x15d880f5258000b 71561 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 71562 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 71563 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52865 71563 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52865 71599 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f5258000c with negotiated timeout 20000 for client /127.0.0.1:52865 71600 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f5258000c, negotiated timeout = 20000 71600 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 71651 [main] INFO o.a.s.d.supervisor - Starting supervisor with id 121423ef-8c7e-4a2d-b149-1b1b15c58907 at host WIN-BQOBV63OBNM 71858 [main] INFO o.a.s.l.ThriftAccessLogger - Request ID: 1 access from: principal: operation: submitTopology 72054 [main] INFO o.a.s.d.nimbus - Received topology submission for StormTopologyTimer1 with conf {"topology.max.task.parallelism" nil, "topology.submitter.principal" "", "topology.acker.executors" nil, "topology.eventlogger.executors" 0, "storm.zookeeper.superACL" nil, "topology.users" (), "topology.submitter.user" "Administrator", "topology.tick.tuple.freq.secs" 10, "topology.kryo.register" nil, "topology.kryo.decorators" (), "storm.id" "StormTopologyTimer1-1-1501226298", "topology.name" "StormTopologyTimer1"} 72178 [main] INFO o.a.s.d.nimbus - uploadedJar 72271 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 72272 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@718f805a 72277 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 72278 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 72278 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52868 72280 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52868 72310 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f5258000d with negotiated timeout 20000 for client /127.0.0.1:52868 72311 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f5258000d, negotiated timeout = 20000 72312 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 72317 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x15d880f5258000d type:create cxid:0x2 zxid:0x26 txntype:-1 reqpath:n/a Error Path:/storm/blobstoremaxkeysequencenumber Error:KeeperErrorCode = NoNode for /storm/blobstoremaxkeysequencenumber 72548 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting 72558 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d880f5258000d 72614 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x15d880f5258000d, likely client has closed socket at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [storm-core-1.0.2.jar:1.0.2] at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [storm-core-1.0.2.jar:1.0.2] at java.lang.Thread.run(Unknown Source) [?:1.8.0_66] 72614 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d880f5258000d closed 72615 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:52868 which had sessionid 0x15d880f5258000d 72615 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down 72616 [main] INFO o.a.s.cluster - setup-path/blobstore/StormTopologyTimer1-1-1501226298-stormconf.ser/WIN-BQOBV63OBNM:6627-1 72878 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 72879 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@4735d6e5 72931 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 72939 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 72942 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52871 72943 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52871 73001 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f5258000e with negotiated timeout 20000 for client /127.0.0.1:52871 73007 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f5258000e, negotiated timeout = 20000 73008 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 73090 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting 73093 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d880f5258000e 73111 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d880f5258000e closed 73111 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:52871 which had sessionid 0x15d880f5258000e 73112 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down 73113 [main] INFO o.a.s.cluster - setup-path/blobstore/StormTopologyTimer1-1-1501226298-stormcode.ser/WIN-BQOBV63OBNM:6627-1 73278 [main] INFO o.a.s.d.nimbus - desired replication count 1 achieved, current-replication-count for conf key = 1, current-replication-count for code key = 1, current-replication-count for jar key = 1 73735 [main] INFO o.a.s.d.nimbus - Activating StormTopologyTimer1: StormTopologyTimer1-1-1501226298 83273 [timer] INFO o.a.s.s.EvenScheduler - Available slots: (["121423ef-8c7e-4a2d-b149-1b1b15c58907" 1027] ["121423ef-8c7e-4a2d-b149-1b1b15c58907" 1028] ["121423ef-8c7e-4a2d-b149-1b1b15c58907" 1029] ["69e4ec2d-f43f-458c-b65f-563a0a1a3094" 1024] ["69e4ec2d-f43f-458c-b65f-563a0a1a3094" 1025] ["69e4ec2d-f43f-458c-b65f-563a0a1a3094" 1026]) 83489 [timer] INFO o.a.s.d.nimbus - Setting new assignment for topology id StormTopologyTimer1-1-1501226298: #org.apache.storm.daemon.common.Assignment{:master-code-dir "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\fcc1cd09-238a-499f-967b-da2e3564a331", :node->host {"121423ef-8c7e-4a2d-b149-1b1b15c58907" "WIN-BQOBV63OBNM"}, :executor->node+port {[2 2] ["121423ef-8c7e-4a2d-b149-1b1b15c58907" 1027], [1 1] ["121423ef-8c7e-4a2d-b149-1b1b15c58907" 1027], [3 3] ["121423ef-8c7e-4a2d-b149-1b1b15c58907" 1027]}, :executor->start-time-secs {[1 1] 1501226309, [2 2] 1501226309, [3 3] 1501226309}, :worker->resources {["121423ef-8c7e-4a2d-b149-1b1b15c58907" 1027] [0.0 0.0 0.0]}} 83715 [Thread-9] INFO o.a.s.d.supervisor - Downloading code for storm id StormTopologyTimer1-1-1501226298 83721 [Thread-9] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 83733 [Thread-9] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@3eb818f8 83743 [Thread-9] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in C:\Users\ADMINI~1\AppData\Local\Temp\fcc1cd09-238a-499f-967b-da2e3564a331\blobs 83745 [Thread-9-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 83746 [Thread-9-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 83748 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52880 83749 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52880 83806 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f5258000f with negotiated timeout 20000 for client /127.0.0.1:52880 83806 [Thread-9-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f5258000f, negotiated timeout = 20000 83807 [Thread-9-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 83866 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting 83868 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d880f5258000f 83921 [Thread-9] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d880f5258000f closed 83921 [Thread-9-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down 83922 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:52880 which had sessionid 0x15d880f5258000f 84422 [Thread-9] INFO o.a.s.d.supervisor - Finished downloading code for storm id StormTopologyTimer1-1-1501226298 84467 [Thread-10] INFO o.a.s.d.supervisor - Launching worker with assignment {:storm-id "StormTopologyTimer1-1-1501226298", :executors [[2 2] [1 1] [3 3]], :resources #object[org.apache.storm.generated.WorkerResources 0x7b1f583d "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor 121423ef-8c7e-4a2d-b149-1b1b15c58907 on port 1027 with id 1f2b5f14-7744-43ac-878e-3cc52c3af546 84482 [Thread-10] INFO o.a.s.d.worker - Launching worker for StormTopologyTimer1-1-1501226298 on 121423ef-8c7e-4a2d-b149-1b1b15c58907:1027 with id 1f2b5f14-7744-43ac-878e-3cc52c3af546 and conf {"topology.builtin.metrics.bucket.size.secs" 60, "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" 1, "logviewer.max.per.worker.logs.size.mb" 2048, "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" 29, "zmq.threads" 1, "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" 1, "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" 60, "drpc.invocations.port" 3773, "supervisor.localizer.cache.target.size.mb" 10240, "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" 1, "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" 5, "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" 0, "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" 1, "pacemaker.host" "localhost", "storm.zookeeper.retry.times" 5, "ui.actions.enabled" true, "zmq.linger.millis" 0, "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" 100, "worker.log.level.reset.poll.secs" 30, "storm.zookeeper.port" 2000, "supervisor.heartbeat.frequency.secs" 5, "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" 64, "supervisor.blobstore.download.thread.count" 5, "drpc.queue.size" 128, "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" 65536, "topology.shellbolt.max.pending" 100, "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" 120, "logviewer.port" 8000, "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" 1024, "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" 2000, "nimbus.inbox.jar.expiration.secs" 3600, "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" 0, "supervisor.localizer.cleanup.interval.ms" 600000, "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" 64, "logviewer.cleanup.age.mins" 10080, "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" 3, "nimbus.credential.renewers.freq.secs" 600, "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" 120, "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" 10, "storm.health.check.timeout.ms" 5000, "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" 500, "topology.workers" 1, "pacemaker.base.threads" 10, "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\4a59ac17-0114-497f-8254-9ae1ae28a531", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" 1, "topology.message.timeout.secs" 30, "topology.state.synchronization.timeout.secs" 60, "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" 600, "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" 100, "storm.messaging.netty.max_retries" 300, "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" 20000, "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" 15000, "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" 60, "nimbus.task.timeout.secs" 30, "drpc.port" 3772, "pacemaker.max.threads" 50, "storm.zookeeper.retry.intervalceiling.millis" 30000, "nimbus.thrift.port" 6627, "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" 5, "supervisor.worker.start.timeout.secs" 120, "storm.zookeeper.retry.interval" 1000, "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" 1048576, "pacemaker.thread.timeout" 10, "task.credentials.poll.secs" 30, "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" 1, "topology.state.checkpoint.interval.ms" 1000, "supervisor.slots.ports" (1027 1028 1029), "topology.transfer.buffer.size" 1024, "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" 4, "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" 600, "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" 1024, "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" 120, "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" 5242880, "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" 1, "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -1, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" 3, "task.refresh.poll.secs" 10, "storm.exhibitor.port" 8080, "task.heartbeat.frequency.secs" 3, "pacemaker.port" 6699, "storm.messaging.netty.max_wait_ms" 1000, "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" 3774, "topology.error.throttle.interval.secs" 10, "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" 100000, "nimbus.cleanup.inbox.freq.secs" 600, "storm.blobstore.replication.factor" 3, "worker.heap.memory.mb" 768, "logviewer.max.sum.worker.logs.size.mb" 4096, "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" 30, "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" 4096, "topology.min.replication.count" 1, "topology.disruptor.wait.timeout.millis" 1000, "storm.nimbus.retry.intervalceiling.millis" 60000, "topology.trident.batch.emit.interval.millis" 50, "storm.auth.simple-acl.users" [], "drpc.invocations.threads" 64, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" 8080, "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" 262144, "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" 1048576, "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" 600} 84492 [Thread-10] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 84493 [Thread-10] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@1a782438 84501 [Thread-10-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 84502 [Thread-10-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 84509 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52883 84512 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52883 84555 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f52580010 with negotiated timeout 20000 for client /127.0.0.1:52883 84556 [Thread-10-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f52580010, negotiated timeout = 20000 84556 [Thread-10-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 84556 [Thread-10-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none 84558 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting 84559 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d880f52580010 84577 [Thread-10] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d880f52580010 closed 84577 [Thread-10-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down 84577 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:52883 which had sessionid 0x15d880f52580010 84578 [Thread-10] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 84579 [Thread-10] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@64d6a1d9 84585 [Thread-10-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 84587 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:52886 84588 [Thread-10-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 84588 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:52886 84611 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d880f52580011 with negotiated timeout 20000 for client /127.0.0.1:52886 84611 [Thread-10-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d880f52580011, negotiated timeout = 20000 84611 [Thread-10-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 84625 [Thread-10] INFO o.a.s.s.a.AuthUtils - Got AutoCreds [] 84648 [Thread-10] INFO o.a.s.d.worker - Reading Assignments. 85087 [Thread-10] INFO o.a.s.d.worker - Registering IConnectionCallbacks for 121423ef-8c7e-4a2d-b149-1b1b15c58907:1027 85526 [Thread-10] INFO o.a.s.d.executor - Loading executor MySpout:[2 2] 85615 [Thread-10] INFO o.a.s.d.executor - Loaded executor tasks MySpout:[2 2] 85933 [refresh-active-timer] INFO o.a.s.d.worker - All connections are ready for worker 121423ef-8c7e-4a2d-b149-1b1b15c58907:1027 with id 1f2b5f14-7744-43ac-878e-3cc52c3af546 86873 [Thread-10] INFO o.a.s.d.executor - Finished loading executor MySpout:[2 2] 86903 [Thread-10] INFO o.a.s.d.executor - Loading executor __acker:[3 3] 86909 [Thread-10] INFO o.a.s.d.executor - Loaded executor tasks __acker:[3 3] 86940 [Thread-10] INFO o.a.s.d.executor - Timeouts disabled for executor __acker:[3 3] 86940 [Thread-10] INFO o.a.s.d.executor - Finished loading executor __acker:[3 3] 87096 [Thread-10] INFO o.a.s.d.executor - Loading executor MyBolt:[1 1] 87098 [Thread-10] INFO o.a.s.d.executor - Loaded executor tasks MyBolt:[1 1] 87104 [Thread-10] INFO o.a.s.d.executor - Finished loading executor MyBolt:[1 1] 87119 [Thread-10] INFO o.a.s.d.executor - Loading executor __system:[-1 -1] 87122 [Thread-10] INFO o.a.s.d.executor - Loaded executor tasks __system:[-1 -1] 87126 [Thread-10] INFO o.a.s.d.executor - Timeouts disabled for executor __system:[-1 -1] 87126 [Thread-10] INFO o.a.s.d.executor - Finished loading executor __system:[-1 -1] 87191 [Thread-10] INFO o.a.s.d.worker - Started with log levels: {"" #object[org.apache.logging.log4j.Level 0x126958a3 "INFO"], "org.apache.zookeeper" #object[org.apache.logging.log4j.Level 0x5f681a7b "WARN"]} 87211 [Thread-10] INFO o.a.s.d.worker - Worker has topology config {"topology.builtin.metrics.bucket.size.secs" 60, "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" 1, "logviewer.max.per.worker.logs.size.mb" 2048, "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" 29, "zmq.threads" 1, "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" 1, "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" 60, "drpc.invocations.port" 3773, "supervisor.localizer.cache.target.size.mb" 10240, "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" 1, "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" 5, "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" 0, "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" 1, "pacemaker.host" "localhost", "storm.zookeeper.retry.times" 5, "ui.actions.enabled" true, "zmq.linger.millis" 0, "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" 100, "worker.log.level.reset.poll.secs" 30, "storm.zookeeper.port" 2000, "supervisor.heartbeat.frequency.secs" 5, "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" 64, "supervisor.blobstore.download.thread.count" 5, "drpc.queue.size" 128, "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" 65536, "topology.shellbolt.max.pending" 100, "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" 120, "logviewer.port" 8000, "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" 1024, "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "topology.submitter.principal" "", "storm.nimbus.retry.interval.millis" 2000, "nimbus.inbox.jar.expiration.secs" 3600, "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" 0, "supervisor.localizer.cleanup.interval.ms" 600000, "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" 64, "logviewer.cleanup.age.mins" 10080, "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" 3, "nimbus.credential.renewers.freq.secs" 600, "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" 120, "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" 10, "storm.health.check.timeout.ms" 5000, "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" 500, "topology.workers" 1, "pacemaker.base.threads" 10, "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\fcc1cd09-238a-499f-967b-da2e3564a331", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" 1, "topology.message.timeout.secs" 30, "topology.state.synchronization.timeout.secs" 60, "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" 600, "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" 100, "storm.messaging.netty.max_retries" 300, "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" 20000, "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" 15000, "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" 60, "nimbus.task.timeout.secs" 30, "storm.zookeeper.superACL" nil, "drpc.port" 3772, "pacemaker.max.threads" 50, "storm.zookeeper.retry.intervalceiling.millis" 30000, "nimbus.thrift.port" 6627, "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" 5, "supervisor.worker.start.timeout.secs" 120, "storm.zookeeper.retry.interval" 1000, "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" 1048576, "pacemaker.thread.timeout" 10, "task.credentials.poll.secs" 30, "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" 1, "topology.state.checkpoint.interval.ms" 1000, "supervisor.slots.ports" [6700 6701 6702 6703], "topology.transfer.buffer.size" 1024, "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" 4, "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" 600, "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" 1024, "backpressure.disruptor.low.watermark" 0.4, "topology.users" [], "nimbus.task.launch.secs" 120, "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" 5242880, "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" 1, "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.submitter.user" "Administrator", "topology.tick.tuple.freq.secs" 10, "drpc.https.port" -1, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" 3, "task.refresh.poll.secs" 10, "storm.exhibitor.port" 8080, "task.heartbeat.frequency.secs" 3, "pacemaker.port" 6699, "storm.messaging.netty.max_wait_ms" 1000, "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" 3774, "topology.error.throttle.interval.secs" 10, "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "topology.kryo.register" nil, "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" 100000, "nimbus.cleanup.inbox.freq.secs" 600, "storm.blobstore.replication.factor" 3, "worker.heap.memory.mb" 768, "logviewer.max.sum.worker.logs.size.mb" 4096, "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" 30, "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" 4096, "topology.min.replication.count" 1, "topology.disruptor.wait.timeout.millis" 1000, "storm.nimbus.retry.intervalceiling.millis" 60000, "topology.trident.batch.emit.interval.millis" 50, "storm.auth.simple-acl.users" [], "drpc.invocations.threads" 64, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" 8080, "topology.kryo.decorators" [], "storm.id" "StormTopologyTimer1-1-1501226298", "topology.name" "StormTopologyTimer1", "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" 262144, "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" 1048576, "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" 600} 87211 [Thread-10] INFO o.a.s.d.worker - Worker 1f2b5f14-7744-43ac-878e-3cc52c3af546 for storm StormTopologyTimer1-1-1501226298 on 121423ef-8c7e-4a2d-b149-1b1b15c58907:1027 has finished loading 87212 [Thread-10] INFO o.a.s.config - SET worker-user 1f2b5f14-7744-43ac-878e-3cc52c3af546 88037 [Thread-20-__system-executor[-1 -1]] INFO o.a.s.d.executor - Preparing bolt __system:(-1) 88076 [Thread-16-__acker-executor[3 3]] INFO o.a.s.d.executor - Preparing bolt __acker:(3) 88111 [Thread-18-MyBolt-executor[1 1]] INFO o.a.s.d.executor - Preparing bolt MyBolt:(1) 88113 [Thread-18-MyBolt-executor[1 1]] INFO o.a.s.d.executor - Prepared bolt MyBolt:(1) 88116 [Thread-14-MySpout-executor[2 2]] INFO o.a.s.d.executor - Opening spout MySpout:(2) 88147 [Thread-20-__system-executor[-1 -1]] INFO o.a.s.d.executor - Prepared bolt __system:(-1) 88183 [Thread-14-MySpout-executor[2 2]] INFO o.a.s.d.executor - Opened spout MySpout:(2) 88184 [Thread-16-__acker-executor[3 3]] INFO o.a.s.d.executor - Prepared bolt __acker:(3) 88192 [Thread-14-MySpout-executor[2 2]] INFO o.a.s.d.executor - Activating spout MySpout:(2) spout:1 sum=1 spout:2 sum=3 spout:3 sum=6 spout:4 sum=10 spout:5 sum=15 spout:6 sum=21 spout:7 sum=28 spout:8 sum=36 spout:9 sum=45 定时时间到了 spout:10 sum=55 spout:11 sum=66 spout:12 sum=78 spout:13 sum=91 spout:14 sum=105 spout:15 sum=120 spout:16 sum=136 spout:17 sum=153 spout:18 sum=171 spout:19 sum=190 定时时间到了 spout:20 sum=210 spout:21 sum=231 spout:22 sum=253 spout:23 sum=276 spout:24 sum=300 spout:25 sum=325 spout:26 sum=351 spout:27 sum=378 spout:28 sum=406 spout:29 sum=435 定时时间到了 spout:30 sum=465 spout:31 sum=496 spout:32 sum=528 spout:33 sum=561 spout:34 sum=595 spout:35 sum=630 spout:36 sum=666 spout:37 sum=703 spout:38 sum=741 spout:39 sum=780 定时时间到了 spout:40 sum=820 spout:41 sum=861 spout:42 sum=903 spout:43 sum=946 spout:44 sum=990 spout:45 sum=1035 spout:46 sum=1081 spout:47 sum=1128 spout:48 sum=1176 spout:49 sum=1225 定时时间到了 spout:50 sum=1275 spout:51 sum=1326 spout:52 sum=1378 spout:53 sum=1431 spout:54 sum=1485 spout:55 sum=1540 spout:56 sum=1596 spout:57 sum=1653 spout:58 sum=1711 spout:59 sum=1770 定时时间到了 spout:60 sum=1830 spout:61 sum=1891 spout:62 sum=1953 spout:63 sum=2016 spout:64 sum=2080 spout:65 sum=2145 spout:66 sum=2211 spout:67 sum=2278 spout:68 sum=2346 spout:69 sum=2415 定时时间到了 spout:70 sum=2485 spout:71 sum=2556 spout:72 sum=2628 spout:73 sum=2701 spout:74 sum=2775 spout:75 sum=2850 spout:76 sum=2926 spout:77 sum=3003 spout:78 sum=3081 spout:79 sum=3160 定时时间到了 spout:80 sum=3240 spout:81 sum=3321 spout:82 sum=3403 spout:83 sum=3486 spout:84 sum=3570 spout:85 sum=3655 spout:86 sum=3741 spout:87 sum=3828 spout:88 sum=3916 spout:89 sum=4005 定时时间到了 spout:90 sum=4095 spout:91 sum=4186 spout:92 sum=4278 spout:93 sum=4371 spout:94 sum=4465 spout:95 sum=4560 spout:96 sum=4656 spout:97 sum=4753 spout:98 sum=4851 spout:99 sum=4950 定时时间到了 spout:100 sum=5050 spout:101 sum=5151 spout:102 sum=5253 spout:103 sum=5356 spout:104 sum=5460 spout:105 sum=5565 spout:106 sum=5671 spout:107 sum=5778 spout:108 sum=5886 spout:109 sum=5995 定时时间到了 spout:110 sum=6105 spout:111 sum=6216 spout:112 sum=6328 spout:113 sum=6441 spout:114 sum=6555 spout:115 sum=6670 spout:116 sum=6786 spout:117 sum=6903 spout:118 sum=7021 spout:119 sum=7140 定时时间到了 spout:120 sum=7260 spout:121 sum=7381 spout:122 sum=7503 spout:123 sum=7626 spout:124 sum=7750 spout:125 sum=7875 spout:126 sum=8001 spout:127 sum=8128 spout:128 sum=8256 spout:129 sum=8385 定时时间到了 spout:130 sum=8515 spout:131 sum=8646 spout:132 sum=8778 spout:133 sum=8911 spout:134 sum=9045 spout:135 sum=9180 spout:136 sum=9316 spout:137 sum=9453 spout:138 sum=9591 spout:139 sum=9730 编写代码 StormTopologyTimer2.java package zhouls.bigdata.stormDemo; import java.util.HashMap; import java.util.Map; import org.apache.storm.Config; import org.apache.storm.Constants; import org.apache.storm.LocalCluster; import org.apache.storm.StormSubmitter; import org.apache.storm.generated.AlreadyAliveException; import org.apache.storm.generated.AuthorizationException; import org.apache.storm.generated.InvalidTopologyException; import org.apache.storm.spout.SpoutOutputCollector; import org.apache.storm.task.OutputCollector; import org.apache.storm.task.TopologyContext; import org.apache.storm.topology.OutputFieldsDeclarer; import org.apache.storm.topology.TopologyBuilder; import org.apache.storm.topology.base.BaseRichBolt; import org.apache.storm.topology.base.BaseRichSpout; import org.apache.storm.tuple.Fields; import org.apache.storm.tuple.Tuple; import org.apache.storm.tuple.Values; import org.apache.storm.utils.Utils; public class StormTopologyTimer2 { public static class MySpout extends BaseRichSpout{ private Map conf; private TopologyContext context; private SpoutOutputCollector collector; public void open(Map conf, TopologyContext context, SpoutOutputCollector collector) { this.conf = conf; this.collector = collector; this.context = context; } int num = 0; public void nextTuple() { num++; System.out.println("spout:"+num); this.collector.emit(new Values(num)); Utils.sleep(1000); } public void declareOutputFields(OutputFieldsDeclarer declarer) { declarer.declare(new Fields("num")); } } public static class MyBolt extends BaseRichBolt{ private Map stormConf; private TopologyContext context; private OutputCollector collector; public void prepare(Map stormConf, TopologyContext context, OutputCollector collector) { this.stormConf = stormConf; this.context = context; this.collector = collector; } int sum = 0; public void execute(Tuple input) { if(input.getSourceComponent().equals(Constants.SYSTEM_COMPONENT_ID)){ System.out.println("定时时间到了"); }else{ Integer num = input.getIntegerByField("num"); sum += num; System.out.println("sum="+sum); } } public void declareOutputFields(OutputFieldsDeclarer declarer) { } public Map<String, Object> getComponentConfiguration() { HashMap<String, Object> hashMap = new HashMap<String, Object>(); hashMap.put(Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS, 10);//给当前bolt设置定时任务 return hashMap; } } public static void main(String[] args) { TopologyBuilder topologyBuilder = new TopologyBuilder(); String spout_id = MySpout.class.getSimpleName(); String bolt_id = MyBolt.class.getSimpleName(); topologyBuilder.setSpout(spout_id, new MySpout()); topologyBuilder.setBolt(bolt_id, new MyBolt()).shuffleGrouping(spout_id); Config config = new Config(); String topology_name = StormTopologyTimer2.class.getSimpleName(); if(args.length==0){ //在本地运行 LocalCluster localCluster = new LocalCluster(); localCluster.submitTopology(topology_name, config, topologyBuilder.createTopology()); }else{ //在集群运行 try { StormSubmitter.submitTopology(topology_name, config, topologyBuilder.createTopology()); } catch (AlreadyAliveException e) { e.printStackTrace(); } catch (InvalidTopologyException e) { e.printStackTrace(); } catch (AuthorizationException e) { e.printStackTrace(); } } } } 停掉，复制粘贴分析分析 7234 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 7235 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:host.name=WIN-BQOBV63OBNM 7235 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:java.version=1.8.0_66 7235 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:java.vendor=Oracle Corporation 7235 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:java.home=C:\Program Files\Java\jre1.8.0_66 7236 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:java.class.path=D:\Code\eclipseMarsCode\stormDemo\target\classes;D:\SoftWare\maven\repository\org\apache\storm\storm-core\1.0.2\storm-core-1.0.2.jar;D:\SoftWare\maven\repository\com\esotericsoftware\kryo\3.0.3\kryo-3.0.3.jar;D:\SoftWare\maven\repository\com\esotericsoftware\reflectasm\1.10.1\reflectasm-1.10.1.jar;D:\SoftWare\maven\repository\org\ow2\asm\asm\5.0.3\asm-5.0.3.jar;D:\SoftWare\maven\repository\com\esotericsoftware\minlog\1.3.0\minlog-1.3.0.jar;D:\SoftWare\maven\repository\org\objenesis\objenesis\2.1\objenesis-2.1.jar;D:\SoftWare\maven\repository\org\clojure\clojure\1.7.0\clojure-1.7.0.jar;D:\SoftWare\maven\repository\com\lmax\disruptor\3.3.2\disruptor-3.3.2.jar;D:\SoftWare\maven\repository\org\apache\logging\log4j\log4j-api\2.1\log4j-api-2.1.jar;D:\SoftWare\maven\repository\org\apache\logging\log4j\log4j-core\2.1\log4j-core-2.1.jar;D:\SoftWare\maven\repository\org\apache\logging\log4j\log4j-slf4j-impl\2.1\log4j-slf4j-impl-2.1.jar;D:\SoftWare\maven\repository\org\slf4j\log4j-over-slf4j\1.6.6\log4j-over-slf4j-1.6.6.jar;D:\SoftWare\maven\repository\javax\servlet\servlet-api\2.5\servlet-api-2.5.jar;D:\SoftWare\maven\repository\org\slf4j\slf4j-api\1.7.7\slf4j-api-1.7.7.jar 7236 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:java.library.path=C:\Program Files\Java\jre1.8.0_66\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:/Program Files/Java/jre1.8.0_66/bin/server;C:/Program Files/Java/jre1.8.0_66/bin;C:/Program Files/Java/jre1.8.0_66/lib/amd64;%WEKA39_HOME%\lib\mysql-connector-java-5.1.21-bin.jar;%WEKA37_HOME%\lib\mysql-connector-java-5.1.21-bin.jar;C:\Program Files\Java\jdk1.8.0_66\jre\lib\ext\mysql-connector-java-5.1.21-bin.jar;C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;D:\SoftWare\MATLAB R2013a\runtime\win64;D:\SoftWare\MATLAB R2013a\bin;C:\Program Files (x86)\IDM Computer Solutions\UltraCompare;C:\Program Files\Java\jdk1.8.0_66\bin;C:\Program Files\Java\jdk1.8.0_66\jre\bin;D:\SoftWare\apache-ant-1.9.0\bin;HADOOP_HOME\bin;D:\SoftWare\apache-maven-3.3.9\bin;D:\SoftWare\Scala\bin;D:\SoftWare\Scala\jre\bin;%MYSQL_HOME\bin;D:\SoftWare\MySQL\mysql-5.7.11-winx64;;D:\SoftWare\apache-tomcat-7.0.69\bin;%C:\Windows\System32;%C:\Windows\SysWOW64;;D:\SoftWare\apache-maven-3.3.9\bin;D:\SoftWare\apache-tomcat-7.0.69\bin;D:\SoftWare\apache-tomcat-7.0.69\bin;D:\SoftWare\Anaconda2;D:\SoftWare\Anaconda2\Scripts;D:\SoftWare\Anaconda2\Library\bin;D:\SoftWare\MySQL Server\MySQL Server 5.0\bin;D:\SoftWare\Python\Python36\Scripts\;D:\SoftWare\Python\Python36\;D:\SoftWare\SSH Secure Shell;D:\SoftWare\eclipse;;. 7236 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:java.io.tmpdir=C:\Users\ADMINI~1\AppData\Local\Temp\ 7236 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:java.compiler=<NA> 7236 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:os.name=Windows 7 7236 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:os.arch=amd64 7236 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:os.version=6.1 7236 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:user.name=Administrator 7236 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:user.home=C:\Users\Administrator 7236 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Server environment:user.dir=D:\Code\eclipseMarsCode\stormDemo 7261 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir C:\Users\ADMINI~1\AppData\Local\Temp\978708b6-e98a-460b-9072-bbb047754867\version-2 snapdir C:\Users\ADMINI~1\AppData\Local\Temp\978708b6-e98a-460b-9072-bbb047754867\version-2 7427 [main] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - binding to port 0.0.0.0/0.0.0.0:2000 7432 [main] INFO o.a.s.zookeeper - Starting inprocess zookeeper at port 2000 and dir C:\Users\ADMINI~1\AppData\Local\Temp\978708b6-e98a-460b-9072-bbb047754867 7628 [main] INFO o.a.s.d.nimbus - Starting Nimbus with conf {"topology.builtin.metrics.bucket.size.secs" 60, "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" 1, "logviewer.max.per.worker.logs.size.mb" 2048, "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" 29, "zmq.threads" 1, "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" 1, "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" 60, "drpc.invocations.port" 3773, "supervisor.localizer.cache.target.size.mb" 10240, "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" 1, "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" 5, "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" 0, "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" 1, "pacemaker.host" "localhost", "storm.zookeeper.retry.times" 5, "ui.actions.enabled" true, "zmq.linger.millis" 0, "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" 100, "worker.log.level.reset.poll.secs" 30, "storm.zookeeper.port" 2000, "supervisor.heartbeat.frequency.secs" 5, "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" 64, "supervisor.blobstore.download.thread.count" 5, "drpc.queue.size" 128, "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" 65536, "topology.shellbolt.max.pending" 100, "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" 120, "logviewer.port" 8000, "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" 1024, "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" 2000, "nimbus.inbox.jar.expiration.secs" 3600, "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" 0, "supervisor.localizer.cleanup.interval.ms" 600000, "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" 64, "logviewer.cleanup.age.mins" 10080, "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" 3, "nimbus.credential.renewers.freq.secs" 600, "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" 120, "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" 10, "storm.health.check.timeout.ms" 5000, "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" 500, "topology.workers" 1, "pacemaker.base.threads" 10, "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\98bd25da-d0be-4fe9-91f6-7d84d1b38bbc", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" 1, "topology.message.timeout.secs" 30, "topology.state.synchronization.timeout.secs" 60, "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" 600, "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" 100, "storm.messaging.netty.max_retries" 300, "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" 20000, "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" 15000, "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" 60, "nimbus.task.timeout.secs" 30, "drpc.port" 3772, "pacemaker.max.threads" 50, "storm.zookeeper.retry.intervalceiling.millis" 30000, "nimbus.thrift.port" 6627, "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" 5, "supervisor.worker.start.timeout.secs" 120, "storm.zookeeper.retry.interval" 1000, "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" 1048576, "pacemaker.thread.timeout" 10, "task.credentials.poll.secs" 30, "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" 1, "topology.state.checkpoint.interval.ms" 1000, "supervisor.slots.ports" [6700 6701 6702 6703], "topology.transfer.buffer.size" 1024, "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" 4, "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" 600, "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" 1024, "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" 120, "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" 5242880, "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" 1, "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -1, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" 3, "task.refresh.poll.secs" 10, "storm.exhibitor.port" 8080, "task.heartbeat.frequency.secs" 3, "pacemaker.port" 6699, "storm.messaging.netty.max_wait_ms" 1000, "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" 3774, "topology.error.throttle.interval.secs" 10, "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" 100000, "nimbus.cleanup.inbox.freq.secs" 600, "storm.blobstore.replication.factor" 3, "worker.heap.memory.mb" 768, "logviewer.max.sum.worker.logs.size.mb" 4096, "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" 30, "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" 4096, "topology.min.replication.count" 1, "topology.disruptor.wait.timeout.millis" 1000, "storm.nimbus.retry.intervalceiling.millis" 60000, "topology.trident.batch.emit.interval.millis" 50, "storm.auth.simple-acl.users" [], "drpc.invocations.threads" 64, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" 8080, "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" 262144, "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" 1048576, "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" 600} 7792 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 7798 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 7798 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:host.name=WIN-BQOBV63OBNM 7798 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:java.version=1.8.0_66 7798 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:java.vendor=Oracle Corporation 7799 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:java.home=C:\Program Files\Java\jre1.8.0_66 7799 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:java.class.path=D:\Code\eclipseMarsCode\stormDemo\target\classes;D:\SoftWare\maven\repository\org\apache\storm\storm-core\1.0.2\storm-core-1.0.2.jar;D:\SoftWare\maven\repository\com\esotericsoftware\kryo\3.0.3\kryo-3.0.3.jar;D:\SoftWare\maven\repository\com\esotericsoftware\reflectasm\1.10.1\reflectasm-1.10.1.jar;D:\SoftWare\maven\repository\org\ow2\asm\asm\5.0.3\asm-5.0.3.jar;D:\SoftWare\maven\repository\com\esotericsoftware\minlog\1.3.0\minlog-1.3.0.jar;D:\SoftWare\maven\repository\org\objenesis\objenesis\2.1\objenesis-2.1.jar;D:\SoftWare\maven\repository\org\clojure\clojure\1.7.0\clojure-1.7.0.jar;D:\SoftWare\maven\repository\com\lmax\disruptor\3.3.2\disruptor-3.3.2.jar;D:\SoftWare\maven\repository\org\apache\logging\log4j\log4j-api\2.1\log4j-api-2.1.jar;D:\SoftWare\maven\repository\org\apache\logging\log4j\log4j-core\2.1\log4j-core-2.1.jar;D:\SoftWare\maven\repository\org\apache\logging\log4j\log4j-slf4j-impl\2.1\log4j-slf4j-impl-2.1.jar;D:\SoftWare\maven\repository\org\slf4j\log4j-over-slf4j\1.6.6\log4j-over-slf4j-1.6.6.jar;D:\SoftWare\maven\repository\javax\servlet\servlet-api\2.5\servlet-api-2.5.jar;D:\SoftWare\maven\repository\org\slf4j\slf4j-api\1.7.7\slf4j-api-1.7.7.jar 7799 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:java.library.path=C:\Program Files\Java\jre1.8.0_66\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:/Program Files/Java/jre1.8.0_66/bin/server;C:/Program Files/Java/jre1.8.0_66/bin;C:/Program Files/Java/jre1.8.0_66/lib/amd64;%WEKA39_HOME%\lib\mysql-connector-java-5.1.21-bin.jar;%WEKA37_HOME%\lib\mysql-connector-java-5.1.21-bin.jar;C:\Program Files\Java\jdk1.8.0_66\jre\lib\ext\mysql-connector-java-5.1.21-bin.jar;C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;D:\SoftWare\MATLAB R2013a\runtime\win64;D:\SoftWare\MATLAB R2013a\bin;C:\Program Files (x86)\IDM Computer Solutions\UltraCompare;C:\Program Files\Java\jdk1.8.0_66\bin;C:\Program Files\Java\jdk1.8.0_66\jre\bin;D:\SoftWare\apache-ant-1.9.0\bin;HADOOP_HOME\bin;D:\SoftWare\apache-maven-3.3.9\bin;D:\SoftWare\Scala\bin;D:\SoftWare\Scala\jre\bin;%MYSQL_HOME\bin;D:\SoftWare\MySQL\mysql-5.7.11-winx64;;D:\SoftWare\apache-tomcat-7.0.69\bin;%C:\Windows\System32;%C:\Windows\SysWOW64;;D:\SoftWare\apache-maven-3.3.9\bin;D:\SoftWare\apache-tomcat-7.0.69\bin;D:\SoftWare\apache-tomcat-7.0.69\bin;D:\SoftWare\Anaconda2;D:\SoftWare\Anaconda2\Scripts;D:\SoftWare\Anaconda2\Library\bin;D:\SoftWare\MySQL Server\MySQL Server 5.0\bin;D:\SoftWare\Python\Python36\Scripts\;D:\SoftWare\Python\Python36\;D:\SoftWare\SSH Secure Shell;D:\SoftWare\eclipse;;. 7799 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:java.io.tmpdir=C:\Users\ADMINI~1\AppData\Local\Temp\ 7799 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:java.compiler=<NA> 7799 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:os.name=Windows 7 7800 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:os.arch=amd64 7800 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:os.version=6.1 7800 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:user.name=Administrator 7800 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:user.home=C:\Users\Administrator 7800 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Client environment:user.dir=D:\Code\eclipseMarsCode\stormDemo 7801 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@6824b913 7826 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 7828 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 7828 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:53355 7835 [main] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in C:\Users\ADMINI~1\AppData\Local\Temp\98bd25da-d0be-4fe9-91f6-7d84d1b38bbc\blobs 7837 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:53355 7842 [SyncThread:0] INFO o.a.s.s.o.a.z.s.p.FileTxnLog - Creating new log file: log.1 7845 [main] INFO o.a.s.d.nimbus - Using default scheduler 7851 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 7854 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@21a9a705 7863 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 7864 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 7865 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:53358 7866 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:53358 7915 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 7916 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@753fd7a1 7934 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 7936 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 7936 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:53361 7938 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:53361 8188 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d881632490000 with negotiated timeout 20000 for client /127.0.0.1:53355 8189 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d881632490000, negotiated timeout = 20000 8191 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d881632490001 with negotiated timeout 20000 for client /127.0.0.1:53358 8191 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d881632490001, negotiated timeout = 20000 8193 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d881632490002 with negotiated timeout 20000 for client /127.0.0.1:53361 8193 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d881632490002, negotiated timeout = 20000 8198 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 8198 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 8198 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 8202 [main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none 8203 [main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none 8355 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting 8369 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d881632490002 8397 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d881632490002 closed 8398 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down 8398 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:53361 which had sessionid 0x15d881632490002 8404 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 8406 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@40de8f93 8419 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 8422 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 8422 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:53364 8422 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 8425 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:53364 8428 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@45ab3bdd 8440 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 8443 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 8445 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:53367 8447 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:53367 8472 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d881632490003 with negotiated timeout 20000 for client /127.0.0.1:53364 8473 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d881632490003, negotiated timeout = 20000 8473 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 8513 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d881632490004 with negotiated timeout 20000 for client /127.0.0.1:53367 8513 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d881632490004, negotiated timeout = 20000 8514 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 9058 [main] INFO o.a.s.zookeeper - Queued up for leader lock. 9114 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x15d881632490001 type:create cxid:0x1 zxid:0x12 txntype:-1 reqpath:n/a Error Path:/storm/leader-lock Error:KeeperErrorCode = NoNode for /storm/leader-lock 9194 [Curator-Framework-0] WARN o.a.s.s.o.a.c.u.ZKPaths - The version of ZooKeeper being used doesn't support Container nodes. CreateMode.PERSISTENT will be used instead. 9322 [main] INFO o.a.s.d.m.MetricsUtils - Using statistics reporter plugin:org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter 9326 [main] INFO o.a.s.d.m.r.JmxPreparableReporter - Preparing... 9387 [main] INFO o.a.s.d.common - Started statistics report plugin... 9406 [main-EventThread] INFO o.a.s.zookeeper - WIN-BQOBV63OBNM gained leadership, checking if it has all the topology code locally. 9451 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 9454 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@ed2f2f6 9501 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 9503 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 9503 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:53370 9504 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:53370 9531 [main-EventThread] INFO o.a.s.zookeeper - active-topology-ids [] local-topology-ids [] diff-topology [] 9532 [main-EventThread] INFO o.a.s.zookeeper - Accepting leadership, all active topology found localy. 9558 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d881632490005 with negotiated timeout 20000 for client /127.0.0.1:53370 9559 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, sessionid = 0x15d881632490005, negotiated timeout = 20000 9559 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 9560 [main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none 9563 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting 9566 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d881632490005 9627 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:53370 which had sessionid 0x15d881632490005 9627 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d881632490005 closed 9627 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down 9629 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 9630 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@29be997f 9639 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 9639 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting 9641 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 9641 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:53373 9644 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:53373 9640 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@7f9e8421 9654 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not attempt to authenticate using SASL (unknown error) 9655 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating session 9656 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:53376 9656 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:53376 9704 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d881632490006 with negotiated timeout 20000 for client /127.0.0.1:53373 9704 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.
2017-11-11
优秀的个人博客,低调大师

Hadoop MapReduce编程 API入门系列之计数器（二十七）

MapReduce 计数器是什么？ 计数器是用来记录job的执行进度和状态的。它的作用可以理解为日志。我们可以在程序的某个位置插入计数器，记录数据或者进度的变化情况。 MapReduce 计数器能做什么？ MapReduce 计数器（Counter）为我们提供一个窗口，用于观察 MapReduce Job 运行期的各种细节数据。对 MapReduce 性能调优很有帮助，MapReduce 性能优化的评估大部分都是基于这些 Counter 的数值表现出来的。 MapReduce 都有哪些内置计数器？ MapReduce 自带了许多默认 Counter，现在我们来分析这些默认 Counter 的含义，方便大家观察 Job 结果，如输入的字节数、输出的字节数、Map 端 输入/输出的字节数和条数、Reduce 端的输入/输出的字节数和条数等。下面我们只需了解这些内置计数器，知道计数器组名称（groupName）和计数器名称（counterName），以后使用计数器会查找groupName和counterName即可。 任务计数器 在任务执行过程中，任务计数器采集任务的相关信息，每个作业的所有任务的结果会被聚集起来。例如，MAP_INPUT_RECORDS 计数器统计每个 map 任务输入记录的总数， 并在一个作业的所有 map 任务上进行聚集，使得最终数字是整个作业的所有输入记录的总数。任务计数器由其关联任务维护，并定期发送给 TaskTracker，再由 TaskTracker 发送给 JobTracker。因此，计数器能够被全局地聚集。下面我们分别了解各种任务计数器。 1、MapReduce任务计数器 MapReduce 任务计数器的 groupName为org.apache.hadoop.mapreduce.TaskCounter，它包含的计数器如下表所示，计数器名称列的括号（）内容即为counterName。 map 输入的记录数（MAP_INPUT_RECORDS） 作业中所有 map 已处理的输入记录数。每次 RecorderReader 读到一条记录并将其传给 map 的 map() 函数时，该计数器的值增加。 map 跳过的记录数（MAP_SKIPPED_RECORDS） 作业中所有 map 跳过的输入记录数。 map 输入的字节数（MAP_INPUT_BYTES） 作业中所有 map 已处理的未经压缩的输入数据的字节数。每次 RecorderReader 读到一条记录并 将其传给 map 的 map() 函数时，该计数器的值增加。 分片（split）的原始字节数（SPLIT_RAW_BYTES） 由 map 读取的输入-分片对象的字节数。这些对象描述分片元数据（文件的位移和长度），而不是分片的数据自身，因此总规模是小的。 map 输出的记录数（MAP_OUTPUT_RECORDS） 作业中所有 map 产生的 map 输出记录数。每次某一个 map 的Context 调用 write() 方法时，该计数器的值增加。 map 输出的字节数（MAP_OUTPUT_BYTES） 作业中所有 map 产生的 未经压缩的输出数据的字节数。每次某一个 map 的 Context 调用 write() 方法时，该计数器的值增加。 map 输出的物化字节数（MAP_OUTPUT_MATERIALIZED_BYTES） map 输出后确实写到磁盘上的字节数；若 map 输出压缩功能被启用，则会在计数器值上反映出来。 combine 输入的记录数（COMBINE_INPUT_RECORDS） 作业中所有 Combiner（如果有）已处理的输入记录数。Combiner 的迭代器每次读一个值，该计数器的值增加。 combine 输出的记录数（COMBINE_OUTPUT_RECORDS） 作业中所有 Combiner（如果有）已产生的输出记录数。每当一个 Combiner 的 Context 调用 write() 方法时，该计数器的值增加。 reduce 输入的组（REDUCE_INPUT_GROUPS） 作业中所有 reducer 已经处理的不同的码分组的个数。每当某一个 reducer 的 reduce() 被调用时，该计数器的值增加。 reduce 输入的记录数（REDUCE_INPUT_RECORDS） 作业中所有 reducer 已经处理的输入记录的个数。每当某个 reducer 的迭代器读一个值时，该计数器的值增加。如果所有 reducer 已经处理完所有输入， 则该计数器的值与计数器 “map 输出的记录” 的值相同。 reduce 输出的记录数（REDUCE_OUTPUT_RECORDS） 作业中所有 map 已经产生的 reduce 输出记录数。每当某一个 reducer 的 Context 调用 write() 方法时，该计数器的值增加。 reduce 跳过的组数（REDUCE_SKIPPED_GROUPS） 作业中所有 reducer 已经跳过的不同的码分组的个数。 reduce 跳过的记录数（REDUCE_SKIPPED_RECORDS） 作业中所有 reducer 已经跳过输入记录数。 reduce 经过 shuffle 的字节数（REDUCE_SHUFFLE_BYTES） shuffle 将 map 的输出数据复制到 reducer 中的字节数。 溢出的记录数（SPILLED_RECORDS） 作业中所有 map和reduce 任务溢出到磁盘的记录数。 CPU 毫秒（CPU_MILLISECONDS） 总计的 CPU 时间，以毫秒为单位，由/proc/cpuinfo获取 物理内存字节数（PHYSICAL_MEMORY_BYTES） 一个任务所用物理内存的字节数，由/proc/cpuinfo获取 虚拟内存字节数（VIRTUAL_MEMORY_BYTES） 一个任务所用虚拟内存的字节数，由/proc/cpuinfo获取 有效的堆字节数（COMMITTED_HEAP_BYTES） 在 JVM 中的总有效内存量（以字节为单位），可由 Runtime().getRuntime().totaoMemory()获取。 GC 运行时间毫秒数（GC_TIME_MILLIS） 在任务执行过程中，垃圾收集器（garbage collection）花费的时间（以毫秒为单位）， 可由 GarbageCollector MXBean.getCollectionTime()获取；该计数器并未出现在1.x版本中。 由 shuffle 传输的 map 输出数（SHUFFLED_MAPS） 有 shuffle 传输到 reducer 的 map 输出文件数。 失败的 shuffle 数（SHUFFLE_MAPS） 在 shuffle 过程中，发生拷贝错误的 map 输出文件数，该计数器并没有包含在 1.x 版本中。 被合并的 map 输出数 在 shuffle 过程中，在 reduce 端被合并的 map 输出文件数，该计数器没有包含在 1.x 版本中。 2、文件系统计数器 文件系统计数器的 groupName为org.apache.hadoop.mapreduce.FileSystemCounter，它包含的计数器如下表所示，计数器名称列的括号（）内容即为counterName。 文件系统的读字节数（BYTES_READ） 由 map 和 reduce 等任务在各个文件系统中读取的字节数，各个文件系统分别对应一个计数器，可以是 Local、HDFS、S3和KFS等。 文件系统的写字节数（BYTES_WRITTEN） 由 map 和 reduce 等任务在各个文件系统中写的字节数。 3、FileInputFormat 计数器 FileInputFormat 计数器的 groupName为org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter，它包含的计数器如下表所示，计数器名称列的括号（）内容即为counterName。 读取的字节数（BYTES_READ） 由 map 任务通过 FileInputFormat 读取的字节数。 4、FileOutputFormat 计数器 FileOutputFormat 计数器的 groupName为org.apache.hadoop.mapreduce.lib.input.FileOutputFormatCounter，它包含的计数器如下表所示，计数器名称列的括号（）内容即为counterName。 写的字节数（BYTES_WRITTEN） 由 map 任务（针对仅含 map 的作业）或者 reduce 任务通过 FileOutputFormat 写的字节数。 作业计数器 作业计数器由 JobTracker（或者 YARN 中的应用宿主）维护，因此无需在网络间传输数据，这一点与包括 “用户定义的计数器” 在内的其它计数器不同。这些计数器都是作业级别的统计量，其值不会随着任务运行而改变。 作业计数器计数器的 groupName为org.apache.hadoop.mapreduce.JobCounter，它包含的计数器如下表所示，计数器名称列的括号（）内容即为counterName。 启用的 map 任务数（TOTAL_LAUNCHED_MAPS） 启动的 map 任务数，包括以 “推测执行” 方式启动的任务。 启用的 reduce 任务数（TOTAL_LAUNCHED_REDUCES） 启动的 reduce 任务数，包括以 “推测执行” 方式启动的任务。 失败的 map 任务数（NUM_FAILED_MAPS） 失败的 map 任务数。 失败的 reduce 任务数（NUM_FAILED_REDUCES） 失败的 reduce 任务数。 数据本地化的 map 任务数（DATA_LOCAL_MAPS） 与输入数据在同一节点的 map 任务数。 机架本地化的 map 任务数（RACK_LOCAL_MAPS） 与输入数据在同一机架范围内、但不在同一节点上的 map 任务数。 其它本地化的 map 任务数（OTHER_LOCAL_MAPS） 与输入数据不在同一机架范围内的 map 任务数。由于机架之间的宽带资源相对较少，Hadoop 会尽量让 map 任务靠近输入数据执行，因此该计数器值一般比较小。 map 任务的总运行时间（SLOTS_MILLIS_MAPS） map 任务的总运行时间，单位毫秒。该计数器包括以推测执行方式启动的任务。 reduce 任务的总运行时间（SLOTS_MILLIS_REDUCES） reduce任务的总运行时间，单位毫秒。该值包括以推测执行方式启动的任务。 在保留槽之后，map 任务等待的总时间（FALLOW_SLOTS_MILLIS_MAPS） 在为 map 任务保留槽之后所花费的总等待时间，单位是毫秒。 在保留槽之后，reduce 任务等待的总时间（FALLOW_SLOTS_MILLIS_REDUCES） 在为 reduce 任务保留槽之后，花在等待上的总时间，单位为毫秒 计数器的该如何使用？ 下面我们来介绍如何使用计数器。 1、定义计数器 1)枚举声明计数器 Context context... //自定义枚举变量Enum Counter counter = context.getCounter(Enum enum) 2)自定义计数器 Context context... //自己命名groupName和counterName Counter counter = context.getCounter(String groupName,String counterName) 2、为计数器赋值 1)初始化计数器 counter.setValue(long value);//设置初始值 2)计数器自增 counter.increment(long incr);//增加计数 3、获取计数器的值 1) 获取枚举计数器的值 Job job... job.waitForCompletion(true); Counters counters=job.getCounters(); Counter counter=counters.findCounter("BAD_RECORDS");//查找枚举计数器，假如Enum的变量为BAD_RECORDS long value=counter.getValue();//获取计数值 2) 获取自定义计数器的值 Job job... job.waitForCompletion(true); Counters counters=job.getCounters(); Counter counter=counters.findCounter("ErrorCounter","toolong");//假如groupName为ErrorCounter，counterName为toolong long value=counter.getValue();//获取计数值 3) 获取内置计数器的值 代码 package zhouls.bigdata.myMapReduce.MyCounter; import java.io.IOException; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.conf.Configured; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Counter; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; import org.apache.hadoop.util.GenericOptionsParser; import org.apache.hadoop.util.Tool; import org.apache.hadoop.util.ToolRunner; public class MyCounter extends Configured implements Tool { public static class MyCounterMap extends Mapper <LongWritable, Text, Text, Text> { // 定义枚举对象 public static enum LOG_PROCESSOR_COUNTER {//枚举对象BAD_RECORDS_LONG来统计长数据，枚举对象BAD_RECORDS_SHORT来统计短数据 BAD_RECORDS_LONG,BAD_RECORDS_SHORT }; protected void map(LongWritable key, Text value, Context context) throws java.io.IOException, InterruptedException { String arr_value[] = value.toString().split("/t"); if (arr_value.length > 3) { /*动态自定义计数器*/ context.getCounter("ErrorCounter", "toolong").increment(1); /*枚举声明计数器*/ context.getCounter(LOG_PROCESSOR_COUNTER.BAD_RECORDS_LONG).increment(1); } else if (arr_value.length < 3) { // 动态自定义计数器 context.getCounter("ErrorCounter", "tooshort").increment(1); // 枚举声明计数器 context.getCounter(LOG_PROCESSOR_COUNTER.BAD_RECORDS_SHORT).increment(1); } else { context.write(value, new Text("")); } } } public int run(String[] args) throws Exception { //TODO Auto-generated method stub Configuration conf=new Configuration(); Path mypath=new Path(args[1]); FileSystem hdfs =mypath.getFileSystem(conf); if(hdfs.isDirectory(mypath)) { hdfs.delete(mypath,true); } Job job = new Job(conf, "MyCounter"); job.setJarByClass(MyCounter.class); job.setMapperClass(MyCounterMap.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(Text.class); FileInputFormat.addInputPath(job, new Path(args[0])); FileOutputFormat.setOutputPath(job, new Path(args[1])); job.waitForCompletion(true); return 0; } public static void main(String[] args) throws Exception { // String[] args0 ={"hdfs://HadoopMaster:9000/counter/counter.txt", // "hdfs://HadoopMaster:9000/out/counter"}; String[] args0 ={"./data/counter/counter.txt", "./out/counter"}; int ec = ToolRunner.run(new Configuration(),new MyCounter(),args0); System.exit(ec); } } 本文转自大数据躺过的坑博客园博客，原文链接：http://www.cnblogs.com/zlslch/p/6169221.html，如需转载请自行联系原作者
2017-11-06
优秀的个人博客,低调大师

Android JNI入门第四篇——Android.mk文件分析

Android.mk文件是在使用NDK编译C代码时必须的文件，Android.mk文件中描述了哪些C文件将被编译且指明了如何编译。掌握Android.mk文件的编写主要是掌握其里头将要使用的一些关键字，先来看一个简单的例子，这个例子使用的是android NDK带的 HellJni的例子。 LOCAL_PATH:=$(callmy-dir) include$(CLEAR_VARS) LOCAL_MODULE:=hello-jni LOCAL_SRC_FILES:=hello-jni.c include$(BUILD_SHARED_LIBRARY) LOCAL_PATH 是描述所有要编译的C文件所在的根目录，这边的赋值为$(call my-dir)，代表根目录即为Android.mk所在的目录。 include $(CLEAR_VARS) 代表在使用NDK编译工具时对编译环境中所用到的全局变量清零，如LOCAL_MODULE，LOCAL_SRC_FILES等，因为在一次NDK编译过程中可能会多次调用Android.mk文件，中间用到的全局变量可能是变化的。关于这个问题看了下面比较复杂的例子可能就明白了。 LOCAL_MODULE 是最后生成库时的名字的一部分，给其加上前缀lib和后缀.so就是生成的共享库的名字libhello-jni.so。 LOCAL_SRC_FILES 指明要被编译的c文件的文件名 include $(BUILD_SHARED_LIBRARY) 指明NDK编译时将生成一些共享库 参考： android编译系统makefile(Android.mk)写法 android makefile(android.mk)分析（序） Android.mk的用法和基础 <!-- JiaThis Button BEGIN --> <div id="ckepop"> <a href="http://www.jiathis.com/share" class="jiathis jiathis_txt" target="_blank"><img src="http://v2.jiathis.com/code_mini/images/btn/v1/jiathis1.gif" border="0" /></a> <a class="jiathis_counter_style_margin:3px 0 0 2px"></a> </div> <script type="text/javascript" src="http://v2.jiathis.com/code_mini/jia.js" charset="utf-8"></script> <!-- JiaThis Button END --><!-- JiaThis Button BEGIN --> <div id="ckepop"> <a href="http://www.jiathis.com/share" class="jiathis jiathis_txt" target="_blank"><img src="http://v2.jiathis.com/code_mini/images/btn/v1/jiathis1.gif" border="0" /></a> <a class="jiathis_counter_style_margin:3px 0 0 2px"></a> </div> <script type="text/javascript" src="http://v2.jiathis.com/code_mini/jia.js" charset="utf-8"></script> <!-- JiaThis Button END --> 本文转自xyz_lmn51CTO博客，原文链接：http://blog.51cto.com/xyzlmn/817204，如需转载请自行联系原作者
2017-11-01
优秀的个人博客,低调大师

《Web安全之机器学习入门》一 2.2 TensorFlow简介与环境搭建

2.2 TensorFlow简介与环境搭建 TensorFlow是一个采用数据流图、用于数值计算的开源软件库（见图2-5）。节点在图中表示数学操作，图中的线则表示在节点间相互联系的多维数据数组，即张量。它灵活的架构使你可以在多种平台上展开计算，例如台式计算机中的一个或多个CPU（或GPU）、 服务器、移动设备等等。TensorFlow 最初由Google大脑小组（隶属于Google机器智能研究机构）的研究员和工程师们开发出来，用于机器学习和深度神经网络方面的研究，但这个系统的通用性使其也可广泛用于其他计算领域。 TensorFlow的特点：高度的灵活性；真正的可移植性；将科研和产品联系在一起；自动求微分；多语言支持；性能最优化。安装方法如下： # 仅使用 CPU 的版本 $ pip install https: //storage.go
2017-09-07
优秀的个人博客,低调大师

《Android App开发入门：使用Android Studio 2.X开发环境》——导读

前 言 学习 Android 程序设计一直困扰着许多初学者，原因有两个。首先，必须学会使用 Java 程序设计语言，并且要懂 Android 的 XML词汇，然后才能开始学习 Android 的程序设计。其次，在学习的过程中常见到一些程序设计老手所使用的行话与习惯，初学者常会苦思不得其解，从而导致在学习中产生挫折感、困顿不前。鉴于此，本书针对 Android 的初学者设计了一套学习流程，期望降低初学者学习的门槛，让学习曲线平滑、顺畅，使初学者能迅速掌握 Android 程序设计的重点，而不用浪费过多的时间。 许多人都说学 Android 需要先学 XML，但是事实上学 Android 并不需要先学 XML，而是要学 Android 的 XML词汇。这两者可谓天壤之别。对于前者，你可能要读完一本厚厚的 XML大全集，但是掌握 Andro
2017-08-31

资源下载

更多资源
Nacos

Nacos

Nacos /nɑ:kəʊs/ 是 Dynamic Naming and Configuration Service 的首字母简称，一个易于构建 AI Agent 应用的动态服务发现、配置管理和AI智能体管理平台。Nacos 致力于帮助您发现、配置和管理微服务及AI智能体应用。Nacos 提供了一组简单易用的特性集，帮助您快速实现动态服务发现、服务配置、服务元数据、流量管理。Nacos 帮助您更敏捷和容易地构建、交付和管理微服务平台。

时间: 2026-04-24 大小: 189MB 下载: 6次
Rocky Linux

Rocky Linux

Rocky Linux（中文名：洛基）是由Gregory Kurtzer于2020年12月发起的企业级Linux发行版，作为CentOS稳定版停止维护后与RHEL（Red Hat Enterprise Linux）完全兼容的开源替代方案，由社区拥有并管理，支持x86_64、aarch64等架构。其通过重新编译RHEL源代码提供长期稳定性，采用模块化包装和SELinux安全架构，默认包含GNOME桌面环境及XFS文件系统，支持十年生命周期更新。

时间: 2026-05-30 大小: 1.42GB 下载: 5次
Sublime Text

Sublime Text

Sublime Text具有漂亮的用户界面和强大的功能，例如代码缩略图，Python的插件，代码段等。还可自定义键绑定，菜单和工具栏。Sublime Text 的主要功能包括：拼写检查，书签，完整的 Python API ， Goto 功能，即时项目切换，多选择，多窗口等等。Sublime Text 是一个跨平台的编辑器，同时支持Windows、Linux、Mac OS X等操作系统。

时间: 2025-12-03 大小: 9.84MB 下载: 39次
WebStorm

WebStorm

WebStorm 是jetbrains公司旗下一款JavaScript 开发工具。目前已经被广大中国JS开发者誉为“Web前端开发神器”、“最强大的HTML5编辑器”、“最智能的JavaScript IDE”等。与IntelliJ IDEA同源，继承了IntelliJ IDEA强大的JS部分的功能。

时间: 2025-12-13 大小: 249.54MB 下载: 22次
热门标签
DeepSeek Jdk25 OpenAi HarmonyOS6 Nacos3 SpringBoot4 SpringCloud Docker Kubernetes Service Mesh Redis Gemini3 Rocky

欢迎您来访！

登录后，您将获得更多功能！
热门文章
联系我们

有任何问题或合作意向欢迎联系我们

Email: 99873273@qq.com

QQ: 99873273

用户登录
用户注册