How to configue session timeout in Hive-低调大师

How to configue session timeout in Hive

2016-01-11 636

This article explains how to configure the following settings in Hive:

hive.server2.session.check.interval
hive.server2.idle.operation.timeout

hive.server2.idle.session.timeout

1). hive.server2.idle.session.timeout
Session will be closed when not accessed for this duration of time, in milliseconds; disable by setting to zero or a negative value.
For example, the value of “86400000” indicate that the session will be timed out after 1 day of inactivity.

2). hive.server2.session.check.interval
The check interval for session/operation timeout, in milliseconds, which can be disabled by setting to zero or a negative value.
For example, the value of “3600000” indicate that the session will be checked every 1 hour.

3) hive.server2.idle.operation.timeout
Operation will be closed when not accessed for this duration of time, in milliseconds; disable by setting to zero. For a positive value, checked for operations in terminal state only (FINISHED, CANCELED, CLOSED, ERROR). For a negative value, checked for all of the operations regardless of state.
For example, the value of “7200000” indicate that the query/operation will be timed out after 2 hours if it is still running.

So if you combine the three settings from above examples, we can summarize the following use cases:

1) If you started a HS2 session, beeline for example, and without doing anything afterwards, HS2 will trigger 24 session checking before it determines that 24 hours has passed since last activity, then session will be closed

2) If you worked on beeline for 2 hours and then leave the beeline open without doing anything afterwards, HS2 will trigger total of 26 session checking (2 while you worked and another 24 while in idle), and the session will be closed 26 hours after initially opened.

3) If you worked on beeline for 2 hours, and you started running a query that will run for 1 hour and then returns result, the idle timer actually starts from the time when data returns, so if you don’t do anything afterwards, HS2 will kill the session after another 24 hours, so in total, the session lasted 27 hours (2+1+24)

4) If, for instance, your query takes longer than the 2 hours defined by hive.server2.idle.operation.timeout, then the query will be cancelled, hence you will lose the result all together

5) If hive.server2.session.check.interval = 0 and hive.server2.idle.session.timeout > 0, then it will have the same effect that hive.server2.idle.session.timeout = 0, because no idle timer will be triggered as check interval is disabled

6) If hive.server2.session.check.interval > hive.server2.idle.session.timeout > 0, then the actual time out will be the same as hive.server2.session.check.interval.

For example if hive.server2.session.check.interval = 20 minutes, but hive.server2.idle.session.timeout = 10 minutes, then timeout will happen when checking happens, which is 20 minutes.

The basic rule of thumb is as follows:
hive.server2.session.check.interval < hive.server2.idle.operation.timeout < hive.server2.idle.session.timeout

And the recommended values are:

hive.server2.session.check.interval = 1 hour
hive.server2.idle.operation.timeout = 1 day
hive.server2.idle.session.timeout = 3 days

This will work for most of clusters, but you can change the value depending on how your cluster behaves and how long each query run for.

微信关注我们

原文链接：https://yq.aliyun.com/articles/238587

转载内容版权归作者及来源网站所有！

低调大师中文资讯倾力打造互联网数据资讯、行业资源、电子商务、移动互联网、网络营销平台。持续更新报道IT业界、互联网、市场资讯、驱动更新,是最及时权威的产业资讯及硬件资讯报道平台。

storm的数据源编程单元Spout学习整理

Spout呢，是Topology中数据流的源头，也是Storm针对数据源的编程单元。一般数据的来源，是通过外部数据源来读取数据项(Tuple)，并读取的数据项传输至作业的其他组件。编程人员一般可通过OutputFieldsDeclarer类的declareStream()方法来声明多个流，指定数据将要发送的流，然后使用SpoutOutputCollector的emit方法将数据发送。这里整理了下ISpout和IComponent接口。 ISpout声明了Spout的核心方法，用于向Topology供给数据项。对于每一个发出的数据项，Storm通过Spout，可以追踪它经历处理过程的有向无环图（竟然也是DAG)。 void open (java,util.Map conf,TopologyContext context,SpoutOutputCollector collector) 用于实例化Spout的一个运行时任务，被急群众的某一进程调用 (conf对象维护Storm中针对该Spout的配置信息，context是一个上下文对象，可用于获取该组件运行时任务的信息,collector用...

2016-01-12

648

实现的方法中，抛的异常只能比父类或接口中的少 import java.io.IOException; public interface IHello { void say(String msg) throws TestException, IOException; } class IHelloImpl implements IHello { @Override public void say(String msg) throws IOException { System.out.println("msg = [" + msg + "]"); } } 类StormyInning既扩展了基类Inning，又实现了接口Storm，而且基类Inning和接口Storm都声明了方法event()： package com.liush.chapter12; //Overridden methods may throw only the exceptions specified in their //base-class versions,or exceptions derived from ...

2016-01-13

599

资源下载

更多资源

Mario

马里奥是站在游戏界顶峰的超人气多面角色。马里奥靠吃蘑菇成长，特征是大鼻子、头戴帽子、身穿背带裤，还留着胡子。与他的双胞胎兄弟路易基一起，长年担任任天堂的招牌角色。

Nacos

Nacos /nɑ:kəʊs/ 是 Dynamic Naming and Configuration Service 的首字母简称，一个易于构建 AI Agent 应用的动态服务发现、配置管理和AI智能体管理平台。Nacos 致力于帮助您发现、配置和管理微服务及AI智能体应用。Nacos 提供了一组简单易用的特性集，帮助您快速实现动态服务发现、服务配置、服务元数据、流量管理。Nacos 帮助您更敏捷和容易地构建、交付和管理微服务平台。

Rocky Linux

Rocky Linux（中文名：洛基）是由Gregory Kurtzer于2020年12月发起的企业级Linux发行版，作为CentOS稳定版停止维护后与RHEL（Red Hat Enterprise Linux）完全兼容的开源替代方案，由社区拥有并管理，支持x86_64、aarch64等架构。其通过重新编译RHEL源代码提供长期稳定性，采用模块化包装和SELinux安全架构，默认包含GNOME桌面环境及XFS文件系统，支持十年生命周期更新。

Sublime Text

Sublime Text具有漂亮的用户界面和强大的功能，例如代码缩略图，Python的插件，代码段等。还可自定义键绑定，菜单和工具栏。Sublime Text 的主要功能包括：拼写检查，书签，完整的 Python API ， Goto 功能，即时项目切换，多选择，多窗口等等。Sublime Text 是一个跨平台的编辑器，同时支持Windows、Linux、Mac OS X等操作系统。