Kylin执行查询语句的时候报错如下：
Error while executing SQL "select t.hotel_id_m,t.live_dt, d.day_of_week,sum(rns) from tableT t join TableD d on t.live_dt = d.daY_no group by t.hotel_id_m,t.live_dt, d.day_of_week LIMIT 50000": <sub-thread for Query ac580b70-96f2-403a-a64a-0557e599d35f GTScanRequest 143ae1ba>The coprocessor thread stopped itself due to scan timeout or scan threshold(check region server log), failing current query...
查看regionserver日志
2017-03-20 11:10:05,436 INFO [Query dc7017bb-fefc-4177-a2c9-5842625beb89-109] endpoint.CubeVisitService: Scanned 9999001 rows from HBase.
2017-03-20 11:10:05,454 INFO [Query dc7017bb-fefc-4177-a2c9-5842625beb89-109] endpoint.CubeVisitService: The cube visit did not finish normally because scan num exceeds threshold
org.apache.kylin.gridtable.GTScanExceedThresholdException: Exceed scan threshold at 10000001
at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService$1.hasNext(CubeVisitService.java:258)
at org.apache.kylin.storage.hbase.cube.v2.HBaseReadonlyStore$1$1.hasNext(HBaseReadonlyStore.java:111)
at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService.visitCube(CubeVisitService.java:290)
at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService.callMethod(CubeVisitProtos.java:4117)
at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7797)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1982)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1964)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33652)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)
2017-03-20 11:10:05,459 WARN [RpcServer.FifoWFPBQ.default.handler=59,queue=5,port=60020] ipc.RpcServer: (responseTooSlow): {"call":"ExecService(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$CoprocessorServiceRequest)","starttimems":1489979376495,"responsesize":359,"method":"ExecService","processingtimems":28964,"client":"10.10.16.102:58720","queuetimems":1,"class":"HRegionServer"}
分析解决：
在${KYLIN_HOME}/conf/kylin.property配置文件中有一个配置项
|
1
|
kylin.query.scan.threshold=10000000
|
这个配置项限制了Hbase scan的行数，当scan超过kylin.query.scan.threshold行的时候还是不能满足查询需求，则kylin取消hbase端的查询。
目前暂时有两种方式解决
1、增加kylin.query.scan.threshold值，这相应也会增加hbase压力，在Hbase能够顶住查询压力的情况下，可以增大该值
2、此次查询语句中有LIMIT 50000，Hbase在scan了kylin.query.scan.threshold行之后，让不能扫描出50000行。可以减小LIMIT的值，在Hbase在scan到达kylin.query.scan.threshold之前，就能满足查询要求。则不会出现这个问题
本文转自巧克力黒 51CTO博客，原文链接：http://blog.51cto.com/10120275/1908363，如需转载请自行联系原作者
