KubeCon: Flink on K8s
目录
- 集群概况
-
玩转Flink on Kubernetes
- WindowJoin
- SQL
集群概况
BigData Manager简介
BigData on Kubernetes
部署向导
运维管理
开发者IDE
玩转Flink on Kubernetes
WindowJoin
简介
Example illustrating a windowed stream join between two data streams.
The example works on two input streams with pairs (name, grade) and (name, salary) respectively. It joins the steams based on "name" within a configurable window. The example uses a built-in sample data generator that generates the steams of pairs at a configurable rate.
- Source1：name, grade
- Source2：name, salary
- Result：name, grade, salary
![image]( "image")
过程
创建Deployment
-
浏览器打开
- 0) http://47.92.41.71:8080/app/
- 1) http://39.98.106.146:8080/app/
- 2) http://39.98.8.131:8080/app/
- 3) http://39.98.7.153:8080/app/
- 4) http://39.98.106.174:8080/app/
- 5) http://39.98.8.106:8080/app/
- 6) http://39.98.106.162:8080/app/
- 7) http://47.92.44.149:8080/app/
- 8) http://39.98.7.241:8080/app/
- 9) http://39.98.7.20:8080/app/
- 创建Deployment
- Organization中输入用户名
-
Configuration：
- Intepreter选择Blink/JAR
- Blink Version: 3.2.1 / blink-3.2-SNAPSHOT
- Jar URI: hdfs:///example/flink-examples-WindowJoin.jar
- entryClass: org.apache.flink.streaming.examples.join.WindowJoin
- 点击Create Deployment
启动job
如上图操作
查看结果
- 点击“Blink UI”，跳转到Apache Flink的dashboard
- Jobs -> Running Jobs -> Windowed Join Example
- 结果日志查看：Task Managers -> Path, ID -> Log
停止Job
- 回到Deployment页面，点击Cancel（集群资源有限，为了后续体验，请一定停掉此job）
SQL
简介
实时热门商品， 每隔5分钟输出最近一小时内点击量最多的前 N 个商品(例子详情，请移步http://wuchong.me/blog/2018/11/07/use-flink-calculate-hot-items/ ，天池大赛的数据）
| 列名称 | 说明 |
|---|---|
| 用户ID | 整数类型，加密后的用户ID |
| 商品ID | 整数类型，加密后的商品ID |
| 商品类目ID | 整数类型，加密后的商品所属类目ID |
| 行为类型 | 字符串，枚举类型，包括(‘pv’, ‘buy’, ‘cart’, ‘fav’) |
| 时间戳 | 行为发生的时间戳，单位秒 |
过程
创建Deployment
- 创建Deployment
-
Configuration：
- Intepreter选择Blink/SQL
- Execution Mode：STREAM
- Blink Version: 3.2.1 / blink-3.2-SNAPSHOT
- Artifact：HotItem
-
Runtime Configuration：
- state.backend.type = rocksdb
- state.backend.rocksdb.ttl.ms = 129600000
Job信息
