KubeCon: Flink on K8s
目录 集群概况 玩转Flink on Kubernetes WindowJoin SQL 集群概况 BigData Manager简介 BigData on Kubernetes 部署向导 运维管理 开发者IDE 玩转Flink on Kubernetes WindowJoin 简介 Example illustrating a windowed stream join between two data streams.The example works on two input streams with pairs (name, grade) and (name, salary) respectively. It joins the steams based on "name" within a configurable window. The example uses a built-in sample data generator that generates the steams of pairs at a configurable rate. Source1：name, grade Source2：name, salary Result：name, grade, salary 过程 创建Deployment 浏览器打开 0) http://47.92.41.71:8080/app/ 1) http://39.98.106.146:8080/app/ 2) http://39.98.8.131:8080/app/ 3) http://39.98.7.153:8080/app/ 4) http://39.98.106.174:8080/app/ 5) http://39.98.8.106:8080/app/ 6) http://39.98.106.162:8080/app/ 7) http://47.92.44.149:8080/app/ 8) http://39.98.7.241:8080/app/ 9) http://39.98.7.20:8080/app/ 创建Deployment Organization中输入用户名 Configuration： Intepreter选择Blink/JAR Blink Version: 3.2.1 / blink-3.2-SNAPSHOT Jar URI: hdfs:///example/flink-examples-WindowJoin.jar entryClass: org.apache.flink.streaming.examples.join.WindowJoin 点击Create Deployment 启动job 如上图操作 查看结果 点击“Blink UI”，跳转到Apache Flink的dashboard Jobs -> Running Jobs -> Windowed Join Example 结果日志查看：Task Managers -> Path, ID -> Log 停止Job 回到Deployment页面，点击Cancel（集群资源有限，为了后续体验，请一定停掉此job） SQL 简介 实时热门商品， 每隔5分钟输出最近一小时内点击量最多的前 N 个商品(例子详情，请移步http://wuchong.me/blog/2018/11/07/use-flink-calculate-hot-items/ ，天池大赛的数据） 列名称 说明 用户ID 整数类型，加密后的用户ID 商品ID 整数类型，加密后的商品ID 商品类目ID 整数类型，加密后的商品所属类目ID 行为类型 字符串，枚举类型，包括(‘pv’, ‘buy’, ‘cart’, ‘fav’) 时间戳 行为发生的时间戳，单位秒 过程 创建Deployment 创建Deployment Configuration： Intepreter选择Blink/SQL Execution Mode：STREAM Blink Version: 3.2.1 / blink-3.2-SNAPSHOT Artifact：HotItem Runtime Configuration： state.backend.type = rocksdb state.backend.rocksdb.ttl.ms = 129600000 Job信息
