首页 文章 精选 留言 我的

精选列表

搜索[学习],共10000篇文章
优秀的个人博客,低调大师

大规模深度学习预测场景下 codegen 的思考与应用

更多关于 RTP 系统的介绍请见 深度预测平台RTP介绍 背景简介 RTP 系统 RTP 系统(即 Rank Service),是一个面向搜索和推荐的 ranking 需求,支持多种模型的在线 inference 服务。RTP 支持 LR、GBDT 以及 tensorflow 等多种模型及模型格式,并依托 suez 在线服务框架,将样本组装和模型预测一气呵成,提升了业务迭代效率,并在性能和稳定性方面予业务以充分保障。 inference 计算 RTP 上的业务大多数是打分类的场景,它的计算流程由一个 Compute Graph 描述(TF Graph)。从计算逻辑上来看,它可以分为这样三部分: 存储读取。根据商品 ID,利用 suez 存储能力,join 读取 Item 字段内容;获取 Query 和 User 信息。 特征生成。根据上一步的原

优秀的个人博客,低调大师

双11知识狂欢,一天不学习,浑身都难受

双11的活动已经开始啦 去年剁掉的手 今天竟然神奇般的长出来啦,哈哈哈 苦哈哈赚了一年的钱 不就是为了这几天的放纵嘛 绝对不能错过这囤货的最佳时期哟~ 各种漂亮衣服、生活用品、家具用品,相信大家已经囤了满满一堆 但是物品总有不喜欢的一天,而生活“物资”也会在一天天减少 难免会觉得空虚与落寞 那么,总会有一些东西不会随着时间的流逝而贬值 反而会充实你的大脑,让你的薪资蹭蹭涨起来! 它,就是龙果学院——IT精品课程! 那么,怎样才能买到最适合你的课程呢? 如何才能花最少的钱,增最多的值呢? 请收好这份攻略 活动多样化 攻略一:精品课程乐享直降盛惠 购买热门精品课程,即可以享受立减哦!大数据课程立减高达1020元,微服务架构课程立减高达900元! 深入大数据架构师之路,问鼎40万年薪 原价:1699元 11.9-12日下单立减1020元 双11狂欢价:679 基于Flink流处理的动态实时电商实时分析系统 原价:699元 11.9-12日下单立减400元 双11狂欢价:299元 大数据DMP画像系统 原价:699元 11.9-12日下单立减200元 双11狂欢价:499元 从无到有构建大型电商微服务架构(第二阶段) 原价:899元 11.9-12日下单立减540元 双11狂欢价:359元 分布式架构课程-组合套餐 原价:1699元 11.9-12日下单立减900元 双11狂欢价:799元 更多优惠详情:https://www.roncoo.com/activity/lj.html 攻略二:11月5日-12日每天6场秒杀 10:00 12:00 16:00 18:00 20:00 22:00 “套路”大概就是这样啦!接下来,贴心的小编为大家总结了一些热门的课程,大家可以参考一波~ 大数据精品课程 微服务架构热门课程

优秀的个人博客,低调大师

go微服务框架go-micro深度学习(二) 入门例子

上一篇帖子简单介绍了go-micro的整体框架结构,这一篇主要写go-micro使用方式的例子,中间会穿插一些go-micro的源码,和调用流程图,帮大家更好的理解go-micro的底层。更详细更具体的调用流程和细节,会在以后的帖子里详细讲解。 例子的github地址:gomicrorpc 跑一遍例子,也就会明白个大概。 安装所需要的环境 go-micro服务发现默认使用的是consul, brew install consul consul agent -dev 或者直接使用使用docker跑 docker run -p 8300:8300 -p 8301:8301 -p 8301:8301/udp -p 8302:8302/udp -p 8302:8302 -p 8400:8400 -p 8500:8500 -p 53:53/udp consul 我个人更喜欢etcdv3原因我上一篇也有提到过,gomicro服务发现不支持consul集群,我之前也写过etcdv3 集群的搭建和使用帖子,有时间大家可以看一下 安装go-micro框架 go get github.com/micro/go-micro 安装protobuf和依赖 prtobuf的基础知识我这里就不讲了,如果不了解的可以看一下官方文档,就是一个跨平台,跨语言的数据序列化库,简单易学。 是go-micro用于帮助我们生成服务接口和一系列的调用代码 brew install protobuf go get -u -v github.com/golang/protobuf/{proto,protoc-gen-go} go get -u -v github.com/micro/protoc-gen-micro protobuf也可以直接从源码安装 wget https://github.com/protocolbuffers/protobuf/releases/download/v3.6.1/protobuf-all-3.6.1.tar.gz tar zxvf protobuf-all-3.6.1.tar.gz cd protobuf-3.6.1/ ./autogen.sh ./configure make make install protoc -h 安装micro工具包,这个安装是可选项,micro提供了一系列的工具来帮助我们更好的使用go-micro。 go get github.com/micro/micro 例子1 创建proto文件common.proto,这个文件包含了传入和返回的参数,参数包含了常用的基础类型、数组、map等。还有一个Say 服务,这个服务里有一个rpc方法。 syntax = "proto3"; package model; message SayParam { string msg = 1; } message Pair { int32 key = 1; string values = 2; } message SayResponse { string msg = 1; // 数组 repeated string values = 2; // map map<string, Pair> header = 3; RespType type = 4; } enum RespType { NONE = 0; ASCEND = 1; DESCEND = 2; } // 服务接口 service Say { rpc Hello(SayParam) returns (SayResponse) {} } 在根目录下运行,生成两个模板文件 protoc --proto_path=$GOPATH/src:. --micro_out=. --go_out=. example1/proto/*.proto 一个文件是proto的go 结构文件,还有一个go-micro rpc的接口文件。 server 端: type Say struct {} func (s *Say) Hello(ctx context.Context, req *model.SayParam, rsp *model.SayResponse) error { fmt.Println("received", req.Msg) rsp.Header = make(map[string]*model.Pair) rsp.Header["name"] = &model.Pair{Key: 1, Values: "abc"} rsp.Msg = "hello world" rsp.Values = append(rsp.Values, "a", "b") rsp.Type = model.RespType_DESCEND return nil } func main() { // 我这里用的etcd 做为服务发现,如果使用consul可以去掉 reg := etcdv3.NewRegistry(func(op *registry.Options){ op.Addrs = []string{ "http://192.168.3.34:2379", "http://192.168.3.18:2379", "http://192.168.3.110:2379", } }) // 初始化服务 service := micro.NewService( micro.Name("lp.srv.eg1"), micro.Registry(reg), ) service.Init() // 注册 Handler model.RegisterSayHandler(service.Server(), new(Say)) // run server if err := service.Run(); err != nil { panic(err) } } 服务发现我使用的是etcdv3 替换了默认的consul micro.NewService 初始化服务,然后返回一个Service接口的实例,newService()方法的大概流程如下, 先是给各个接口初始化默认值,再使用传入的值替换默认值,这也是go-micro可替换插件的地方。 service有一个Init()可选方法,这是一个单例方法, func (s *service) Init(opts ...Option) { // process options for _, o := range opts { o(&s.opts) } s.once.Do(func() { // save user action action := s.opts.Cmd.App().Action // set service action s.opts.Cmd.App().Action = func(c *cli.Context) { .........//这里就不把代码全显示出来了 ......... } } 用于始化cmd的一些信息 service.Run()方法 调用流程 因为在初始化的时候没有指定端口,系统会自动分配一个端口号分给Server,并把这个server的信息注册到Register。 BeferStart和AfterStart也都是可以自定义的 client 端: func main() { // 我这里用的etcd 做为服务发现,如果使用consul可以去掉 reg := etcdv3.NewRegistry(func(op *registry.Options){ op.Addrs = []string{ "http://192.168.3.34:2379", "http://192.168.3.18:2379", "http://192.168.3.110:2379", } }) // 初始化服务 service := micro.NewService( micro.Registry(reg), ) service.Init() sayClent := model.NewSayService("lp.srv.eg1", service.Client()) rsp, err := sayClent.Hello(context.Background(), &model.SayParam{Msg: "hello server"}) if err != nil { panic(err) } fmt.Println(rsp) } 上面根据proto文件的生成的两个文件中有一个是rpc的接口文件,接口文件已经帮我们把调用方法的整个流程封装好了。 只需要给出服务名称和licent就可以。然后调用Hello方法 源码: func (c *sayService) Hello(ctx context.Context, in *SayParam, opts ...client.CallOption) (*SayResponse, error) { req := c.c.NewRequest(c.name, "Say.Hello", in) out := new(SayResponse) err := c.c.Call(ctx, req, out, opts...) if err != nil { return nil, err } return out, nil } 主要的流程里都在c.c.Call方法里。简单来说流程如下 就是得到节点信息address,根据address去查询 pool里是否有连接,如果有则取出来,如果没有则创建,然后进行数据传输,传输完成后把client放回到pool内。pool的大小也是可以控制的,这部分的代码读起来特别爽,具体的细节和处理流程会在以后的帖子里详细讲解 例子2 例子1,做了一个简单的服务,已经不能再简单了,只是为了能让大家熟悉一下go-micro。看完例子1后应该会有更多的想法,想使用更多的go-micro的功能,比如protobuf生成的类都在一起,如果想model和api分开怎么处理,怎么使用go-micro的双向流,怎么使用消息推送,等等。所以我就双做了一个小例子,这个例子里包含了一些东西。 这个例子我就只说一下组织结构,也没有多少代码,大家有时间看一下就ok了。 proto下的两个文件夹,一个model一个rpcapi,是把数据和api分开,api引用了model 看一下rpcapi syntax = "proto3"; package rpcapi; import "github.com/lpxxn/gomicrorpc/example2/proto/model/common.proto"; // 服务接口 service Say { rpc Hello(model.SayParam) returns (model.SayResponse) {} rpc Stream(model.SRequest) returns (stream model.SResponse) {} } import了model里的common.proto 在生成的时候一个只要go_out另一个只要micro_out就好了 protoc --proto_path=$GOPATH/src:. --go_out=. example2/proto/model/*.proto protoc --proto_path=$GOPATH/src:. --micro_out=. example2/proto/rpcapi/*.proto 订阅一个信息 // Register Subscribers if err := server.Subscribe(server.NewSubscriber(common.Topic1, subscriber.Handler)); err != nil { panic(err) } 当有信息发送时,所有订阅了lp.srv.eg2.topic1这个信息的服务都会收到信息 客户端发送信息 p := micro.NewPublisher(common.Topic1, service.Client()) p.Publish(context.TODO(), &model.SayParam{Msg: lib.RandomStr(lib.Random(3, 10))}) 如果是生产环境一定不要用go-micro默认的信息发布和订阅处理方式,micro的插件plugin里是有很多成熟的插件。 使用双向流的小功能 这个方法只是每次向客户端发送一些数据,每次只发送一部分。比如我们给客户端推送的数据很大时,一次性全都推过去,是不太正确的做法,分批推送还是比较好的。 func (s *Say) Stream(ctx context.Context, req *model.SRequest, stream rpcapi.Say_StreamStream) error { for i := 0; i < int(req.Count); i++ { rsp := &model.SResponse{} for j := lib.Random(3, 5); j < 10; j++ { rsp.Value = append(rsp.Value, lib.RandomStr(lib.Random(3, 10))) } if err := stream.Send(rsp); err != nil { return err } // 模拟处理过程 time.Sleep(time.Microsecond * 50) } return nil return nil } 希望这个小例子能让大家入门go-micro.

优秀的个人博客,低调大师

SQL Serever学习16——索引,触发器,数据库维护

sqlserver2014数据库应用技术 《清华大学出版社》 索引 这是一个很重要的概念,我们知道数据在计算机中其实是分页存储的,就像是单词存在字典中一样 数据库索引可以帮助我们快速定位数据在哪个存储页区,而不用扫描整个数据库 索引一旦被创建就会数据库自动管理和维护,增删改插座数据库都会对索引做修改 索引分类: 聚集索引 非聚集索引 包含性列索引 索引视图 全文索引 xml索引 聚集索引,就是相当于排序的字典(将表中的数据完全重新排序),一个表只有一个,所占空间相当于表中数据的120%,数据建立聚集索引,会改变数据行的存储物理结构 非聚集索引,不改变数据行的物理存储结构,CREATE INDEX默认建立非聚集索引,理论一个表可以有249个非聚集索引 索引和约束 设置主键,会自动创建PRIMARY KEY 和创建一个聚集索引 创建UNIQUE 约束会自动创建一个唯一非聚集索引 创建表的索引 使用SQL语句 CREATE INDEX IX_name_mj ON 买家表(买家名称) GO 查看索引 EXEC sp_helpindex 买家表 分析索引 查看查询计划,使用的索引(优先使用聚集索引) SET SHOWPLAN_ALL ON GO SELECT * FROM 买家表 GO SET SHOWPLAN_ALL OFF GO 显示统计信息,查看所花费的磁盘io活动量 SET STATISTICS IO ON GO SELECT * FROM 买家表 GO SET STATISTICS IO OFF GO 维护索引 数据表的增删改操作会产生大量索引碎片,索引表不连续,降低索引性能,需要整理索引 查看索引碎片SQL DBCC SHOWCONTIG(买家表,PK_买家表) GO ssms查看索引 索引碎片整理 DBCC INDEXDEFRAG(销售管理,买家表,PK_买家表) 触发器 触发器是一个高级的数据约束,他是特殊的存储过程,不能通过执行sql触发,由增删改等事件自动触发 sqlserver2014提供3种触发器: DML触发器,包括事后触发器,替代触发器,CLR运行时触发器 DDL触发器,修改表结构触发 LOGIN触发器,登录的时候触发 DML触发器 INSERT触发器 如果员工年龄不到18岁不执行插入操作 CREATE TRIGGER Employee_Insert ON Employee AFTER INSERT AS BEGIN --从INSERTED表获取新插入员工的出生年月 DECLARE @birthday date SELECT @birthday=birthday FROM inserted --判断新员工年龄 IF(YEAR(GETDATE())-YEAR(@birthday)<18) BEGIN PRINT '该员工年龄不到18岁,不能入职!' ROLLBACK TRANSACTION --回滚这个节点之前的所有操作,然后继续执行后面的语句 END END 验证 INSERT Employee VALUES('小明','2012-10-10') 再验证 INSERT Employee VALUES('小明','1912-10-10') 注意主键id仍然会增长,即使刚刚的操作回滚了,id还是增加了1 UPDATE触发器 防止用户修改员工姓名name字段 CREATE TRIGGER Employee_Update ON Employee AFTER UPDATE AS BEGIN IF(UPDATE(NAME)) BEGIN PRINT '禁止修改员工姓名!' ROLLBACK TRANSACTION --回滚这个节点之前的所有操作,然后继续执行后面的语句 END END 验证 UPDATE Employee SET NAME='XX' 数据库的维护 备份 使用存储过程创建备份设备 EXEC sp_addumpdevice 'DISK','COMB','E"\DATA\COMB.BAK' 删除备份设备 EXEC sp_dropdevice 'COMB' 使用SQL创建数据库备份 BACKUP DATABASE 销售管理 TO COMB 使用SQL还原数据库 RESTORE DATABASE COMB FROM DISK='E:/DATA/COMB.BAK'

优秀的个人博客,低调大师

docker学习系列14 使用haproxy实现mysql集群的负载均衡

在上节中我们创建了 mysql 集群。 实际工作中,我们不希望让某一数据库节点处理所有的请求,这样的话单个负载高,性能差。 image.png 在这里我们使用haproxy作为负载均衡的中间件,类似的还有LVS,但是好像不支持虚拟机,在docker中用不了。 image.png 实现流程: 下载镜像 docker pull haproxy 宿主机创建 haproxy 的配置文件,比如路径是 D:\Docker\haproxy\haproxy.cfg 最重要的就是配置文件了。这里内容如下: global daemon # nbproc 1 # pidfile /var/run/haproxy.pid # 工作目录 chroot /usr/local/etc/haproxy defaults log 127.0.0.1 local0 err #[err warning info debug] mode http #默认的模式mode { tcp|http|health },tcp是4层,http是7层,health只会返回OK retries 2 #两次连接失败就认为是服务器不可用,也可以通过后面设置 option redispatch #当serverId对应的服务器挂掉后,强制定向到其他健康的服务器 option abortonclose #当服务器负载很高的时候,自动结束掉当前队列处理比较久的链接 option dontlognull #日志中不记录负载均衡的心跳检测记录 maxconn 4096 #默认的最大连接数 timeout connect 5000ms #连接超时 timeout client 30000ms #客户端超时 timeout server 30000ms #服务器超时 #timeout check 2000 #=心跳检测超时 ######## 监控界面配置 ################# listen admin_status # 监控界面访问信息 bind 0.0.0.0:8888 mode http # URI相对地址 stats uri /dbs # 统计报告格式 stats realm Global\ statistics # 登录账户信息 stats auth admin:123456 ########frontend配置############## ######## mysql负载均衡配置 ############### listen proxy-mysql bind 0.0.0.0:3306 mode tcp # 负载均衡算法 # static-rr 权重, leastconn 最少连接, source 请求IP, 轮询 roundrobin balance roundrobin # 日志格式 option tcplog # 在 mysql 创建一个没有权限的haproxy用户,密码为空。 haproxy用户 # create user 'haproxy'@'%' identified by ''; FLUSH PRIVILEGES; option mysql-check user haproxy # 这里是容器中的IP地址,由于配置的是轮询roundrobin,weight 权重其实没有生效 server MYSQL_1 172.18.0.2:3306 check weight 1 maxconn 2000 server MYSQL_2 172.18.0.3:3306 check weight 1 maxconn 2000 server MYSQL_3 172.18.0.4:3306 check weight 1 maxconn 2000 # 使用keepalive检测死链 # option tcpka ######################################### 启动 haproxy 的容器,镜像名称为 h1,网络名称使用上节中创建的 pxc-network,就是和 mysql 集群处于同一网络。docker run -it -d -p 4001:8888 -p 4002:3306 -v D:/Docker/haproxy:/usr/local/etc/haproxy --name h1 --net=pxc-network 进去容器,并让 haproxy 加载配置docker exec -it h1 bashhaproxy -f /usr/local/etc/haproxy/ 宿主机打开 http://localhost:4001/dbs 这是haproxy 提供的图形界面 image.png 可以看到每个mysql节点运行状态是绿色,说明正常。 测试,停掉一个数据库节点 docker stop pxc_node1 ,发现有一个变红了。 image.png 项目中可以使用配置的 4002 来连接数据库,这样请求会被分发到各个子节点。 总结: 数据库的负载均衡配置还是比较简单的,关键是负载均衡算法,如果每个数据库节点配置都一样,可以使用轮询算法,如果不一样,可以使用权重算法,让配置高的多接收请求。 官方的教程 问题: 启动停掉的节点 docker start pxc_node1,mysql恢复不起来了。待研究。有谁知道麻烦告诉我。 image.png

优秀的个人博客,低调大师

[雪峰磁针石博客]python机器学习、web开发等书籍汇总

Learning Python Web Penetration - Christian Martorella - 2018.pdf 下载地址 Leverage the simplicity of Python and available libraries to build web security testing tools for your application Key Features Understand the web application penetration testing methodology and toolkit using Python Write a web crawler/spider with the Scrapy library Detect and exploit SQL injection vulnerabilities by creating a script all by yourself Book Description Web penetration testing is the use of tools and code to attack a website or web app in order to assess its vulnerability to external threats. While there are an increasing number of sophisticated, ready-made tools to scan systems for vulnerabilities, the use of Python allows you to write system-specific scripts, or alter and extend existing testing tools to find, exploit, and record as many security weaknesses as possible. Learning Python Web Penetration Testing will walk you through the web application penetration testing methodology, showing you how to write your own tools with Python for each activity throughout the process. The book begins by emphasizing the importance of knowing how to write your own tools with Python for web application penetration testing. You will then learn to interact with a web application using Python, understand the anatomy of an HTTP request, URL, headers and message body, and later create a script to perform a request, and interpret the response and its headers. As you make your way through the book, you will write a web crawler using Python and the Scrappy library. The book will also help you to develop a tool to perform brute force attacks in different parts of the web application. You will then discover more on detecting and exploiting SQL injection vulnerabilities. By the end of this book, you will have successfully created an HTTP proxy based on the mitmproxy tool. What you will learn Interact with a web application using the Python and Requests libraries Create a basic web application crawler and make it recursive Develop a brute force tool to discover and enumerate resources such as files and directories Explore different authentication methods commonly used in web applications Enumerate table names from a database using SQL injection Understand the web application penetration testing methodology and toolkit Who this book is for Learning Python Web Penetration Testing is for web developers who want to step into the world of web application security testing. Basic knowledge of Python is necessary. Table of Contents Introduction to Web Application Penetration Testing Interacting with Web Applications Web Crawling with Scrapy – Mapping the Application Discovering resources Password Testing Detecting and Exploiting SQL Injection Vulnerabilities Intercepting HTTP Requests Python For Offensive PenTest(conv) - 2018.pdf 下载地址 Key Features Comprehensive information on building a web application penetration testing framework using Python Master web application penetration testing using the multi-paradigm programming language Python Detect vulnerabilities in a system or application by writing your own Python scripts Book Description Python is an easy-to-learn and cross-platform programming language that has unlimited third-party libraries. Plenty of open source hacking tools are written in Python, which can be easily integrated within your script. This book is packed with step-by-step instructions and working examples to make you a skilled penetration tester. It is divided into clear bite-sized chunks, so you can learn at your own pace and focus on the areas of most interest to you. This book will teach you how to code a reverse shell and build an anonymous shell. You will also learn how to hack passwords and perform a privilege escalation on Windows with practical examples. You will set up your own virtual hacking environment in VirtualBox, which will help you run multiple operating systems for your testing environment. By the end of this book, you will have learned how to code your own scripts and mastered ethical hacking from scratch. What you will learn Code your own reverse shell (TCP and HTTP) Create your own anonymous shell by interacting with Twitter, Google Forms, and SourceForge Replicate Metasploit features and build an advanced shell Hack passwords using multiple techniques (API hooking, keyloggers, and clipboard hijacking) Exfiltrate data from your target Add encryption (AES, RSA, and XOR) to your shell to learn how cryptography is being abused by malware Discover privilege escalation on Windows with practical examples Countermeasures against most attacks Who This Book Is For This book is for ethical hackers; penetration testers; students preparing for OSCP, OSCE, GPEN, GXPN, and CEH; information security professionals; cybersecurity consultants; system and network security administrators; and programmers who are keen on learning all about penetration testing. Table of Contents Warming up - Your First Anti-Virus Free Persistence Shell Advanced Scriptable Shell Passwords Hacking Catch Me If You Can! Miscellaneous Fun in Windows Abuse of cryptography by malware Cracking Codes with Python - 2018.epub 下载地址 Introduction Chapter 1 - Making Paper Cryptography Tools Chapter 2 -Programming in the Interactive Shell Chapter 3 - Strings and Writing Programs Chapter 4 - The Reverse Cipher Chapter 5 - The Caesar Cipher Chapter 6 - Hacking the Caesar Cipher with Brute-Force Chapter 7 - Encrypting with the Transposition Cipher Chapter 8 - Decrypting with the Transposition Cipher Chapter 9 - Programming a Program to Test Your Program Chapter 10 - Encrypting and Decrypting Files Chapter 11 - Detecting English Programmatically Chapter 12 - Hacking the Transposition Cipher Chapter 13 - A Modular Arithmetic Module for the Affine Cipher Chapter 14 - Programming the Affine Cipher Chapter 15 - Hacking the Affine Cipher Chapter 16 - Programming the Simple Substitution Cipher Chapter 17 - Hacking the Simple Substitution Cipher Chapter 18 - Programming the Vigenere Cipher Chapter 19 - Frequency Analysis Chapter 20 - Hacking the Vigenere Cipher Chapter 21 - The One-Time Pad Cipher Chapter 22 - Finding and Generating Prime Numbers Chapter 23 - Generating Keys for the Public Key Cipher Chapter 24 - Programming the Public Key Cipher Pandas for Everyone Python Data Analysis -2018.pdf 下载地址 Copyright 2018 Dimensions: 7" x 9-1/8" Pages: 416 Edition: 1st Book ISBN-10: 0-13-454693-8 ISBN-13: 978-0-13-454693-3 The Hands-On, Example-Rich Introduction to Pandas Data Analysis in Python Today, analysts must manage data characterized by extraordinary variety, velocity, and volume. Using the open source Pandas library, you can use Python to rapidly automate and perform virtually any data analysis task, no matter how large or complex. Pandas can help you ensure the veracity of your data, visualize it for effective decision-making, and reliably reproduce analyses across multiple datasets. Pandas for Everyone brings together practical knowledge and insight for solving real problems with Pandas, even if you’re new to Python data analysis. Daniel Y. Chen introduces key concepts through simple but practical examples, incrementally building on them to solve more difficult, real-world problems. Chen gives you a jumpstart on using Pandas with a realistic dataset and covers combining datasets, handling missing data, and structuring datasets for easier analysis and visualization. He demonstrates powerful data cleaning techniques, from basic string manipulation to applying functions simultaneously across dataframes. Once your data is ready, Chen guides you through fitting models for prediction, clustering, inference, and exploration. He provides tips on performance and scalability, and introduces you to the wider Python data analysis ecosystem. Work with DataFrames and Series, and import or export data Create plots with matplotlib, seaborn, and pandas Combine datasets and handle missing data Reshape, tidy, and clean datasets so they’re easier to work with Convert data types and manipulate text strings Apply functions to scale data manipulations Aggregate, transform, and filter large datasets with groupby Leverage Pandas’ advanced date and time capabilities Fit linear models using statsmodels and scikit-learn libraries Use generalized linear modeling to fit models with different response variables Compare multiple models to select the “best” Regularize to overcome overfitting and improve performance Use clustering in unsupervised machine learning Building Machine Learning Systems with Python Third Edition - 2018.pdf 下载地址 Get more from your data by creating practical machine learning systems with Python Key Features Develop your own Python-based machine learning system Discover how Python offers multiple algorithms for modern machine learning systems Explore key Python machine learning libraries to implement in your projects Book Description Machine learning allows systems to learn things without being explicitly programmed to do so. Python is one of the most popular languages used to develop machine learning applications, which take advantage of its extensive library support. This third edition of Building Machine Learning Systems with Python addresses recent developments in the field by covering the most-used datasets and libraries to help you build practical machine learning systems. Using machine learning to gain deeper insights from data is a key skill required by modern application developers and analysts alike. Python, being a dynamic language, allows for fast exploration and experimentation. This book shows you exactly how to find patterns in your raw data. You will start by brushing up on your Python machine learning knowledge and being introduced to libraries. You'll quickly get to grips with serious, real-world projects on datasets, using modeling and creating recommendation systems. With Building Machine Learning Systems with Python, you'll gain the tools and understanding required to build your own systems, all tailored to solve real-world data analysis problems. By the end of this book, you will be able to build machine learning systems using techniques and methodologies such as classification, sentiment analysis, computer vision, reinforcement learning, and neural networks. What you will learn Build a classification system that can be applied to text, images, and sound Employ Amazon Web Services (AWS) to run analysis on the cloud Solve problems related to regression using scikit-learn and TensorFlow Recommend products to users based on their past purchases Understand different ways to apply deep neural networks on structured data Address recent developments in the field of computer vision and reinforcement learning Who this book is for Building Machine Learning Systems with Python is for data scientists, machine learning developers, and Python developers who want to learn how to build increasingly complex machine learning systems. You will use Python's machine learning capabilities to develop effective solutions. Prior knowledge of Python programming is expected. MicroPython for BBC micro:bit Technical Workshop - 2018 pdf 下载地址 BBC micro:bit is a development board to learn embedded system easily. This book is designed to help you to get started with BBC micro:bit development using MicroPython platform. The following is a list of highlight content in this book. * Development environment preparation * Set up MicroPython on BBC micro:bit Board * Display Programming * BBC micro:bit GPIO * Reading Analog Input and PWM * Working with SPI * Working with I2C * Working with Accelerator and Compass Sensors Django - The Easy Way pdf - 2017 PDF 下载地址 Django is a very powerful Python Web Framework. You can use it to build everything from simple websites to big high traffic systems. But starting with Django can be a daunting experience for beginners. The purpose of this book is to guide you through the essential concepts with pragmatic step-by-step examples. You will learn how to build a complete website and deploy it in a real world production environment. The focus is on Django basic concepts so covering other technologies is kept at minimum. It’s helpful to know some Python, HTML, and CSS but you don’t need to have any previous experience with those or web development in general to be able to follow the book. You will learn things like: How to setup PyCharm for Django (you can use any editor). How to organize the project and add a base app to hold common assets. How template inheritance works. How to reuse common template items like grids and pagination. How to work with models, views and urls. How to use GIT and Bitbucket to version control and deploy your code. How to style all features with SASS (or CSS) and Gulp. How to create a responsive design. How to generate thumbnails. How to use relationships (ManyToMany, OneToMany and Foreignkey) in practical contexts. How to create custom forms to add and edit content. How to create and extend class based views. How to create a custom search. How to create an authentication system (sign-in, login, logout and reset password). How to restrict access with groups, permissions and decorators. How to add a user profile page. How to add inline fields to the admin area. How to do test driven development (TDD). How to translate the website. How to create custom error pages. How to setup a production environment with Digitalocean, PostgreSQL, Nginx and Gunicorn. How to use fixtures to apply initial data. How to setup domain, HTTPS, Email and Caching with Memcached. … and a lot more.

资源下载

更多资源
Mario

Mario

马里奥是站在游戏界顶峰的超人气多面角色。马里奥靠吃蘑菇成长,特征是大鼻子、头戴帽子、身穿背带裤,还留着胡子。与他的双胞胎兄弟路易基一起,长年担任任天堂的招牌角色。

Nacos

Nacos

Nacos /nɑ:kəʊs/ 是 Dynamic Naming and Configuration Service 的首字母简称,一个易于构建 AI Agent 应用的动态服务发现、配置管理和AI智能体管理平台。Nacos 致力于帮助您发现、配置和管理微服务及AI智能体应用。Nacos 提供了一组简单易用的特性集,帮助您快速实现动态服务发现、服务配置、服务元数据、流量管理。Nacos 帮助您更敏捷和容易地构建、交付和管理微服务平台。

Spring

Spring

Spring框架(Spring Framework)是由Rod Johnson于2002年提出的开源Java企业级应用框架,旨在通过使用JavaBean替代传统EJB实现方式降低企业级编程开发的复杂性。该框架基于简单性、可测试性和松耦合性设计理念,提供核心容器、应用上下文、数据访问集成等模块,支持整合Hibernate、Struts等第三方框架,其适用范围不仅限于服务器端开发,绝大多数Java应用均可从中受益。

Rocky Linux

Rocky Linux

Rocky Linux(中文名:洛基)是由Gregory Kurtzer于2020年12月发起的企业级Linux发行版,作为CentOS稳定版停止维护后与RHEL(Red Hat Enterprise Linux)完全兼容的开源替代方案,由社区拥有并管理,支持x86_64、aarch64等架构。其通过重新编译RHEL源代码提供长期稳定性,采用模块化包装和SELinux安全架构,默认包含GNOME桌面环境及XFS文件系统,支持十年生命周期更新。

用户登录
用户注册