首页 文章 精选 留言 我的

精选列表

搜索[汽车相关识别],共10006篇文章
优秀的个人博客,低调大师

SpringBoot相关

目录 SpringBoot 安装SpringBootCLI Springboot的测试模块 参考博客 配置文件 多种配置文件并切换 yml方式 yml和Properties结合 Web模块 上传下载文件 错误页面跳转配置 跨域 HTTPS的配置 线程池 项目部署 生成指定文件 war jar 构建docker镜像 gradle结合docker 目录创建于2017-12-18 SpringBoot 安装SpringBootCLI 安装SDKMAN 使用git bash运行 curl -s get.sdkman.io | bash source "/Users/{yourname}/.sdkman/bin/sdkman-init.sh"根据实际目录去运行 spring –version 注意:所有Controller类要和*Application类 同包或子包* Springboot的测试模块 可以使用MockMvc来测试Controller层的代码 可以使用MockMvc的SpringSecurity支持来测试安全模块 使用 WebIntegraionTest 测试运行中的Web容器 启动嵌入式的Servlet容器来进行测试,下断言 使用随机端口启动服务器 配置local.server.port=0 使用Selenium来测试HTML页面,模拟浏览器的动作,查看系统运行状态 参考博客 Springboot探索 配置文件 配置文件的使用 Spring boot配置文件 application.properties SpringBoot常用配置 使用Gradle整合SpringBoot+Vue.js-开发调试与打包 配置文件加密 自定义配置文件将应用配置外置并注入成bean 多种配置文件并切换 yml方式 单文件配置文件 application.yml spring: profiles: active: development # 选用开发模式 --- spring: profiles: development //一系列配置 --- spring: profiles: production //一系列配置 或者 多文件放 application-{profile}.yml yml和Properties结合 格式:application-{profile}.properties 将上面的开发部分,发行部分的配置创建两个配置文件 application-dev.properties 和 application-prod.properties 在主配置文件application.yml中指明 spring: profiles: active: dev或者是prod Web模块 上传下载文件 第一种直接上传到应用的webroot或者resources目录下,第二种上传到数据库中,第三种使用ftp。 Springboot上传文件 上传文件有大小限制,使用如下方法进行配置 参考博客 @Bean public MultipartConfigElement multipartConfigElement() { MultipartConfigFactory factory = new MultipartConfigFactory(); //单个文件最大 factory.setMaxFileSize("80MB"); //KB,MB // 设置总上传数据总大小 factory.setMaxRequestSize("102400KB"); return factory.createMultipartConfig(); } 错误页面跳转配置 @Configuration public class MvcConfig extends WebMvcConfigurerAdapter { @Bean public EmbeddedServletContainerCustomizer containerCustomizer() { return (container -> { ErrorPage error401Page = new ErrorPage(HttpStatus.FORBIDDEN, "/403.html"); ErrorPage error404Page = new ErrorPage(HttpStatus.NOT_FOUND, "/404.html"); ErrorPage error500Page = new ErrorPage(HttpStatus.INTERNAL_SERVER_ERROR, "/500.html"); container.addErrorPages(error401Page, error404Page, error500Page); }); } } 跨域 不同的域名(主机)端口都会导致跨域问题 @Configuration public class CorsConfig { private CorsConfiguration buildConfig() { CorsConfiguration corsConfiguration = new CorsConfiguration(); corsConfiguration.addAllowedOrigin("*"); // 允许任何域名使用 corsConfiguration.addAllowedHeader("*"); // 允许任何头 corsConfiguration.addAllowedMethod("*"); // 允许任何方法(post、get等) return corsConfiguration; } @Bean public CorsFilter corsFilter() { UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource(); source.registerCorsConfiguration("/**", buildConfig()); // 4 return new CorsFilter(source); } } HTTPS的配置 参考博客 签发证书: keytool -genkey -alias tomcat -storetype PKCS12 -keyalg RSA -keysize 2048 -keystore keystore.p12 -validity 3650 server: context-path: /myth ssl: key-store: classpath:keystore.p12 key-store-password: demo1429336 key-store-type: PKCS12 key-alias: tomcat port: 8888 session: timeout: 3000 任意的一个@Configuration注解类里添加 @Bean public TomcatEmbeddedServletContainerFactory servletContainerFactory() { TomcatEmbeddedServletContainerFactory factory = new TomcatEmbeddedServletContainerFactory() { @Override protected void postProcessContext(Context context) { //SecurityConstraint必须存在,可以通过其为不同的URL设置不同的重定向策略。 SecurityConstraint securityConstraint = new SecurityConstraint(); securityConstraint.setUserConstraint("CONFIDENTIAL"); SecurityCollection collection = new SecurityCollection(); collection.addPattern("/*"); securityConstraint.addCollection(collection); context.addConstraint(securityConstraint); } }; factory.addAdditionalTomcatConnectors(createHttpConnector()); return factory; } private Connector createHttpConnector() { Connector connector = new Connector("org.apache.coyote.http11.Http11NioProtocol"); connector.setScheme("http"); connector.setSecure(false); connector.setPort(8887);//http端口(这是要新增加的一个端口) connector.setRedirectPort(8888);// https 端口配置文件中tomcat启动的默认端口 return connector; } 另一种方式 参考博客 方式不一样,没有成功 ############ 证书颁发机构 # CA机构私钥 openssl genrsa -out ca.key 2048 # CA证书 openssl req -x509 -new -key ca.key -out ca.crt ############ 服务端 # 生成服务端私钥 openssl genrsa -out server.key 2048 # 生成服务端证书请求文件 openssl req -new -key server.key -out server.csr # 使用CA证书生成服务端证书 关于sha256,默认使用的是sha1,在新版本的chrome中会被认为是不安全的,因为使用了过时的加密算法。 openssl x509 -req -sha256 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 3650 -out server.crt # 打包服务端的资料为pkcs12格式(非必要,只是换一种格式存储上一步生成的证书) 生成过程中,需要创建访问密码,请记录下来。 openssl pkcs12 -export -in server.crt -inkey server.key -out server.pkcs12 # 生成服务端的keystore(.jks文件, 非必要,Java程序通常使用该格式的证书) 生成过程中,需要创建访问密码,请记录下来。 keytool -importkeystore -srckeystore server.pkcs12 -destkeystore server.jks -srcstoretype pkcs12 # 把ca证书放到keystore中(非必要) keytool -importcert -keystore server.jks -file ca.crt 线程池 参考博客 多线程以及异常处理 参考博客 因为多线程的特性,所以异常只能在子线程中处理不能抛出到主线程里,但是 Spring实现的线程池可以返回一个异常信息对象 项目部署 生成指定文件 war 部署为war必须的类,一般在创建项目时选war就会自动生成,选jar就要手动添加 public class ServletInitializer extends SpringBootServletInitializer { @Override protected SpringApplicationBuilder configure(SpringApplicationBuilder application) { return application.sources(DemoApplication.class); } } maven: mvn war 即可 mvn package -DskipTests gradle: gradle war 然后 gradle bootRepackage 即可 jar 没有特殊的配置,打包即用 maven: mvn package 即可生成可执行的jar gradle:gradle jar 然后 gradle bootRepackage 也生成可执行jar 构建docker镜像 方便监控应用状态,cpu 内存 流量 先构建得到war或jar,然后根据dockerfile构建一个镜像 FROM frolvlad/alpine-oraclejdk8:slim ADD weixin-1.0.0.war app.war ENTRYPOINT ["java","-jar","/app.war"] gradle结合docker build.gradle apply plugin: 'docker' task buildDocker(type: Docker, dependsOn: build) { push = true applicationName = jar.baseName dockerfile = file('src/main/docker/Dockerfile') doFirst { copy { from war into stageDir } } } Dockerfile FROM frolvlad/alpine-oraclejdk8:slim VOLUME /tmp ADD weixin-1.0.0.war app.war ENTRYPOINT ["java","-jar","/app.war"]

优秀的个人博客,低调大师

AutoML相关论文

本文为Awesome-AutoML-Papers的译文。 1、AutoML简介 Machine Learning几年来取得的不少可观的成绩,越来越多的学科都依赖于它。然而,这些成果都很大程度上取决于人类机器学习专家来完成如下工作: 数据预处理 Preprocess the data 选择合适的特征 Select appropriate features 选择合适的模型族 Select an appropriate model family 优化模型参数 Optimize model hyperparameters 模型后处理 Postprocess machine learning models 分析结果 Critically analyze the results obtained 随着大多数任务的复杂度都远超非机器学习专家的能力范畴,机器学习应用的不断增长使得人们对现成的机器学习方法有了极大的需求。因为这些现成的机器学习方法使用简单,并且不需要专业知识。我们将由此产生的研究领域称为机器学习的逐步自动化。 AutoML借鉴了机器学习的很多知识,主要包括: 贝叶斯优化 Bayesian optimization 结构化数据的大数据的回归模型 Regression models for structured data and big data 元学习 Meta learning 迁移学习 Transfer learning 组合优化 Combinatorial optimization. 2、目录 Papers Automated Feature Engineering Expand Reduce Hierarchical Organization of Transformations Meta Learning Reinforcement Learning Architecture Search Evolutionary Algorithms Local Search Meta Learning Reinforcement Learning Transfer Learning Hyperparameter Optimization Bayesian Optimization Evolutionary Algorithms Lipschitz Functions Local Search Meta Learning Particle Swarm Optimization Random Search Transfer Learning Performance Prediction Performance Prediction Frameworks Miscellaneous Tutorials Bayesian Optimization Meta Learning Articles Bayesian Optimization Meta Learning Slides Bayesian Optimization Books Meta Learning Projects Prominent Researchers Papers Automated Feature Engineering Expand Reduce 2017 | AutoLearn — Automated Feature Generation and Selection | Ambika Kaul, et al. | ICDM | PDF 2017 | One button machine for automating feature engineering in relational databases | Hoang Thanh Lam, et al. | arXiv | PDF 2016 | Automating Feature Engineering | Udayan Khurana, et al. | NIPS | PDF 2016 | ExploreKit: Automatic Feature Generation and Selection | Gilad Katz, et al. | ICDM | PDF 2015 | Deep Feature Synthesis: Towards Automating Data Science Endeavors | James Max Kanter, Kalyan Veeramachaneni | DSAA | PDF Hierarchical Organization of Transformations 2016 | Cognito: Automated Feature Engineering for Supervised Learning | Udayan Khurana, et al. | ICDMW | PDF Meta Learning 2017 | Learning Feature Engineering for Classification | Fatemeh Nargesian, et al. | IJCAI | PDF Reinforcement Learning 2017 | Feature Engineering for Predictive Modeling using Reinforcement Learning | Udayan Khurana, et al. | arXiv | PDF 2010 | Feature Selection as a One-Player Game | Romaric Gaudel, Michele Sebag | ICML | PDF Architecture Search Evolutionary Algorithms 2017 | Large-Scale Evolution of Image Classifiers | Esteban Real, et al. | PMLR | PDF 2002 | Evolving Neural Networks through Augmenting Topologies | Kenneth O.Stanley, Risto Miikkulainen | Evolutionary Computation | PDF Local Search 2017 | Simple and Efficient Architecture Search for Convolutional Neural Networks | Thomoas Elsken, et al. | ICLR | PDF Meta Learning 2016 | Learning to Optimize | Ke Li, Jitendra Malik | arXiv | PDF Reinforcement Learning 2018 | Efficient Neural Architecture Search via Parameter Sharing | Hieu Pham, et al. | arXiv | PDF 2017 | Neural Architecture Search with Reinforcement Learning | Barret Zoph, Quoc V. Le | ICLR | PDF Transfer Learning 2017 | Learning Transferable Architectures for Scalable Image Recognition | Barret Zoph, et al. | arXiv | PDF Frameworks 2017 | Google Vizier: A Service for Black-Box Optimization | Daniel Golovin, et al. | KDD |PDF 2017 | ATM: A Distributed, Collaborative, Scalable System for Automated Machine Learning | T. Swearingen, et al. | IEEE | PDF 2015 | AutoCompete: A Framework for Machine Learning Competitions | Abhishek Thakur, et al. | ICML | PDF Hyperparameter Optimization Bayesian Optimization 2016 | Bayesian Optimization with Robust Bayesian Neural Networks | Jost Tobias Springenberg, et al. | NIPS | PDF 2016 | Scalable Hyperparameter Optimization with Products of Gaussian Process Experts | Nicolas Schilling, et al. | PKDD | PDF 2016 | Taking the Human Out of the Loop: A Review of Bayesian Optimization | Bobak Shahriari, et al. | IEEE | PDF 2016 | Towards Automatically-Tuned Neural Networks | Hector Mendoza, et al. | JMLR | PDF 2016 | Two-Stage Transfer Surrogate Model for Automatic Hyperparameter Optimization | Martin Wistuba, et al. | PKDD | PDF 2015 | Efficient and Robust Automated Machine Learning | PDF 2015 | Hyperparameter Optimization with Factorized Multilayer Perceptrons | Nicolas Schilling, et al. | PKDD | PDF 2015 | Hyperparameter Search Space Pruning - A New Component for Sequential Model-Based Hyperparameter Optimization | Martin Wistua, et al. | PDF 2015 | Joint Model Choice and Hyperparameter Optimization with Factorized Multilayer Perceptrons | Nicolas Schilling, et al. | ICTAI | PDF 2015 | Learning Hyperparameter Optimization Initializations | Martin Wistuba, et al. | DSAA | PDF 2015 | Scalable Bayesian optimization using deep neural networks | Jasper Snoek, et al. | ACM | PDF 2015 | Sequential Model-free Hyperparameter Tuning | Martin Wistuba, et al. | ICDM | PDF 2013 | Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms | PDF 2013 | Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures | J. Bergstra | JMLR | PDF 2012 | Practical Bayesian Optimization of Machine Learning Algorithms | PDF 2011 | Sequential Model-Based Optimization for General Algorithm Configuration(extended version) | PDF Evolutionary Algorithms 2018 | Autostacker: A Compositional Evolutionary Learning System | Boyuan Chen, et al. | arXiv | PDF 2017 | Large-Scale Evolution of Image Classifiers | Esteban Real, et al. | PMLR | PDF Lipschitz Functions 2017 | Global Optimization of Lipschitz functions | C´edric Malherbe, Nicolas Vayatis | arXiv | PDF Local Search 2009 | ParamILS: An Automatic Algorithm Configuration Framework | Frank Hutter, et al. | JAIR | PDF Meta Learning 2008 | Cross-Disciplinary Perspectives on Meta-Learning for Algorithm Selection | PDF Particle Swarm Optimization 2017 | Particle Swarm Optimization for Hyper-parameter Selection in Deep Neural Networks | Pablo Ribalta Lorenzo, et al. | GECCO | PDF 2008 | Particle Swarm Optimization for Parameter Determination and Feature Selection of Support Vector Machines | Shih-Wei Lin, et al. | Expert Systems with Applications | PDF Random Search 2016 | Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization | Lisha Li, et al. | arXiv | PDF 2012 | Random Search for Hyper-Parameter Optimization | James Bergstra, Yoshua Bengio | JMLR | PDF 2011 | Algorithms for Hyper-parameter Optimization | James Bergstra, et al. | NIPS | PDF Transfer Learning 2016 | Efficient Transfer Learning Method for Automatic Hyperparameter Tuning | Dani Yogatama, Gideon Mann | JMLR | PDF 2016 | Flexible Transfer Learning Framework for Bayesian Optimisation | Tinu Theckel Joy, et al. | PAKDD | PDF 2016 | Hyperparameter Optimization Machines | Martin Wistuba, et al. | DSAA | PDF 2013 | Collaborative Hyperparameter Tuning | R´emi Bardenet, et al. | ICML | PDF Miscellaneous 2018 | Accelerating Neural Architecture Search using Performance Prediction | Bowen Baker, et al. | ICLR | PDF 2017 | Automatic Frankensteining: Creating Complex Ensembles Autonomously | Martin Wistuba, et al. | SIAM | PDF Tutorials Bayesian Optimization 2010 | A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning | PDF Meta Learning 2008 | Metalearning - A Tutorial | PDF Articles Bayesian Optimization 2016 | Bayesian Optimization for Hyperparameter Tuning | Link Meta Learning 2017 | Why Meta-learning is Crucial for Further Advances of Artificial Intelligence? | Link 2017 | Learning to learn | Link Slides Automated Feature Engineering Automated Feature Engineering for Predictive Modeling | Udyan Khurana, etc al. | PDF Hyperparameter Optimization Bayesian Optimization Bayesian Optimisation | PDF A Tutorial on Bayesian Optimization for Machine Learning | PDF Books Meta Learning 2009 | Metalearning - Applications to Data Mining | Springer | PDF Projects Advisor | Python | Open Source | Code auto-sklearn | Python | Open Source | Code Auto-WEKA | Java | Open Source | Code Hyperopt | Python | Open Source | Code Hyperopt-sklearn | Python | Open Source | Code SigOpt | Python | Commercial | Link SMAC3 | Python | Open Source | Code RoBO | Python | Open Source | Code BayesianOptimization | Python | Open Source | Code Scikit-Optimize | Python | Open Source | Code HyperBand | Python | Open Source | Code BayesOpt | C++ | Open Source | Code Optunity | Python | Open Source | Code TPOT | Python | Open Source | Code ATM | Python | Open Source | Code Cloud AutoML | Python | Commercial| Link H2O | Python | Commercial | Link DataRobot | Python | Commercial | Link MLJAR | Python | Commercial | Link MateLabs | Python | Commercial | Link MARSGGBO原创 2018-7-14

优秀的个人博客,低调大师

docker相关配置

一、概述: 1、centos7下,默认firewalld为防火墙,systemctl status firewalld.service 2、关闭firewalld, systemctl stop firewalld.service &&setenforce 0 3、安装iptables,yum install iptables-services #安装 systemctl restart iptables.service #最后重启防火墙使配置生效 systemctl enable iptables.service #设置防火墙开机启动 Docker配置文件:/etc/sysconfig/docker 主要参数解释: -H 表示Docker Daemon绑定的地址, -H=unix:///var/run/docker.sock 或者-H=tcp://0.0.0.0:2375 --registry-mirror表示Docker Registry的镜像地址, --registry-mirror=http://xxxx --insecure-registry表示(本地)私用Docker Registry的地址, --insecure-registry ${privateRegistryHost}:5000 --selinux-enabled是否开始SELinux,默认开启 --selinux-enabled=true; 开启SELinux --bip表示网桥docker0使用指定的CIDR网络地址, --bip=172.17.42.1 -b 表示采用已经创建好的网桥, -b=xxx 下面是代理的设置: http_proxy=xxxx:8080 https_proxy=xxxx:8080 Docker配置文件(Centos 7) cat /usr/lib/systemd/system/docker.service [Unit]Description=Docker Application Container EngineDocumentation=https://docs.docker.comAfter=network.target [Service]Type=notify# the default is not to use systemd for cgroups because the delegate issues still# exists and systemd currently does not support the cgroup feature set required# for containers run by dockerExecStart=/usr/bin/dockerdExecReload=/bin/kill -s HUP $MAINPID# Having non-zero Limit*s causes performance problems due to accounting overhead# in the kernel. We recommend using cgroups to do container-local accounting.LimitNOFILE=infinityLimitNPROC=infinityLimitCORE=infinityEnvironment="HTTP_PROXY=http://USRNAME:PASSWD@HOST:PORT/"Environment="HTTPS_PROXY=http://USRENAME:PASSWD@HOST:PORT/"# Uncomment TasksMax if your systemd version supports it. # Only systemd 226 and above support this version.#TasksMax=infinityTimeoutStartSec=0# set delegate yes so that systemd does not reset the cgroups of docker containersDelegate=yes# kill only the docker process, not all processes in the cgroupKillMode=process [Install]WantedBy=multi-user.target

优秀的个人博客,低调大师

Shuffle相关分析

Shuffle描述是一个过程,表现出的是多对多的依赖关系。Shuffle是连接map阶段和Reduce阶段的纽带,每个Reduce Task都会从Map Task产生的数据里读取其中的一片数据。Shuffle通常分为两个部分:Map阶段的数据准备和Reduce阶段的数据副本。 Map阶段根据Reduce阶段的Task数量来决定每个Map Task输出的数据分片的个数,这些数据分片可能保存在内存中或者磁盘上,这些分片的存在形式可能是每个分片一个文件,也可能是多个分片放在一个数据文件中,外加一个索引来记录每个分片在数据文件中的偏移量。(RDD中的窄依赖除外,恰好是一对一的) 1、 Shuffle写 Spark中Shuffle输出的ShuffleMapTask会为每个ResultTask创建对应的Bucket,ShuffleMapTask产生的结果会根据设置的partitionner得到对应的BucketId.然后填充到对应的Bucket中去,所以每个ShuffleMapTask创建Bucket的数据是和ResultTask的数目相等的。 ShuffleMapTask创建的Bucket对应磁盘上的一个文件,用于存储结果,此文件也被成为BlockFile.通过spark.shuffle.file.buffer.kb属性配置的缓冲区就是用来创建FastBufferedOutputStream输出流的。如果在配置文件中设置了spark.shuffle.consolidateFiles属性为true,则ShuffleMapTask所产生的Bucket就不一定单独对应一个文件了,而是对应文件的一部分,这样做会大大减少产生的BlockFile文件数量。 2、 Shuffle读 Spark可以通过两种方式读数据,一种是普通的socket方式,另一种是使用Netty框架。Netty方式可以通过配置spark.shuffle.use.netty属性为true启动。Netty框架时,BlockManager会创建ShuffleSender专门用于发送数据,如果ResultTask所需要的数据恰好在本节点,则直接去磁盘上读即可,不再通过网络获取。MapReduce取数据时,即使数据在本地还是要走一遍网络传输。

资源下载

更多资源
Mario

Mario

马里奥是站在游戏界顶峰的超人气多面角色。马里奥靠吃蘑菇成长,特征是大鼻子、头戴帽子、身穿背带裤,还留着胡子。与他的双胞胎兄弟路易基一起,长年担任任天堂的招牌角色。

腾讯云软件源

腾讯云软件源

为解决软件依赖安装时官方源访问速度慢的问题,腾讯云为一些软件搭建了缓存服务。您可以通过使用腾讯云软件源站来提升依赖包的安装速度。为了方便用户自由搭建服务架构,目前腾讯云软件源站支持公网访问和内网访问。

Spring

Spring

Spring框架(Spring Framework)是由Rod Johnson于2002年提出的开源Java企业级应用框架,旨在通过使用JavaBean替代传统EJB实现方式降低企业级编程开发的复杂性。该框架基于简单性、可测试性和松耦合性设计理念,提供核心容器、应用上下文、数据访问集成等模块,支持整合Hibernate、Struts等第三方框架,其适用范围不仅限于服务器端开发,绝大多数Java应用均可从中受益。

WebStorm

WebStorm

WebStorm 是jetbrains公司旗下一款JavaScript 开发工具。目前已经被广大中国JS开发者誉为“Web前端开发神器”、“最强大的HTML5编辑器”、“最智能的JavaScript IDE”等。与IntelliJ IDEA同源,继承了IntelliJ IDEA强大的JS部分的功能。

用户登录
用户注册