5.6. Elasticsearch-低调大师

5.6. Elasticsearch

2017-12-25 556

5.6.1. 内嵌 Elasticsearch

内嵌 Elasticsearch 应用，你不需要一个 Elasticsearch 服务器，启动 Spring boot 即可使用 Elasticsearch 服务。

5.6.1.1. Maven

需要下面两个依赖

			
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
		</dependency>
		<!-- com.sun.jna for elasticsearch -->
		<dependency>
			<groupId>com.sun.jna</groupId>
			<artifactId>jna</artifactId>
			<version>3.0.9</version>
		</dependency>

			
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<groupId>com.example</groupId>
	<artifactId>api</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>jar</packaging>

	<name>api</name>
	<description>Demo project for Spring Boot</description>

	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>1.5.6.RELEASE</version>
		<relativePath /> <!-- lookup parent from repository -->
	</parent>

	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
		<java.version>1.8</java.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-jpa</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-jdbc</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-security</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
		</dependency>
		<dependency>
			<groupId>mysql</groupId>
			<artifactId>mysql-connector-java</artifactId>
			<scope>runtime</scope>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-redis</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
		</dependency>
		<!-- https://mvnrepository.com/artifact/javax.persistence/persistence-api -->
		<dependency>
			<groupId>javax.persistence</groupId>
			<artifactId>persistence-api</artifactId>
			<version>1.0.2</version>
		</dependency>
		<!-- https://mvnrepository.com/artifact/org.json/json -->
		<dependency>
			<groupId>org.json</groupId>
			<artifactId>json</artifactId>
		</dependency>
		<!-- com.sun.jna for elasticsearch -->
		<dependency>
			<groupId>com.sun.jna</groupId>
			<artifactId>jna</artifactId>
			<version>3.0.9</version>
		</dependency>

	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>

			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-surefire-plugin</artifactId>
				<configuration>
					<skip>true</skip>
				</configuration>
			</plugin>
		</plugins>
	</build>
</project>

5.6.1.2. src/main/resources/application.properties

			
spring.data.elasticsearch.repositories.enabled=true
#spring.data.elasticsearch.cluster-name=elasticsearch
#spring.data.elasticsearch.cluster-nodes=119.29.241.95:9200
spring.data.elasticsearch.local=false
spring.data.elasticsearch.properties.transport.tcp.connect_timeout=60s
spring.data.elasticsearch.properties.host=127.0.0.1
spring.data.elasticsearch.properties.port=9200
spring.data.elasticsearch.properties.path.home=/tmp

5.6.1.3. Domain Class

			
package com.example.api.domain.elasticsearch;

import java.io.Serializable;
import java.util.Date;

import javax.persistence.Id;

import org.springframework.data.annotation.CreatedDate;
import org.springframework.data.elasticsearch.annotations.DateFormat;
import org.springframework.data.elasticsearch.annotations.Document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldIndex;
import org.springframework.data.elasticsearch.annotations.FieldType;

import com.fasterxml.jackson.annotation.JsonFormat;

@Document(indexName = "information", type = "article")
public class Article implements Serializable {

	/**
	 * 
	 */
	private static final long serialVersionUID = 8789505663320446079L;
	@Id
	private int id;
	private String title;
	private String description;
	private String author;
	private String source;
	private String content;
	@JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyyMMdd'T'HHmmss.SSS'Z'")
    @Field(type = FieldType.Date, format = DateFormat.basic_date_time, index = FieldIndex.not_analyzed)
    @CreatedDate
	private Date ctime;

	public int getId() {
		return id;
	}

	public void setId(int id) {
		this.id = id;
	}

	public String getTitle() {
		return title;
	}

	public void setTitle(String title) {
		this.title = title;
	}

	public String getDescription() {
		return description;
	}

	public void setDescription(String description) {
		this.description = description;
	}

	public Date getCtime() {
		return ctime;
	}

	public void setCtime(Date ctime) {
		this.ctime = ctime;
	}

	public String getAuthor() {
		return author;
	}

	public void setAuthor(String author) {
		this.author = author;
	}

	public String getSource() {
		return source;
	}

	public void setSource(String source) {
		this.source = source;
	}

	public String getContent() {
		return content;
	}

	public void setContent(String content) {
		this.content = content;
	}

	@Override
	public String toString() {
		return "Article [id=" + id + ", title=" + title + ", description=" + description + ", author=" + author + ", source=" + source + ", content=" + content + ", ctime=" + ctime + "]";
	}

}

5.6.1.4. ElasticsearchRepository

			
package com.example.api.repository.elasticsearch;

import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;
import org.springframework.stereotype.Repository;

import com.example.api.domain.elasticsearch.Article;

@Repository
public interface ArticleElasticsearchRepository extends ElasticsearchRepository<Article, Integer> {
	Page<Article> findByTitleLike(String title, Pageable page);

	Page<Article> findByDescription(String description, Pageable pageable);

	Page<Article> findByDescriptionNot(String description, Pageable pageable);

	Page<Article> findByDescriptionLike(String description, Pageable pageable);
}

5.6.1.5. SearchRestController

			
package com.example.api.restful;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.springframework.data.domain.Sort;
import org.springframework.data.web.PageableDefault;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import com.example.api.domain.elasticsearch.Article;
import com.example.api.repository.elasticsearch.ArticleElasticsearchRepository;

@RestController
@RequestMapping("/restful/search")
public class SearchRestController {
	@Autowired
	private ArticleElasticsearchRepository articleElasticsearchRepository;
	
	@RequestMapping(value = "/article/create")
	public  Article create() {
		Article article = new Article();
		article.setId(1);
		article.setTitle("sssss");
		article.setContent("test");
		return articleElasticsearchRepository.save(article);
	}
	@RequestMapping(value = "/article/{articleId}")
	public  Article get(@PathVariable int articleId) {
		return articleElasticsearchRepository.findOne(articleId);
	}
}

5.6.1.6. 测试

MacBook-Pro:~ neo$ curl http://test:test@localhost:8443/restful/search/article/create.json
{"id":1,"title":"sssss","description":null,"author":null,"source":null,"content":"test","ctime":null}

MacBook-Pro:~ neo$ curl http://test:test@localhost:8443/restful/search/article/1.json
{"id":1,"title":"sssss","description":null,"author":null,"source":null,"content":"test","ctime":null}

5.6.2. 集群模式

查看 cluster.name 配置项

root@netkiller ~ % grep ^cluster.name /etc/elasticsearch/elasticsearch.yml
cluster.name: elasticsearch

src/main/resources/application.properties

spring.data.elasticsearch.cluster-name=elasticsearch  
spring.data.elasticsearch.cluster-nodes=172.16.0.100:9200  
spring.data.elasticsearch.local=false  
spring.data.elasticsearch.repositories.enabled=true

5.6.3. Document

		
@Document(indexName = "customer", type = "external", shards = 1, replicas = 0, refreshInterval = "-1")

5.6.4. Elasticsearch 删除操作

		
package com.example.api.schedule;

import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.rest.RestStatus;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;

import com.example.api.domain.elasticsearch.ElasticsearchTrash;
import com.example.api.repository.elasticsearch.ElasticsearchTrashRepository;

@Component
public class ScheduledTasks {
	private static final Logger logger = LoggerFactory.getLogger(ScheduledTasks.class);

	@Autowired
	private TransportClient client;

	@Autowired
	private ElasticsearchTrashRepository alasticsearchTrashRepository;

	public ScheduledTasks() {
	}

	@Scheduled(fixedRate = 1000 * 60) // 60秒运行一次调度任务
	public void cleanTrash() {
		for (ElasticsearchTrash elasticsearchTrash : alasticsearchTrashRepository.findAll()) {
			DeleteResponse response = client.prepareDelete("information", "article", elasticsearchTrash.getId() + "").get();
			RestStatus status = response.status();
			logger.info("delete {} {}", elasticsearchTrash.getId(), status.toString());
			if (status == RestStatus.OK || status == RestStatus.NOT_FOUND) {
				alasticsearchTrashRepository.delete(elasticsearchTrash);
			}
		}
	}
}

5.6.5. FAQ

5.6.5.1. java.lang.IllegalStateException: Received message from unsupported version: [2.0.0] minimal compatible version is: [5.0.0]

spring-boot-starter-data-elasticsearch 目前还不支持 5.0.0 版本

原文出处：Netkiller 系列手札
本文作者：陈景峯
转载请与作者联系，同时请务必标明文章原始出处和作者信息及本声明。

微信关注我们

原文链接：https://yq.aliyun.com/articles/320063

转载内容版权归作者及来源网站所有！

低调大师中文资讯倾力打造互联网数据资讯、行业资源、电子商务、移动互联网、网络营销平台。持续更新报道IT业界、互联网、市场资讯、驱动更新,是最及时权威的产业资讯及硬件资讯报道平台。

2.32. Spring boot with ELK(Elasticsearch + Logstash + Kibana)

将 Spring boot 日志写入 ELK 有多种实现方式，这里仅提供三种方案： Spring boot -> logback -> Tcp/IP -> logstash -> elasticsearch 这种方式实现非常方便不需要而外包或者软件 Spring boot -> logback -> Redis -> logstash -> elasticsearch 利用 Redis 提供的发布订阅功能将日志投递到 elasticsearch Spring boot -> logback -> Kafka -> logstash -> elasticsearch Kafka 方法适合大数据的情况。 2.32.1.TCP 方案 logstash 配置 input { tcp { host => "172.16.1.16" port => 9250 mode => "server" tags => ["tags"] codec => json_lines //可能需要更新logstash...

2017-12-26

590

[Spark][Python][DataFrame][SQL]Spark对DataFrame直接执行SQL处理的例子 $cat people.json {"name":"Alice","pcode":"94304"} {"name":"Brayden","age":30,"pcode":"94304"} {"name":"Carla","age":19,"pcoe":"10036"} {"name":"Diana","age":46} {"name":"Etienne","pcode":"94104"} $ hdfs dfs -put people.json $pyspark sqlContext = HiveContext(sc) peopleDF = sqlContext.read.json("people.json") peopleDF.registerTempTable("people") tmpDF=sqlContext.sql(""" select * FROM people WHERE name like "A%" """) tmpDF.limit(3).show() ...

2017-12-26

602

资源下载

更多资源

Mario

马里奥是站在游戏界顶峰的超人气多面角色。马里奥靠吃蘑菇成长，特征是大鼻子、头戴帽子、身穿背带裤，还留着胡子。与他的双胞胎兄弟路易基一起，长年担任任天堂的招牌角色。

腾讯云软件源

为解决软件依赖安装时官方源访问速度慢的问题，腾讯云为一些软件搭建了缓存服务。您可以通过使用腾讯云软件源站来提升依赖包的安装速度。为了方便用户自由搭建服务架构，目前腾讯云软件源站支持公网访问和内网访问。

Nacos

Nacos /nɑ:kəʊs/ 是 Dynamic Naming and Configuration Service 的首字母简称，一个易于构建 AI Agent 应用的动态服务发现、配置管理和AI智能体管理平台。Nacos 致力于帮助您发现、配置和管理微服务及AI智能体应用。Nacos 提供了一组简单易用的特性集，帮助您快速实现动态服务发现、服务配置、服务元数据、流量管理。Nacos 帮助您更敏捷和容易地构建、交付和管理微服务平台。

Rocky Linux

Rocky Linux（中文名：洛基）是由Gregory Kurtzer于2020年12月发起的企业级Linux发行版，作为CentOS稳定版停止维护后与RHEL（Red Hat Enterprise Linux）完全兼容的开源替代方案，由社区拥有并管理，支持x86_64、aarch64等架构。其通过重新编译RHEL源代码提供长期稳定性，采用模块化包装和SELinux安全架构，默认包含GNOME桌面环境及XFS文件系统，支持十年生命周期更新。