JSON 性能测试之二 - wastjson 每秒 6GB 解析速度完胜 simdjson-java！
前一篇JSON性能测试对比Fastjson2和DslJson，有评论提到和simdjson-java对比，那就测试走起.
说下simdjson-java这个库，之前也听说过，由于早前官网测试用的文本非常离谱，而且依赖JDK18+就没怎么关注，现在只要搜索simdjson（c++）基本上就能看到宣传最多的就是每秒千兆解析超越同类C++库甚至25倍的都有,不确定simdjson和simdjson-java性能差异有多大，下面只和simdjson-java做下对比测试。
测试环境: openJdk21 + window10 + i5 单线程
jvm参数: -Xms3g -Xmx3g --add-opens=java.base/java.time=ALL-UNNAMED --add-modules=jdk.incubator.vector
maven 依赖截止当前最新版本
<dependency>
<groupId>org.simdjson</groupId>
<artifactId>simdjson-java</artifactId>
<version>0.3.0</version>
</dependency>
<dependency>
<groupId>io.github.wycst</groupId>
<artifactId>wast</artifactId>
<version>0.0.19</version>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-core</artifactId>
<version>1.36</version>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>1.36</version>
</dependency>
场景测试一: 测试数据摘自simdjson-java官方(twitter.json内容已经相对正常多了)，数据大小: 631kb
import com.alibaba.fastjson2.JSON;
import com.alibaba.fastjson2.JSONObject;
import io.github.wycst.wast.common.utils.IOUtils;
import io.github.wycst.wast.json.JSONNode;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import org.openjdk.jmh.runner.options.TimeValue;
import org.simdjson.JsonValue;
import org.simdjson.SimdJsonParser;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Set;
/**
* @Date 2024/11/24 15:46
* @Created by wangyc
*/
public class TwitterJmhTest {
final static byte[] buffer;
final static SimdJsonParser simdJsonParser = new SimdJsonParser();
static {
try {
buffer = IOUtils.readBytes(TwitterJmhTest.class.getResourceAsStream("/data/json/twitter.json"));
System.out.println("data size: " + (buffer.length >> 10) + "kb");
} catch (Exception e) {
throw new RuntimeException(e);
}
}
// @Benchmark
// public int parseAndSelect_Fastjson2() {
// JSONObject jsonObject = (JSONObject) JSON.parse(buffer);
// Set<String> defaultUsers = new HashSet<>();
// Iterator<Object> tweets = jsonObject.getJSONArray("statuses").iterator();
// while (tweets.hasNext()) {
// JSONObject tweet = (JSONObject) tweets.next();
// JSONObject user = (JSONObject) tweet.get("user");
// if (user.getBoolean("default_profile")) {
// defaultUsers.add(user.getString("screen_name"));
// }
// }
// return defaultUsers.size();
// }
//
// @Benchmark
// public int schemaBasedParseAndSelect_Fastjson2() {
// Set<String> defaultUsers = new HashSet<>();
// SimdJsonTwitter twitter = JSON.parseObject(buffer, SimdJsonTwitter.class);
// for (SimdJsonStatus status : twitter.statuses) {
// SimdJsonUser user = status.user;
// if (user.default_profile) {
// defaultUsers.add(user.screen_name);
// }
// }
// return defaultUsers.size();
// }
@Benchmark
public int parseAndSelect_WastJson() {
List<JSONNode> users = JSONNode.collect(buffer, "/statuses/*/user");
Set<String> defaultUsers = new HashSet<>();
for (JSONNode user : users) {
if (user.getChildValue("default_profile", boolean.class)) {
defaultUsers.add(user.getChildValue("screen_name", String.class));
}
}
return defaultUsers.size();
}
@Benchmark
public int schemaBasedParseAndSelect_WastJson() {
Set<String> defaultUsers = new HashSet<>();
SimdJsonTwitter twitter = io.github.wycst.wast.json.JSON.parseObject(buffer, SimdJsonTwitter.class);
for (SimdJsonStatus status : twitter.statuses) {
SimdJsonUser user = status.user;
if (user.default_profile) {
defaultUsers.add(user.screen_name);
}
}
return defaultUsers.size();
}
@Benchmark
public int parseAndSelect_SimdjsonJava() {
JsonValue simdJsonValue = simdJsonParser.parse(buffer, buffer.length);
Set<String> defaultUsers = new HashSet<>();
Iterator<JsonValue> tweets = simdJsonValue.get("statuses").arrayIterator();
while (tweets.hasNext()) {
JsonValue tweet = tweets.next();
JsonValue user = tweet.get("user");
if (user.get("default_profile").asBoolean()) {
defaultUsers.add(user.get("screen_name").asString());
}
}
return defaultUsers.size();
}
@Benchmark
public int schemaBasedParseAndSelect_SimdjsonJava() {
Set<String> defaultUsers = new HashSet<>();
SimdJsonTwitter twitter = simdJsonParser.parse(buffer, buffer.length, SimdJsonTwitter.class);
for (SimdJsonStatus status : twitter.statuses()) {
SimdJsonUser user = status.user();
if (user.default_profile()) {
defaultUsers.add(user.screen_name());
}
}
return defaultUsers.size();
}
record SimdJsonUser(boolean default_profile, String screen_name) {
}
record SimdJsonStatus(SimdJsonUser user) {
}
record SimdJsonTwitter(List<SimdJsonStatus> statuses) {
}
public static void main(String[] args) throws RunnerException {
Options options = new OptionsBuilder()
.include(TwitterJmhTest.class.getName())
.mode(Mode.Throughput)
.warmupIterations(1)
.warmupTime(TimeValue.seconds(3))
.measurementTime(TimeValue.seconds(3))
.forks(1)
.threads(1)
.build();
new Runner(options).run();
}
}
测试结果（每秒运行次数）
Benchmark Mode Cnt Score Error Units
TwitterJmhTest.parseAndSelect_SimdjsonJava thrpt 5 1370.652 ± 37.171 ops/s
TwitterJmhTest.parseAndSelect_WastJson thrpt 5 1997.455 ± 275.386 ops/s
TwitterJmhTest.schemaBasedParseAndSelect_SimdjsonJava thrpt 5 2092.163 ± 98.596 ops/s
TwitterJmhTest.schemaBasedParseAndSelect_WastJson thrpt 5 2654.975 ± 102.240 ops/s
parseAndSelect： WastJson 大约是 SimdjsonJava的1.45倍（解析速度wast 1.23g/s, simdjson 0.84g/s）
schemaBasedParseAndSelect：WastJson大约是SimdjsonJava的1.26倍(解析速度wast 1.6g/s, simdjson 1.28g/s)
以上是在simdjson-java官方提供数据下，WastJson都能完胜.
场景测试二: 全部为ascii编码的字节数组，大小1464kb
测试代码
package com.jmh.test.simdjson;
import io.github.wycst.wast.json.JSON;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import org.openjdk.jmh.runner.options.TimeValue;
import org.simdjson.JsonValue;
import org.simdjson.SimdJsonParser;
import java.util.HashMap;
import java.util.Map;
/**
* @Date 2024/11/27
* @Created by wangyc
*/
public class AsciiSourceJmhTest {
final static byte[] buffer;
final static SimdJsonParser simdJsonParser = new SimdJsonParser();
static {
try {
int length = 100000;
StringBuilder value = new StringBuilder("a".repeat(length));
Map map = new HashMap();
map.put("a", value);
map.put("b", value);
map.put("c", value);
map.put("d", value);
map.put("e", value);
map.put("f", value);
map.put("g", value);
map.put("h", value);
map.put("i", value);
map.put("j", value);
map.put("k", value);
map.put("l", value);
map.put("m", value);
map.put("n", value);
map.put("o", value);
buffer = io.github.wycst.wast.json.JSON.toJsonBytes(map);
System.out.println("data size: " + (buffer.length >> 10) + "kb");
} catch (Exception e) {
throw new RuntimeException(e);
}
}
@Benchmark
public long ascii_WastJson() {
long result = 0;
Map map = (Map) JSON.parse(buffer);
result += map.get("a").toString().length();
result += map.get("b").toString().length();
result += map.get("c").toString().length();
result += map.get("d").toString().length();
result += map.get("e").toString().length();
result += map.get("f").toString().length();
result += map.get("g").toString().length();
result += map.get("h").toString().length();
result += map.get("i").toString().length();
result += map.get("j").toString().length();
result += map.get("k").toString().length();
result += map.get("l").toString().length();
result += map.get("m").toString().length();
result += map.get("n").toString().length();
result += map.get("o").toString().length();
return result;
}
@Benchmark
public long ascii_SimdjsonJava() {
long result = 0;
JsonValue simdJsonValue = simdJsonParser.parse(buffer, buffer.length);
result += simdJsonValue.get("a").asString().length();
result += simdJsonValue.get("b").asString().length();
result += simdJsonValue.get("c").asString().length();
result += simdJsonValue.get("d").asString().length();
result += simdJsonValue.get("e").asString().length();
result += simdJsonValue.get("f").asString().length();
result += simdJsonValue.get("g").asString().length();
result += simdJsonValue.get("h").asString().length();
result += simdJsonValue.get("i").asString().length();
result += simdJsonValue.get("j").asString().length();
result += simdJsonValue.get("k").asString().length();
result += simdJsonValue.get("l").asString().length();
result += simdJsonValue.get("m").asString().length();
result += simdJsonValue.get("n").asString().length();
result += simdJsonValue.get("o").asString().length();
return result;
}
public static void main(String[] args) throws RunnerException {
Options options = new OptionsBuilder()
.include(AsciiSourceJmhTest.class.getName())
.mode(Mode.Throughput)
.warmupIterations(1)
.warmupTime(TimeValue.seconds(3))
.measurementTime(TimeValue.seconds(3))
.forks(1)
.threads(1)
.build();
new Runner(options).run();
}
}
测试结果（每秒运行次数）
Benchmark Mode Cnt Score Error Units
AsciiSourceJmhTest.ascii_SimdjsonJava thrpt 5 1231.578 ± 682.959 ops/s
AsciiSourceJmhTest.ascii_WastJson thrpt 5 4212.839 ± 2349.629 ops/s
该场景下WastJson是Simdjson的3.4倍（解析速度 wast: 6.16GB/s, simdjson 1.8GB/s）
以上测试均为个人电脑环境下面的测试，感兴趣的也可以下载代码测试，如果有专业人士能测试不同平台的性能差异万分感谢！
测试代码仓库: https://gitee.com/xiaoch0209/wast-jmh-test/tree/main-openjdk23/ (以上测试代码和测试结果都已上传)
