您现在的位置是:首页 > 文章详情

LazyLLM教程 | 第8讲:不止是cosine!匹配策略决定你召回的质量

日期:2025-09-09点击:24

在前面教程中,我们介绍了如何通过查询重写、各种优化检索策略和召回重排策略来提升检索模块的召回率。其中影响检索召回文档质量的一个关键组件为 similarity,它的作用是用来计算检索的文档和查询 query 之间的相似度。

LazyLLM 默认提供的相似度计算函数为 bm25(分为英文和中文) 相似度计算,余弦相似度计算方法。其中 bm25 算法主要针对文本进行计算,而余弦相似度算法主要针对 embedding 进行计算。如果 LazyLLM 提供的默认相似度计算方法不能满足自己的需求,可以自己来设计定义符合自己需求的相似度计算方法。与 Similarity 组件相似,本教程同时介绍 Transform的自定义方法。

本教程主要介绍如何使用自定义 Similarity 组件和 Transform的方法,读完本教程,您将学会 LazyLLM 中自定义 Similarity 和 Transform的方法,并基于 Similarity 和 Transform分别搭建一个简单的RAG应用。

 

为什么要自定义 Similarity

在 RAG服务中,检索模块的效果直接影响生成结果的相关性与准确性。不同应用场景对“相似度”的理解和需求各不相同。例如,在法律或医疗文档中,强调关键词级别的精确匹配;在通用问答场景中,更注重语义层面的理解;而在多模态场景中,则可能涉及文本与图像或结构化数据之间的对齐匹配。为适配多样化需求,LazyLLM 提供了两种通用的相似度计算方法:BM25/BM25_chinese余弦相似度。尽管这两者在通用场景中具有良好表现,但在某些特定任务中仍存在局限性:

  • BM25 无法理解语义近似:它基于词频和词面匹配,无法处理近义词、同义术语或行业特定表述的归一化问题;

  • 余弦相似度粒度较粗:在长文本或结构复杂的文档中,局部相关内容容易被整体平均,从而稀释匹配效果。

👇我们来看一个示例,关于BM25 无法理解领域语义(近义词、术语归一)的问题。

① 示例背景

  • 用户查询:“高血糖的饮食干预方法”

  • 查询说明:用户查询医学相关的资料,使用的是专业术语的同义表达。

  • 候选文档:“糖尿病患者应控制碳水化合物摄入,以调节血糖水平。”

 

② 理想匹配预期

“高血糖”是“糖尿病”的核心表现之一。尽管查询和文档表述不同,但在医学语境中高度相关。理想的检索系统应能理解两者间的语义等价关系,从而高质量地命中该文档。

③ 问题分析

使用 BM25 或 BM25_chinese 时,由于它们仅关注词面重合,“高血糖”与“糖尿病”词面不同,因此得分较低,极有可能被错过。这类基于表层匹配的方式在专业领域语义检索中显得力不从心。因此,为提升检索质量,在实际系统中往往需要引入更语义敏感、领域定制化的相似度策略,以适配复杂多变的业务需求。

👇接着,看一下另一个示例,关于余弦相似度在长文档中被稀释的问题。

① 示例背景

  • 用户查询:“谁具备优秀的项目管理能力”

  • 查询说明:用户想找一段描述“张三的项目管理能力”的内容。

  • 候选文档(长文档摘要):“张三毕业于某重点高校,具有多年软件开发经验。他参与多个大型项目的研发与部署,包括某知名 Agent应用开发框架。他的主要职责包括代码实现、团队协作与任务分配。此外,他还在多个关键节点中承担项目负责人的角色,体现出良好的组织能力和推进力。”

② 理想匹配预期

用户查询和文档中 “项目负责人”,“组织能力”等 部分高度相关,理想的系统应该能够识别出局部相关性,并提升该文档排名。

③ 问题分析

使用基于 TF-IDF 向量化的余弦相似度时,文档的所有词语都被纳入整体向量表示。在这种情况下,像“开发经验”、“代码实现”等内容虽然词频较高,却与查询目标无关,反而稀释了项目管理相关内容的权重,导致整体相似度偏低,文档难以被正确召回或排序靠前。因此,在处理包含多个主题的长文档时,单一的全文向量匹配显得力不从心,更需要引入局部语义感知机制(如片段级匹配、摘要引导、注意力机制等)来增强相关片段的识别与表达。

如何自定义 Similarity

1.环境准备

如果您的电脑上安装了Python,请通过下方命令安装lazyllm及必要的依赖包。关于 LazyLLM 的环境更详细的准备可以参考 第2讲:10分钟上手一个最小可用RAG系统 中对应的内容。

pip install lazyllm

2.实现函数

由于 LazyLLM 只提供了 BM25 和余弦相似度两种相似度计算方式,在实际应用中可能不能满足要求,这就需要自己根据实际需求来定制 similarity。这里我们以 TF-IDF 算法来实现一个 similarity 计算的例子来说明怎么定义及应用。

TF-IDF 原理与示例

TF-IDF 全称为 Term Frequency - Inverse Document Frequency,是一种在信息检索和文本挖掘中广泛使用的关键词提取算法。它的核心思想是:一个词如果在一篇文档中频繁出现,但在所有文档中不常见,那么它很可能是这篇文档的重要关键词。

1. 词频 TF(Term Frequency)

衡量词语在文档中的重要性:

其中, f(t,d) 表示词 t 在文档 d 中出现的次数,分母为该文档中所有词的出现次数之和。

2. 逆文档频率 IDF(Inverse Document Frequency)

衡量词语在语料库中的稀有程度:

其中, N 为总文档数, df(t) 为包含词 t 的文档数。

3. 最终计算公式:TF-IDF

4. 示例分析

文档集:

📄 文档1:「大模型 正在 改变 世界」

📄 文档2:「大模型 是 人工 智能 的 关键 技术」

📄 文档3:「我 喜欢 编程」

查询:「大模型 技术」

🔹 TF(以文档2为例) :共7词

“大模型”出现1次 → TF = 1 / 7 ≈ 0.143

“技术”出现1次 → TF = 1 / 7 ≈ 0.143

🔹 IDF(N = 3)

“大模型”:df = 2 → IDF = log(3 / (1+2)) = log(1) = 0

“技术”:df = 1 → IDF = log(3 / 2) ≈ 0.405

🔹 TF-IDF(以文档2为例)

TF-IDF("大模型") = 0.143 × 0 = 0

TF-IDF("技术") = 0.143 × 0.405 ≈ 0.058

 

🚨注意:为了避免IDF为负值,Sklearn中使用了平滑版,做了一些处理避免了除以 0,也确保了 IDF ≥ 1。

 

虽然“大模型”是 query 的关键词,但由于它在多个文档中频繁出现,IDF 为 0,无法帮助区分文档。而“技术”这个词仅在文档2中出现,它的 TF-IDF 值高,对区分文档有贡献。

 

代码示例:

 

(代码GitHub链接:

https://github.com/LazyAGI/Tutorial/blob/7abc91dbb82a007a78731845dd8c360ac0cc1e75/rag/codes/chapter8/rag_tfidf_similarity.py#L1

 

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
import numpy as np
from scipy.linalg import norm
import sys
import heapq
from typing import List, Tuple
from lazyllm.tools.rag import DocNode
def tfidf_similarity(query: str, nodes: List[DocNode], **kwargs) -> List[Tuple[DocNode, float]]:
    def add_space(s):
        return ' '.join(list(s))
    corpus = [add_space(node.get_text()) for node in nodes]
    query = add_space(query)
    topk = min(len(nodes), kwargs.get("topk", sys.maxsize))
    cv = TfidfVectorizer(tokenizer=lambda s: s.split())
    tfidf_matrix = cv.fit_transform(corpus+[query])
    query_vec = tfidf_matrix[-1]
    doc_vecs = tfidf_matrix[:-1]
    similairyties = cosine_similarity(query_vec, doc_vecs).flatten()
    indexes = heapq.nlargest(topk, range(len(similairyties)), similairyties.__getitem__)
    results = [(nodes[i], similairyties[i]) for i in indexes]
    return results

在上面的代码中,我们首先通过把 query 和 node 中的文本中插入空格,以用于后面的 tokenizer 处理。tfidf算法是调用 sklearn 中的 TfidfVectorizer 来实现的。计算完 query 和 node 的相似度之后,提取 topk 的结果作为输出。现在我们使用下面的例子进行测试。

query = "今天天气怎么样"
candidates = [
    DocNode(text="今天阳光明媚"),
    DocNode(text="衬衫的价格是100元"),
    DocNode(text="今天天气非常好"),
    DocNode(text="我喜欢吃苹果"),
    DocNode(text="今天天气真糟糕")
]
results = tfidf_similarity(query, candidates)

最后得到如下的输出结果:

3.注册函数

在上一节中我们定义了一个 tfidf 相似度计算方法,如果想在 LazyLLM 中使用该计算方法,则需要先把该方法注册进 LazyLLM 框架中。对相似度计算方法进行注册是通过装饰器来进行的,即通过 register_similarity 装饰器函数进行注册。

 

【装饰器(Decorator)是 Python 中的一种高级功能,它允许你在不修改原函数代码的情况下,给函数添加额外的功能(比如日志记录、性能测试、权限验证等)。其本质上是一个 “接受函数作为参数,并返回新函数” 的函数。你可以把它想象成 “函数的包装盒”——把函数放进去,它就会自动获得新能力。实现代码如下。】

代码如下:

import lazyllm@lazyllm.tools.rag.register_similarity(mode='text', batch=True)def tfidf_similarity(query: str, nodes: List[DocNode], **kwargs) -> List[Tuple[DocNode, float]]:    ...

register_similarity装饰器函数有四个参数,这里我们详细介绍一下。

 

  • func,它是一个可选的调用对象,表示要被装饰的函数,如果没有提供,则会返回一个装饰器函数。

  • mode,它是可选的字面类型,取值为 "text" 或 "embedding",表示相似度计算的模式:

    →"text" 表示该相似度计算方法主要是针对文本进行计算的;

    →"embedding" 表示该相似度计算方法主要是针对嵌入向量进行计算的。

  • descend,它是一个布尔值,指示结果是否按降序排序,默认值为 True,需要注意这个参数不是在在计算相似度输出时生效,而是在索引的时候才会生效。

  • batch,它也是一个布尔值,指示是否支持批量处理,默认值为 False。

 

下面简单展示一下装饰器的三种使用方式:​​​​​​

# 使用方式1:作为函数使用func1 = lazyllm.tools.rag.register_similarity(tfidf)# 使用方式2:作为无参数的装饰器使用@lazyllm.tools.rag.register_similaritydef tfidf(query: str, nodes: List[DocNode],              **kwargs) -> List[Tuple[DocNode, float]]:    ……# 使用方式3:作为有参数的装饰器使用@lazyllm.tools.rag.register_similarity(mode=‘text’)def tfidf(query: str, nodes: List[DocNode],              **kwargs) -> List[Tuple[DocNode, float]]:

4.参数详解

下面我们通过一些小例子来比较每个参数的作用:

1. 参数 func

关于 register_similarity函数的 func 参数,有两种方式使用,一种是正常函数调用的方式传参进去,一种是通过装饰器的方式直接在函数定义时注册。下面为了说明这个参数传进去和没有传进去的区别,我们使用函数调用的方式来演示。

func1 = lazyllm.tools.rag.register_similarity(tfidf_similarity)func2 = lazyllm.tools.rag.register_similarity()print(f"func1: {func1.__name__}, func2: {func2.__name__}")

输出结果为:

func1tfidf_similarity, func2: decorator

func1 对应的是注册函数传入了相似度计算函数,所以它的返回值是被装饰的相似度计算函数,而 func2 对应的是是没有传入相似度计算函数,所以它的返回值是装饰器函数。

2. 参数 mode

Mode,可选的字面类型,取值为 "text" 或 "embedding",表示相似度计算的模式。

  • "text" 表示该相似度计算方法主要是针对文本进行计算的;

  • "embedding" 表示该相似度计算方法主要是针对嵌入向量进行计算的;

📌 从表中可以看出,我们的输入、mode和参数三者类型必须完全一致才可以使用。但实际使用时,只需要保证函数的node参数类型和注册时候的mode参数一致即可,无需关心query的类型。

这里我们先定义一个欧式距离的计算函数:

import lazyllmfrom typing import Listimport numpy as npdef euclidean_distance(query: List[float], node: List[float], **kwargs) -> float:    point1 = np.array(query)    point2 = np.array(node)    return np.linalg.norm(point1 - point2)

然后把这个相似度计算函数注册一下,这里使用不同的mode:​​​​​​​

func1 = lazyllm.tools.rag.register_similarity(euclidean_distance, mode="text")func2 = lazyllm.tools.rag.register_similarity(euclidean_distance, mode="embedding")

然后我们定义不同的 query 和 node 来用不同模式的相似度计算函数计算相似度,看看效果:​​​​​​​

query_t = "hello world."node_t = [DocNode(text="hello lazyllm.")]query_e = {"key": [1.00.42.1]}node_e = [DocNode(embedding={"key": [4.22.13.9]})]

用 func1 计算文本的相似度​​​​​​​

ret = func1(query_tnode_t)print(f"ret: {ret}")

输出结果报错了:

TypeError: unsupported operand type(s) for -: 'str' and 'DocNode'

用 func1 计算向量相似度​​​​​​​

ret = func1(query_e, node_e)print(f"ret: {ret}")

输出结果报错了:

TypeError: unsupported operand type(s) for -: 'dict' and 'DocNode'

用 func2 计算文本的相似度​​​​​​​

ret = func2(query_tnode_t)print(f"ret: {ret}")

输出结果报错了:

AssertionError: query must be of dict type, used for similarity calculation.

用 func2 计算向量相似度​​​​​​​

ret_2e = func2(query_e, node_e)print(f"ret_2e: {ret_2e}")

输出结果为:

ret: {'key': [(<Node id=2865d5c9-730b-4fda-8077-57b706944ad9>, 4.045985664828782)]}

输出结果符合预期,因为 LazyLLM 是支持多 embedding 的,所以前面的 key 表示的是哪类 embedding,然后 tuple 里面的第一项 Node 表示的是哪个节点和 query 进行计算的,第二项表示的是相似度分数。

通过上面四种结果比较,就可以非常清楚 mode 参数的作用了。

3. 参数 descend

descend,布尔值,指示结果是否按降序排序,默认值为 True。需要注意这个参数不是在计算相似度输出时生效,而是在索引的时候才会生效。因为输出时会去重,所以只是返回了topK个node,并不一定以按序排列的。这里我们通过查看 Retriever 检索的结果是否和 query 相似就可以看出该参数的效果了。

我们还是继续使用上面定义的欧氏距离的计算函数,把这个相似度计算函数注册一下,我们先使用 descend 为默认值True的情况:​​​​​​​

import lazyllmfrom typing import Listimport numpy as np@lazyllm.tools.rag.register_similarity(mode="embedding")def euclidean_distance(query: List[float], node: List[float], **kwargs) -> float:    point1 = np.array(query)    point2 = np.array(node)    return np.linalg.norm(point1 - point2)

然后我们使用前面的 html 文档进行测试,我们先定义一个 Retriever 对象:​​​​​​​

import lazyllmimport osfrom lazyllm import OnlineEmbeddingModule, SentenceSplitter, Retrieverfrom lazyllm.tools.rag import Documentprompt = 'You will play the role of an AI Q&A assistant and complete a dialogue task. In this task, you need to provide your answer based on the given context and question.'documents = Document(dataset_path=os.path.join(os.getcwd(), "rag_data"), embed=OnlineEmbeddingModule(source="glm", embed_model_name="embedding-2"), manager=False)documents.create_node_group(name="sentences", transform=SentenceSplitter, chunk_size=1024, chunk_overlap=100)ppl = Retriever(documents, group_name="CoarseChunk", similarity="euclidean_distance", similarity_cut_off=0.003, topk=3)nodes = ppl("全国住房城乡建设工作会议的主要内容")for node in nodes:    print(f"node: {node.text}")

 

这里我们在定义 Retriever 的时候把前面注册过的 similarity 函数名称传进去即可,然后我们选取 top 3的结果。

输出结果为:​​​​​​​​​​​​​

node: cn/release/blogv2/dist/pc/themesSkin/skin-1024/skin-1024-ecd36efea2.min.css">    <script src="https://g.csdnimg.cn/lib/jquery/1.12.4/jquery.min.js" type="text/javascript"></script>    <script src="https://g.csdnimg.cn/lib/jquery-migrate/1.4.1/jquery-migrate.js" type="text/javascript"></script>    <script type="text/javascript">        var isCorporate = false;        var username =  "star_nwe";        var skinImg = "white";        var blog_address = "https://blog.csdn.net/star_nwe";        var currentUserName = "wangjian052163";        var isOwner = false;        var loginUrl = "http://passport.csdn.net/account/login?from=https://blog.csdn.net/star_nwe/article/details/141174167";        var blogUrl = "https://blog.csdn.net/";        var avatar = "https://profile-avatar.csdnimg.cn/9d615c5dd45743bea262227ce1ce205e_star_nwe.jpg!1";        var articleTitle = "大模型入门到进阶:什么是 RAG?为什么需要 RAG?RAG 的流程";        var articleDesc = "文章浏览阅读5k次,点赞22次,收藏25次。学习AI大模型是一个系统的过程,需要从基础开始,逐步深入到更高级的技术。这里给大家精心整理了一份全面的AI大模型学习资源,包括:AI大模型全套学习路线图(从入门到实战)、精品AI大模型学习书籍手册、视频教程、实战学习等录播视频,免费分享!_大模型rag";        var articleTitles = "大模型入门到进阶:什么是 RAG?为什么需要 RAG?RAG 的流程_大模型rag-CSDN博客";        var nickName = "AI老猴子";        var articleDetailUrl = "https://blog.csdn.net/star_nwe/article/details/141174167";        var vipUrlV = "https://mall.csdn.net/vip?vipSource=learningVip";        if(window.location.host.split('.').length == 3) {            blog_address = blogUrl + username;        }        var skinStatus = "White";        var blogStaticHost = "https://csdnimg.cn/release/blogv2/"          var payColumn = false    </script>        <meta name="toolbar" content='{"type":"0","fixModel":"1"}'>    <script src="https://g.csdnimg.cn/??fixed-sidebar/1.1.7/fixed-sidebar.js" type="text/javascript"></script>      <script src="https://cdn-static-devbit.csdn.net/ai100/chat/chat-search.js?v=2" type="text/javascript"></script>    <script src='//g.csdnimg.cn/common/csdn-report/report.js' type='text/javascript'></script>    <link rel="stylesheet" type="text/css" href="https://csdnimg.cn/public/sandalstrap/1.4/css/sandalstrap.min.css">    <style>        .MathJax.MathJax_Message.MathJax_Preview{            display: none        }    </style>    <script src="https://dup.baidustatic.com/js/ds.js"></script>      <script type="text/javascript">        (function(c,l,a,r,i,t,y){            c[a]=c[a]||function(){(c[a].q=c[a].q||[]).push(arguments)};            t=l.createElement(r);t.async=1;t.src="https://www.clarity.ms/tag/"+i;            y=l.getElementsByTagName(r)[0];y.parentNode.insertBefore(t,y);        })(windowdocument"clarity""script""lgtpix6r85");      </script>    <script src="/cdn_cgi_bs_captcha/static/js/waf_captcha_embedded_bs.node: opener.location.href;                    }                } catch (e) {}            }            if (referrer) {                $.post(web_root_url + "/api/waplog", {                    uuid: $("#uuid").val(),                    refer: referrer,                    ua: navigator.userAgent                }, function(e) {                    //console.log('waplog:' + e);                });            } else {                //$.post(web_root_url + "/api/waplog", {uuid:'1234567890123', refer:'https://www.baidu.com', ua:'user-agent!'}, function(e){                //console.log('waplog:' + e);                //});            }            //console.log(navigator.userAgent);            // 记录分享行为            var urlSearch = window.location.href.substring(window.location.href.indexOf("?")+1) || "";            let shareuser = "";            let shareUrl = ""            let searchArr = urlSearch.split("&")            searchArr.forEach((item,index) => {                let ab = item.split("=");                if(ab[0] === 'shareuser'){                    shareuser = ab[1];                    shareUrl = item;                }            })            // console.log('shareUrl1 ',shareUrl)            if(shareuser){                $.ajax({                    type'GET',                    url: web_root_url + '/japi/pv?article_uuid=' + uuid + '&user_uuid=' + shareuser,                    dataType'JSON',                    successfunction(st) {                        if (st.suc == 1 && st.info == 'success') {                        }else{                        }                    },                    errorfunction() {}                })            }        });    </script>    <!-- 与预览页公用 -->    <script src="https://static.bjnews.com.cn/wap/js/article.js"></script>    <script src="https://res.wx.qq.com/open/js/jweixin-1.6.0.js"></script>    <!--分享-->    <script type="text/javascript">        $.ajax({            type: "GET",            url: "https://m.bjnews.com.cn/api/wxshare?app_id=wxbad48ed25d961bb9",            // url: "https://m.bjnews.com.cn/api/wxshare?app_id=wxbad48ed25d961bb9",            dataType: 'json',            success: function(data) {                var signpackage = data.signpackage;                wx.config({                    // debug: true,                    appId: "wxbad48ed25d961bb9", // 必填,公众号的唯一标识                    // appId: "wxbad48ed25d961bb9", // 必填,公众号的唯一标识                    timestamp: signpackage.timestamp, // 必填,生成签名的时间戳                    nonceStr: signpackage.nonceStr, // 必填,生成签名的随机串                    signature: signpackage.signature, // 必填,签名,见附录1                    jsApiList: [                        'checkJsApi',                        'onMenuShareTimeline',                        'onMenuShareAppMessage'                    ],                    openTagList: ['wx-open-launch-app']                });            }        });        wx.ready(function() {            var share_uuid = "1735095611129204";            // 获取url分享后缀            var wxurlSearch = window.location.href.substring(window.location.href.indexOf("?")+1) || "";            let wxshareuser = "";//url 分享信息            if(wxurlSearch){                let wxsearchArr = wxurlSearch.split("&")                wxsearchArr.forEach((item,index) => {                    let ab = item.split("=");                    if(ab[0] === 'shareuser'){                        wxshareuser = item;                    }                })            }            // 处理分享链接            var shareLinkUrl;            shareLinkUrl = web_root_url + "/detail/" + share_uuid + ".html";            if (wxshareuser) {                shareLinkUrl = shareLinkUrl + "?"node: </span>llm<span class="token punctuation">.</span>llm <span class="token keyword">import</span> ChatSparkLLM<span class="token punctuation">,</span> ChunkPrintHandler<span class="token keyword">from</span> sparkai<span class="token punctuation">.</span>core<span class="token punctuation">.</span>messages <span class="token keyword">import</span> ChatMessage<span class="token keyword">from</span> dotenv <span class="token keyword">import</span> load_dotenvload_dotenv<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token keyword">if</span> __name__ <span class="token operator">&#61;&#61;</span> <span class="token string">&#39;__main__&#39;</span><span class="token punctuation">:</span>    spark <span class="token operator">&#61;</span> ChatSparkLLM<span class="token punctuation">(</span>        spark_api_url<span class="token operator">&#61;</span>os<span class="token punctuation">.</span>environ<span class="token punctuation">[</span><span class="token string">&#34;SPARKAI_URL&#34;</span><span class="token punctuation">]</span><span class="token punctuation">,</span>        spark_app_id<span class="token operator">&#61;</span>os<span class="token punctuation">.</span>environ<span class="token punctuation">[</span><span class="token string">&#34;SPARKAI_APP_ID&#34;</span><span class="token punctuation">]</span><span class="token punctuation">,</span>        spark_api_key<span class="token operator">&#61;</span>os<span class="token punctuation">.</span>environ<span class="token punctuation">[</span><span class="token string">&#34;SPARKAI_API_KEY&#34;</span><span class="token punctuation">]</span><span class="token punctuation">,</span>        spark_api_secret<span class="token operator">&#61;</span>os<span class="token punctuation">.</span>environ<span class="token punctuation">[</span><span class="token string">&#34;SPARKAI_API_SECRET&#34;</span><span class="token punctuation">]</span><span class="token punctuation">,</span>        spark_llm_domain<span class="token operator">&#61;</span>os<span class="token punctuation">.</span>environ<span class="token punctuation">[</span><span class="token string">&#34;SPARKAI_DOMAIN&#34;</span><span class="token punctuation">]</span><span class="token punctuation">,</span>        streaming<span class="token operator">&#61;</span><span class="token boolean">False</span><span class="token punctuation">,</span>    <span class="token punctuation">)</span>    prompt <span class="token operator">&#61;</span> <span class="token string-interpolation"><span class="token string">f&#34;&#34;&#34;              你是一个善于回答问题的助手。请使用以下提供的检索内容和自身知识来回答问题。如果你不知道答案&#xff0c;请直接说不知道&#xff0c;不要杜撰答案。请用三句话以内回答&#xff0c;保持简洁。              问题&#xff1a;</span><span class="token interpolation"><span class="token punctuation">{<!-- --></span>query<span class="token punctuation">}</span></span><span class="token string">              检索内容&#xff1a;</span><span class="token interpolation"><span class="token punctuation">{<!-- --></span>retrieval_content<span class="token punctuation">}</span></span><span class="token string">              &#34;&#34;&#34;</span></span>    messages <span class="token operator">&#61;</span> <span class="token punctuation">[</span>ChatMessage<span class="token punctuation">(</span>        role <span class="token operator">&#61;</span> <span class="token string">&#34;user&#34;</span><span class="token punctuation">,</span>        content <span class="token operator">&#61;</span> prompt    <span class="token punctuation">)</span><span class="token punctuation">]</span>    handler <span class="token operator">&#61;

可以从输出的结果中看到检索出来的文本和请求的query几乎完全没关系,因为我们使用的相似度计算方法是欧氏距离,距离越近说明文本越相似,由于是降序排列,所以 top3 对应的是距离最大的三段文本,那这三段文本和query不相似是符合预期的。

接下来我们在注册相似度计算函数是把 descend 设置为 False:

import lazyllm
from typing import List
import numpy as np
@lazyllm.tools.rag.register_similarity(mode="embedding", descend=False)
def euclidean_distance(query: List[float], node: List[float], **kwargs) -> float:
    point1 = np.array(query)
    point2 = np.array(node)
    return np.linalg.norm(point1 - point2)

然后我们再运行一次上面的检索代码,发现输出结果为:​​​​​​​

node: <!DOCTYPE html>    <html data-log-pv='{"mpc":39}'>    <head>    <title>全国住房城乡建设工作会议:大力推进商品住房销售制度改革 有力有序推行现房销售_发展</title><meta http-equiv="Cache-Control" content="no-transform" /><meta http-equiv="Cache-Control" content="no-siteapp" /><meta name="copyright" content="Copyright © 2017 Sohu.com Inc.All Rights Reserved."/><meta name="mediaid" content="财联社"/><meta property="og:type" content="news"/><meta property="og:image" content=""/><meta property="og:url" content="www.sohu.com/a/841580226_222256"/><meta property="og:release_date" content="2024-12-25 10:27"/><meta itemprop="dateUpdate" content="2024-12-25 10:27" /><meta itemprop="datePublished" content="2024-12-25 10:27" /><link rel="canonical" href="https://www.sohu.com/a/841580226_222256"/><link rel="alternate" media="only screen and(max-width: 640px)" href="m.sohu.com/a/841580226_222256"/><meta name="keywords" content="商品,现房,住房,制度,工作,改革,销售,会议,发展,大力,现房,住房,工作会议,房地产,财联社" /><meta name="description" content="财联社12月25日电,全国住房城乡建设工作会议24日至25日在北京召开。会议指出,2025年,推动构建房地产发展新模式。一是着力优化和完善住房供应体系,加快发展保障性住房,满足城镇住房困难工薪群体刚性住房需求…" /><meta property="og:description" content="财联社12月25日电,全国住房城乡建设工作会议24日至25日在北京召开。会议指出,2025年,推动构建房地产发展新模式。一是着力优化和完善住房供应体系,加快发展保障性住房,满足城镇住房困难工薪群体刚性住房需求…"/><meta property="og:title" content="全国住房城乡建设工作会议:大力推进商品住房销售制度改革 有力有序推行现房销售_发展"/>        <metacharset="utf-8"/><metaname="data-spm"content="smpc"><metaname="renderer"content="webkit"><metahttp-equiv="X-UA-Compatible"content="IE=Edge,chrome=1"/><metaname="viewport"content="width=device-width,initial-scale=1,maximum-scale=1"/><linkrel="dns-prefetch"href="//statics.itc.cn"><linkrel="dns-prefetch"href="//g1.itc.cn"><linkrel="dns-prefetch"href="//js.sohu.com"><linkrel="icon"href="//statics.itc.cn/web/static/images/pic/sohu-logo/favicon.ico"type="image/x-icon"/><linkrel="shortcut icon"href="//statics.itc.cn/web/static/images/pic/sohu-logo/favicon.ico"type="image/x-icon"/><linkrel="apple-touch-icon"sizes="57x57"href="//statics.itc.cn/web/static/images/pic/sohu-logo/logo-57.png"/><linkrel="apple-touch-icon"sizes="72x72"href="//statics.itc.cn/web/static/images/pic/sohu-logo/logo-72.png"/><linkrel="apple-touch-icon"sizes="114x114"href="//statics.itc.cn/web/static/images/pic/sohu-logo/logo-114.png"/><linkrel="apple-touch-icon"sizes="144x144"href="//statics.itc.node: js"></script>        <script>try {        var cfgs = {        channel_id: "39",        news_id: "841580226",        cms_id: "$mpNews.cmsId",        media_id: "222256",        passport: "pj6PliNNPJ@virtual-author.com",        weboUrl: "https://mp.sohu.com/profile?xpt=c29odXptdG5hdnVnMDhAc29odS5jb20=",        title: "全国住房城乡建设工作会议:大力推进商品住房销售制度改革 有力有序推行现房销售",        channel_url: "//house.sohu.com",        integralLevel: "7",        categoryId: "-1",        //abData_fd用于abtest        abData: "",        // abData_discuss:"4", // 讨论        abData_discuss: "", // 讨论        abData_fd: "",        abData_tw: "",        originalId: "$mpNews.originalId",        originalStatus: "10",        isBaiDuAd: "",        isPure: "${pure}",        reprint: false,        reprintSign: "",        secureScore: '100',        sGrade: '0',        editor: '',        hideAd: '',        keywords: "[商品, 现房, 住房, 制度, 工作, 改革, 销售, 会议, 发展, 大力, 现房, 住房, 工作会议, 房地产, 财联社]",        mpNewsExt: {            "modelId"""        },        imgsList: [                                ],        topNavigation: [                                     {                "url""http://news.sohu.com/",                "name""新闻",            }                                    , {                "url""http://sports.sohu.com/",                "name""体育",            }                                    , {                "url""http://auto.sohu.com/",                "name""汽车",            }                                    , {                "url""http://www.focus.cn/",                "name""房产",            }                                    , {                "url""http://travel.sohu.com/",                "name""旅游",            }                                    , {                "url""http://learning.sohu.com/",                "name""教育",            }                                    , {                "url""http://fashion.sohu.com/",                "name""时尚",            }                                    , {                "url""http://it.sohu.com/",                "name""科技",            }                                    , {                "url""http://business.sohu.com/",                "name""财经",            }                                    , {                "url""http://yule.sohu.com/",                "name""娱乐",            }                                    , {                "url""http://baobao.sohu.com/",                "name""母婴",            }                                    , {                "url""https://healthnews.sohu.com/",                "name""健康",            }                                    , {                "url""http://history.sohu.com/",                "name""历史",            }                                    , {                "url""http://mil.sohu.com/",                "name""军事",            }                                    , {                "url""http://chihe.sohu.com/",                "name""美食",            }                                    , {                "url""http://cul.sohu.com/",                "name""文化",            }                                    , {                "url""http://astro.sohu.com/",                "name""星座",            }                                    , {                "url""https://www.sohu.com/xchannel/TURBd01EQXhPVGt5",                "name""专题",            }                                    , {                "url""http://game.sohu.com/",                "name""游戏",            }                                    ,node: 三是大力推进商品住房销售制度改革,有力有序推行现房销售,优化预售<span>资金监管</span>。四是加快建立房屋全生命周期安全管理制度,为房屋安全提供有力保障。五是完善房地产全过程监管,整治房地产市场秩序,切实维护群众合法权益。<a href="//www.sohu.com/?strategyid=00001 " target="_blank" title="点击进入搜狐首页" id="backsohucom" style="white-space: nowrap;"><span class="backword"><i class="backsohu"></i>返回搜狐,查看更多</span></a></p>                <!-- 政务账号添加来源标示处理 -->        <!-- 政务账号添加来源标示处理 -->        <p data-role="editor-name">责任编辑:<span></span></p>    </article><div id="articleTransfer"><transfer/></div><!-- 评论禁言通知 --><div id="bannedNotice"><banned-notice/></div><div class="statement">平台声明:该文观点仅代表作者本人,搜狐号系信息发布平台,搜狐仅提供信息存储空间服务。</div><div class="bottom-relate-wrap clear type-3">            <div id="article-like" data-like-type="type-3">            <article-like/>        </div>        <div class="read-wrap">        <span class="read-num">阅读 (<em data-role="pv" data-val="$articleStat.pv"></em>)</span>    </div>    <div id="articleReport">        <report/>    </div></div>                            <div id="sohu-play-content"></div>                        </div>                    </div>                    <div data-spm="middle-banner-ad">                                            </div>                                            <div id="articleAllsee" style='height:629px'><all-see-list/></div>                                        <div class="_0u4o3bh76zbp"></div>                    <div class="god-article-bottom" id="god_bottom_banner" data-spm="ad-text-bottom" style="display:block"></div><div class="user-god clear" id="user-post" style="display:none"></div>                    <!-- 评论 -->                    <div id="meComment" style='min-height: 100px;'>                        <me-comment/>                    </div>                    <div id="commentList">                        <comment-list></comment-list>                    </div>                    <div id="discuss"></div>                    <!-- 推荐阅读 -->                                        <div style='min-height:1500px' id="groomRead">                        <groom-read/>                    </div>                                    </div>                <!-- 右侧边栏 -->                <div class="sidebar right" id="right-side-bar" data-a="${isBaiDuAd}">                    <right-side-bar/>                </div>             </div>            <div id="float-btn"> <float-btn/> </div>            <div class="left-bottom-float-fullScreenSleepContainer" style="display:none;">    <div class="left-bottom-float-fullScreenSleep" style="display:none;" data-spm="ad-fullScreenSleep">        <div class="close-tag"></div>    </div></div><div class="left-bottom-float" id="left-bottom-god" data-spm="ad-ss"></div>        </div>        <script type="text/javascript">                        window.deployEnv = "prod"                    </script>        <script src="//js.sohu.com/pv.js"></script><script src="https://g1.itc.cn/msfe-pcarti-prod/300000000000/assets/js/vendors-d407b7.js"></script><script src="https://g1.itc.cn/msfe-pcarti-prod/300000000000/assets/js/main_article-64e941.js"></script>        <script>try {        var cfgs = {        channel_id: "39",        news_id: "841580226",        cms_id: "$mpNews.cmsId",        media_id: "222256",        passport: "pj6PliNNPJ@virtual-author.com",        weboUrl: "https://mp.sohu.

这次返回的结果中可以看到都是和请求query相似度很高的文本了,符合预期。

4. 参数 batch

batch,布尔值,指示是否将所有的文档(Node)一次性给到相似度计算公式,默认值为 False。Batch主要是在需要计算均值、方差等统计特征时需要,而不是为了并行计算,LazyLLM 在 similarity 外面已经考虑了并行计算。

欧式距离计算函数

这里我们先以上面定义的欧式距离计算函数为例,把这个相似度计算函数注册一下,这里使用不同的 batch:​​​​​​​

func1 = lazyllm.tools.rag.register_similarity(euclidean_distance, mode="embedding", batch = True)func2 = lazyllm.tools.rag.register_similarity(euclidean_distance, mode="embedding")

我们先计算 batch 设置为 True 的相似度计算函数:​​​​​​

ret = func1(query_e, node_e)print(f"ret: {ret}")

输出结果报错了:

TypeError'numpy.float64' object is not iterable

我们再计算 batch 设为 False 的相似度计算函数:​​​​​​​

ret = func2(query_e, node_e)print(f"ret: {ret}")

输出结果为:

ret: {'key': [(<Node id=07cce4d6-6d4f-466b-9895-b0d77f607dbd>, 4.045985664828782)]}

因为该函数支持非批量计算,所及结果符合预期。

tfidf 相似度计算函数

接下来我们使用前面自定义的 tfidf_similarity 来看一下效果,也是先注册一下,这里使用不同的 batch:​​​​​​​

func3 = lazyllm.tools.rag.register_similarity(tfidf_similarity, mode="text", batch=True)func4 = lazyllm.tools.rag.register_similarity(tfidf_similarity, mode="text")

我们先计算 batch 设置为 True 的相似度计算函数:​​​​​​​

ret = func3(query_tnode_t)print(f"ret: {ret}")

输出结果为:

ret: [(<Node id=4a04a93d-28b4-4dbc-83d5-91150f60a648>, 0.7031197927433698)]

因为前面定义的 tfidf 相似度计算函数是支持批量计算的,所以结果符合预期。

我们再计算 batch 为 False 的相似度计算函数:​​​​​​​

ret = func4(query_tnode_t)print(f"ret: {ret}")

输出结果报错了:

TypeError'DocNode' object is not iterable

因为该函数不支持非批量计算,所及结果符合预期。

使用自定义 Similarity 搭建 RAG 应用

我们先使用 LazyLLM 默认的 similarity 函数搭建一个 RAG 应用。这里我们还是使用前面用过的 html 数据为例进行演示:​​​​​​​

import osimport lazyllmfrom lazyllm import pipeline, bind, OnlineEmbeddingModule, SentenceSplitter, Retriever, Rerankerfrom lazyllm.tools.rag import Documentprompt = 'You will play the role of an AI Q&A assistant and complete a dialogue task. In this task, you need to provide your answer based on the given context and question.'documents = Document(dataset_path=os.path.join(os.getcwd(), "rag_data"), embed=OnlineEmbeddingModule(source="glm", embed_model_name="embedding-2"), manager=False)documents.create_node_group(name="sentences", transform=SentenceSplitter, chunk_size=1024, chunk_overlap=100)with pipeline() as ppl:    ppl.prl = Retriever(documents, group_name="CoarseChunk", similarity="bm25_chinese", similarity_cut_off=0.003, topk=3)    ppl.reranker = Reranker("ModuleReranker", model=OnlineEmbeddingModule(type="rerank", source="glm", embed_model_name="rerank"), topk=1, output_format='content', join=True) | bind(query=ppl.input)    ppl.formatter = (lambda nodes, query: dict(context_str=nodes, query=query)) | bind(query=ppl.input)    ppl.llm = lazyllm.OnlineChatModule(source="glm", model="glm-4", stream=False).prompt(lazyllm.ChatPrompter(prompt, extra_keys=["context_str"]))print(ppl("为什么需要RAG"))

输出结果为:​​​​​​​​​​​​​​

需要RAG(检索增强生成)的原因在于大语言模型存在一些固有的局限性。这些局限性主要包括:1. **时效性**:大语言模型的训练数据集是截至某个时间点的,这意味着之后发生的事件、新发现、新趋势或数据更新都不会包含在模型的知识库中。由于模型的训练需要庞大的计算资源和时间,频繁更新模型以包含最新信息是不现实的。2. **覆盖性**:尽管大模型的训练数据集非常庞大,但仍可能无法涵盖所有领域的知识或特定领域的深入信息。例如,一些专业的医学、法律或技术问题可能只在特定文献中被详细讨论,而这些文献并未包含在模型的训练数据中。当大模型面对其训练数据集并未覆盖的问题时,可能会产生不准确或误导性的答案,也就是所谓的“幻觉”。为了解决这些问题,RAG通过以下流程来增强大语言模型的性能:- **检索**:根据用户的查询内容,从外部知识库中检索相关信息。- **增强**:将用户的查询内容和检索到的相关知识整合到一个提示词模板中。- **生成**:将这个经过增强的提示词输入到大语言模型中,生成更准确、更有针对性的答案。通过这样的方式,RAG能够弥补大语言模型的不足,减少误导性信息的产生,提高答案的可信度和准确性。

接下来我们在这个 RAG 应用中修改为使用前面自定义的 tfidf_similarity 方法,因为前面 tfidf_similarity 函数已经注册过了,所以这里使用很简单,只需要在 Retriever 初始化的时候指定参数 similarity 为我们自定义的相似度计算函数名即可。代码如下:​​​​​​​​​

import lazyllmimport osfrom lazyllm import pipeline, bind, OnlineEmbeddingModule, SentenceSplitter, Retriever, Rerankerfrom lazyllm.tools.rag import Documentprompt = 'You will play the role of an AI Q&A assistant and complete a dialogue task. In this task, you need to provide your answer based on the given context and question.'documents = Document(dataset_path=os.path.join(os.getcwd(), "rag_data"), embed=OnlineEmbeddingModule(source="glm", embed_model_name="embedding-2"), manager=False)documents.create_node_group(name="sentences", transform=SentenceSplitter, chunk_size=1024, chunk_overlap=100)with pipeline() as ppl:    ppl.prl = Retriever(documents, group_name="CoarseChunk", similarity="tfidf_similarity", similarity_cut_off=0.003, topk=3)    ppl.reranker = Reranker("ModuleReranker", model=OnlineEmbeddingModule(type="rerank", source="glm", embed_model_name="rerank"), topk=1, output_format='content', join=True) | bind(query=ppl.input)    ppl.formatter = (lambda nodes, query: dict(context_str=nodes, query=query)) | bind(query=ppl.input)    ppl.llm = lazyllm.OnlineChatModule(source="glm", model="glm-4", stream=False).prompt(lazyllm.ChatPrompter(prompt, extra_keys=["context_str"]))print(ppl("为什么需要RAG"))

输出内容为:​​​​​​​

需要RAG(检索增强生成)的原因在于,大规模语言模型虽然能力强大,但在生成回答或文本时仍存在一些局限性。以下是几个具体的原因:1. **知识更新限制**:大语言模型的知识是基于训练数据集的,这意味着它的知识是有限的,并且可能不会包含最新的信息。RAG通过检索外部知识库,可以引入最新的信息来增强生成的内容。2. **上下文敏感性**:在没有额外信息的情况下,大模型可能无法针对特定上下文生成最精确的回答。RAG能够检索与当前上下文相关的信息,使得生成的文本更加准确和贴合。3. **减少误导性信息**:大模型有时可能会生成不准确或误导性的信息。RAG通过整合检索到的准确信息,有助于减少这种误导性信息的产生。4. **提升生成质量**:通过结合检索到的外部信息,RAG能够为大模型提供更丰富的上下文,从而优化生成结果,提高文本的质量和相关性。5. **应对未知或稀疏数据**:对于某些特定领域或稀疏数据集,大模型可能缺乏足够的训练数据来生成高质量的输出。RAG通过检索相关领域的知识,可以帮助模型更好地处理这些情况。综上所述,RAG旨在利用检索技术来弥补大语言模型的不足,提高生成文本的准确性和相关性,同时降低误导性信息的风险。

因为 tfidf 算法不如 BM25 算法的效果好,所以最后输出的结果也不如上面的好,符合预期。

 

1.查看Similarity分数

(完整GitHub代码链接:

https://github.com/LazyAGI/Tutorial/blob/7abc91dbb82a007a78731845dd8c360ac0cc1e75/rag/codes/chapter8/view_similarity_score.py#L1

这里再补充一点,如何查看检索出来的 node 的 similarity 分数?我们其实可以通过 node 的 similarity_score 属性即可查看。这里我们还是以 tfidf_similarity 为例,代码如下:​​​​​​

import lazyllmimport osfrom lazyllm import OnlineEmbeddingModule, SentenceSplitter, Retrieverfrom lazyllm.tools.rag import Documentprompt = 'You will play the role of an AI Q&A assistant and complete a dialogue task. In this task, you need to provide your answer based on the given context and question.'documents = Document(dataset_path=os.path.join(os.getcwd(), "rag_data"), embed=OnlineEmbeddingModule(source="glm", embed_model_name="embedding-2"), manager=False)documents.create_node_group(name="sentences", transform=SentenceSplitter, chunk_size=1024, chunk_overlap=100)ppl = Retriever(documents, group_name="CoarseChunk", similarity="tfidf_similarity", similarity_cut_off=0.003, topk=3)nodes = ppl("全国住房城乡建设工作会议的主要内容")print(f"nodes: {nodes}")for node in nodes:    print(f"node: {node.similarity_score}")

输出结果为:​​​​​​​

nodes: [<Node id=8076e2ef-61c5-46b4-b622-91e9fddf1294>, <Node id=c68319a3-8f34-4cd4-948b-2cb4de22cf0c>, <Node id=7e6c3032-c9b8-4929-bce4-8fbc42417df2>]node0.24727356514169233node0.20530975832300122node0.0970966448783235

可以看到检索到的 node 有三个,最高分是0.247,另两个是0.205和0.097。注意:Retriever 返回的节点是经过去重的,所以节点的相似度分数不一定是有序的。我们怎么能把分数比较低的节点过滤掉呢?这就需要设置 Retriever 中的 similarity_cut_off 参数,这个参数的作用就是过滤掉相似度分数低于此阈值的节点。这次我们把 Retriever 中的 similarity_cut_off 设为0.21,再看一下效果:​​​​​​​

nodes: [<Node id=f2c8f5eb-4250-4aa1-ba32-81f0fa4c1017>]node0.24727356514169233

可以看到这次输出只有一个节点了,说明 similarity_cut_off 参数配置生效了。

 

【补充】在上面 RAG 的代码中,我们可以看见 Reranker 实例化时传入了 output_format 和 join 这两个参数,它们是做什么用的呢?其实它们不光是可以在 Reranker里面设置,还可以在 Retriever里设置。它们是用来做后处理的。

 

2.Retriever参数详解

(完整GitHub代码链接:

https://github.com/LazyAGI/Tutorial/blob/7abc91dbb82a007a78731845dd8c360ac0cc1e75/rag/codes/chapter8/reranker_parameters_explanation.py#L1

 

我们先来看 Retriever 里面的这两个参数的取值。output_format 的有效取值是 content 、dict 和 None,content 表示 Retriever 检索输出的是 Node 的内容,即 str 类型。dict 表示 Retriever 检索输出的是字典类型,即把 Node 中的内容转换成字典进行输出。None 表示不做任何后处理,直接以 Node 类型输出。join 的有效取值是布尔值、字符串值。当 join 为 False 时,只会对 output_format 为 content 有影响,即不对输出内容进行拼接,是以 List[str] 格式进行输出。当 join 为 True 时,会给 join 赋值空字符串。当 join 为字符串时同时 output_format 为 content,则使用 join 对 nodes中的文本进行拼接输出,即以 str 形式输出。

下面我们用代码演示一下。我们以上面的代码为例进行说明,并使用 LazyLLM 提供的 bm25_chinese相似度计算方法,然后使用国学的数据集(数据集下载方式:https://huggingface.co/datasets/LazyAGI/Chinese_Classics_Articles/tree/main)​​​​​​

from lazyllm import OnlineEmbeddingModule, SentenceSplitter, Retrieverfrom lazyllm.tools.rag import Documentprompt = 'You will play the role of an AI Q&A assistant and complete a dialogue task. In this task, you need to provide your answer based on the given context and question.'documents = Document(dataset_path="rag_master", embed=OnlineEmbeddingModule(source="glm", embed_model_name="embedding-2"), manager=False)documents.create_node_group(name="sentences", transform=SentenceSplitter, chunk_size=1024, chunk_overlap=100)ppl = Retriever(documents, group_name="sentences", similarity="cosine", similarity_cut_off=0.003, topk=3)nodes = ppl("何为天道")print(f"nodes: {nodes}")

结果输出为:

nodes: [<Node id=e25dbd6c-fb37-4181-88e5-4dc734afe2bf><Node id=3c92362f-f553-4325-9eb9-080d4ceb8757><Node id=e863a9b9-9962-4592-9c31-62bbab55052f>]

可以看到什么都不设置的话,检索出来的结果是以 Node 形式输出的。

下面我们把 output_format 设置为 dict:

ppl = Retriever(documents, group_name="sentences", similarity="cosine", similarity_cut_off=0.003, topk=3, output_format="dict")

运行上面的代码,结果输出为:

nodes: [{'content''第一章\n道可道,非常道。\n名可名,非常名。\n无,名天地之始;有,名万物之母。\n故常无,欲以观其妙;常有,欲以观其徼。\n此两者同出而异名,同谓之玄,玄之又玄,众妙之门。\n第二章\n天下皆知美之为美,斯恶已;\n皆知善之为善,斯不善已。\n故有无相生,难易相成,长短相形,高下相盈,音声相和,前后相随,恒也。\n是以圣人处无为之事,行不言之教。\n万物作而弗始,生而不有,为而不恃,功成而弗居。\n夫唯弗居,是以不去。\n第三章\n不尚贤,使民不争。\n不贵难得之货,使民不为盗。\n不见可欲,使民心不乱。\n是以圣人之治:\n虚其心,实其腹;弱其志,强其骨。\n常使民无知、无欲,使夫知者不敢为也。\n为无为,则无不治。\n第四章\n道冲而用之或不盈。\n渊兮似万物之宗,湛兮似若存。\n吾不知谁之子,象帝之先。\n第五章\n天地不仁,以万物为刍狗。\n圣人不仁,以百姓为刍狗。\n天地之间,其犹橐龠乎?\n虚而不屈,动而愈出。\n多言数穷,不如守中。\n第六章\n谷神不死,是谓玄牝。\n玄牝之门,是谓天地根。\n绵绵若存,用之不勤。\n第七章\n天长地久。\n天地所以能长且久者,以其不自生,故能长生。\n是以圣人后其身而身先,外其身而身存。\n非以其无私邪!\n故能成其私。\n第八章\n上善若水。\n水善利万物而不争,\n处众人之所恶,\n故几于道。\n居善地,心善渊,与善仁,言善信,政善治,事善能,动善时。\n夫唯不争,故无尤。\n第九章\n持而盈之,不如其已;揣而锐之,不可长保。\n金玉满堂,莫之能守;富贵而骄,自遗其咎。\n功遂身退,天之道也。\n第十章\n载营魄抱一,能无离乎。\n专气致柔,能如婴儿乎。\n涤除玄览,能无疵乎。\n爱民治国,能无为乎。\n天门开阖,能为雌乎。\n明白四达,能无知乎。\n第十一章\n三十辐共一毂,当其无,有车之用。\n埏埴以为器,当其无,有器之用。\n凿户牖以为室,当其无,有室之用。\n故有之以为利,无之以为用。''embedding': {'__default__': [0.028055770.023282783, -0.0104093730.0533456730.054548774, -0.021444315, -0.034773310.009788121, -0.00026711644, -0.0197367820.011505052, -0.0037637316, -0.0059634910.0413838550.0406101270.000985921, -0.01662170.0251560880.0017116133, -0.004511433, -0.0266573650.0275760070.01825779, -0.0191363820.0116620570.0094236360.00249212070.00252042970.0037775566, -0.0226183480.0013726661, -0.035238307, -0.0138659390.0015586380.0223869220.0080831750.073206470.08896838, -0.0036581368, -0.0025714708, -0.0211764740.03489965, -0.040128466, -0.038380902, -0.015130355, -0.013884746, -0.0152689070.025365353, -0.039423740.019842712, -0.0049218170.0370636660.0496963820.0022446876, -0.021358738, -0.006035082, -0.03891001, -0.0012354126, -0.0047493350.04006749, -0.0154858650.01656961, -0.00696180570.02573444, -0.01582435, -0.0034912361, -0.025114788, -0.0543287550.0043460196, -0.0277100240.0322668440.033886887, -0.056940170.024670593, -0.042889997, -0.07643343, -0.027408268, -0.00376454320.0498627240.0342872370.00198416550.01078216, -0.0059950864, -0.0107188010.0359498970.043880112, -0.0108695550.00253485240.06355861, -0.0128483950.0225853380.0326188170.0448307540.022901794, -0.00062107980.0231774620.005330909, -0.046414472, -0.0045889826, -0.014616943, -0.033123583, -0.0512881, -0.0020927102, -0.007836456, -0.048342444, -0.0070576523, -0.0135629430.02388182, -0.026922677, -0.0260926080.0120449630.016464080.040478050.064621150.023438517, -0.0164784230.023246740.0311713130.03628420.018051483, -0.010536480.0214650310.023477495, -0.00013451152, -0.0117159290.034541240.013441222, -0.0465133340.0061332174, -0.0035214778, -0.0214540380.050061430.0169180180.0220307520.0079460850.0459351580.0349738340.006296960.00733702350.0251897130.057120710.0099551080.0507827070.0210995820.016486313, -0.050984617, -0.04715113, -0.0297273970.040591490.0337881070.0349457450.058944894.011729e-14, -0.011751112, -0.0509789550.05046298, -0.071110204, -0.0019882438, -0.018479101, -0.051272694, -0.0100077380.006884054, -0.0345837361.1196757e-170.0039053970.020478980.0355524270.0076197950.05529369, -0.020593314, -0.0486013260.0081280590.02897840.017017765, -0.019920338, -0.028185569, -0.025367916, -0.026009448, -0.027133418, -0.0361261220.014420889, -0.0030427186, -0.002751916, -0.022126038, -0.026038717, -0.034733053, -0.0075950925, -0.020967247, -0.020053484, -0.0027370602, -0.0378914440.0049866363, -0.048347210.0342924560.0275330080.0183617980.027508227, -0.02045841, -0.0172876030.0034947034, -0.043515395, -0.0053127245, -0.0431651470.08445005, -0.027922887, -0.0160335490.035319480.0205942650.03838893, -0.06151098, -0.020862881, -0.01452097952.2020384e-05, -0.01942640.06360542, -0.0044012740.017582126, -0.0586755350.0390209780.055404720.0141373820.018975371, -0.027120763, -0.0006975507, -0.0052835485, -0.0394603760.0057850983, -0.0182543790.056609638, -0.0600446130.0202918930.006809394, -0.018061927, -0.020031512, -0.022705525, -0.027026203, -0.0183459240.0169099660.021607758, -0.0012275928, -0.013516523, -0.05039473, -0.051422264, -0.0267462540.070365995, -0.0057425452, -0.0426367070.04210949, -0.015758580.01585634, -0.0140724750.061945125, -0.061828148, -0.09160385, -0.0196488780.0052054050.04710268, -0.0231015460.0098416050.0466741920.0382449780.0411876550.063515360.0065300423, -0.0185760190.00050830960.0081122685, -0.008605757, -0.00222082460.0281585640.026416830.0018387830.025095450.00271082530.023963299, -0.041949738, -0.013529197, -0.0284067340.012666118, -0.041362340.0259419810.0293643050.0039287350.013956154, -0.0295760760.005434449, -0.028226330.033941615, -0.04924180.00779560630.048233870.035364430.00699917270.028203390.016064752, -0.042256713, -0.100301660.0083232360.00068435570.022676226, -0.000903736570.00702673760.030184641, -0.01695967, -0.0269805160.0632163, -0.0101480030.019774279, -0.0179600210.026519297, -0.045076273, -0.022787727, -0.0092743190.02229007, -0.0125021170.030040009, -0.023357610.0096850710.05651463, -0.024839891, -0.0198005880.03620323, -0.0131369330.00065234746, -0.010354659, -0.0540405770.0154526085, -0.00081201950.00162557310.0272765090.081515360.0532379630.0407132580.030182822, -0.02873101, -0.063819155, -0.01401389, -0.03257143, -0.0130671365, -0.049955063, -0.020377934, -0.0157533470.0099929640.0132772320.0108399260.0599396860.028921248, -0.0373055930.023923561, -0.010837180.041034095, -0.0136486270.039294437, -0.00560477140.032690780.030684272, -0.019674730.029130636, -0.064092520.0096028430.013003515, -0.0014473207, -0.000596308560.00079287290.03944530.0260991380.0013514259, -0.049095940.0098357810.00135663190.042203527, -0.028487109, -0.045480847, -0.0285819860.0231740470.0268214010.08383023, -0.06158942, -0.028400997, -0.0078259390.003234013, -0.0577255670.00642910230.0128539490.020342965, -0.0054711130.032541998, -0.0010940416, -0.0081099050.019187180.021351868, -0.014318267, -0.012899077, -0.02170698, -0.00698731750.009628323, -0.024537625, -0.015283444, -0.033228494, -0.0014736207, -0.056704417, -0.0048376825, -0.00083587086, -0.0132819950.020264463, -0.049195595, -0.024637522, -0.011718843, -0.0218256530.0047005790.0159636960.0321051630.00172681450.0093950460.039657350.0100193130.035936770.00356119990.0106997320.0522260070.0203465840.038028255, -0.0076610018, -0.0406218840.011577985, -0.0052217530.0185975490.031076666, -0.0048610980.0170330930.0390259060.046661437, -0.006155237, -0.0127578790.03810979, -0.009047799, -0.030244311, -0.0102574820.0459909360.02887998, -0.019625623, -0.031448547, -0.012739220.0139016970.02527813, -0.031447240.022464182, -0.027144372, -0.02345889, -0.04950639, -0.0185023560.0176885170.0103694250.02123409, -0.028073085, -0.011926072, -0.042963635, -0.0584508960.055865563, -0.00594622830.0451358740.00777392230.002659368, -0.016431257, -0.02149337, -0.03735157, -0.06444941, -0.00141055990.0057793890.078287676, -0.0042932495, -0.021970661, -0.03339254, -0.0315411870.016650775, -0.006059932, -0.044975210.02074348, -0.018448783, -0.0050848476, -0.0282535630.0144409350.01009404750.053356335, -0.083560206, -0.011833814, -0.0423676450.0367354080.0282770060.067322310.014818252, -0.0297179220.0019335257, -0.0441338350.08157332, -0.037129093, -0.023763556, -0.0210094040.024365442, -0.0225667120.036159890.019918706, -0.0045654653, -0.0147744175, -0.08768809, -0.0274345690.008535524, -0.0146432520.04218006, -0.04418376, -0.0063099093, -0.05534630.00388294020.06492348, -0.010223985, -0.025831020.04767369, -0.0031157823, -0.037336517, -0.0105275610.0128480880.012727778, -0.0731882150.019639697, -0.00357460930.011461775, -0.011808149, -0.024037987, -0.01767288, -0.017726846, -0.05001298, -0.015326746, -0.015128984, -0.039607592, -0.041593075, -0.0163928050.027638663, -0.0070960990.012386369, -0.0085322010.0218596420.0082190090.009499561, -0.013660722, -0.011760268, -0.019189020.030194886, -0.0324026570.0206891840.0384855420.040394958, -0.042311437, -0.005354799, -0.00537600370.013059683, -0.0118251150.0421208030.0107077550.0106666280.0649305950.022026783, -0.045309015, -0.013215051, -0.0035855586, -0.018754315, -0.042371470.009189613, -0.0018612920.013056050.0161309730.0167979470.042276040.00101692450.000450871130.0398595970.010306915, -0.0213128630.0136678520.00454012160.041180596, -0.0008014906, -0.016648965, -0.0473582040.0246415680.063572116, -0.053236230.027804086, -0.0231327, -0.008182450.008933412, -0.02742569, -0.024679849, -0.00702328840.079393360.0119111540.0380455480.015694190.00526080840.01345627, -0.032691434, -0.0063472530.044911380.012246293, -0.0145465150.00907633550.022790618, -0.0367795680.04574143, -0.0231121460.0396024170.0245397180.029363053, -0.057744678, -0.004233740.041941483, -0.018338485, -0.0069817240.036857042, -0.0051898616, -0.02998108, -0.0253155550.0112493010.013763156, -0.0014266107, -0.0035575930.029689435, -0.0084161720.00556582350.041912172, -0.00665963860.0144264490.0279647760.0416108440.030743865, -0.019479757, -0.0158804360.0134146790.0336855170.0250849840.0119731445, -0.032747474, -0.016790409, -0.0108213850.020628633, -0.019503403, -0.00031055597, -0.038631473, -0.005416922, -0.0442600850.009548929, -0.054613594, -0.027896475, -0.0282400510.0133055510.00268847570.0029414417, -0.04595156, -0.06401382, -0.00262401560.018469386, -0.0224765130.0114970570.0870085060.026063265, -0.069855325, -0.00500086, -0.01826556, -0.0116408490.015342398, -0.0325319840.0334236320.037746735, -0.0631916750.0099253650.0321674570.011919864, -0.0243394930.034512870.0016408766, -0.0272046010.022536965, -0.0034854359, -0.05221978, -0.0029543320.00039458516, -0.0370608680.04824446, -0.038343217, -0.008858709, -0.020064302, -0.034424454, -0.0008146635, -0.032290930.0209897830.0380410.044623570.0226276610.0214877240.0161952750.0589266570.0016035914, -0.0781729, -0.01440017, -0.065230064, -0.04473913, -0.053472330.00565643050.0267735010.0093747275, -0.00230141660.034559432, -0.0142950650.0592273440.016397202, -0.010564607, -0.0160852930.027223522, -0.039766714, -0.013913507, -0.0217241870.0198551140.046961125, -0.0509339240.028491676, -0.025045706, -0.050752120.0348408670.0308454760.037544426, -0.040440420.064948010.024027450.027269753, -0.029849720.032292705, -0.0031673112, -0.013952973, -0.024952147, -0.00554732370.020091854, -0.0025514988, -0.030173488, -0.017058272, -0.0277872640.0258212650.035203192, -0.004649104, -0.00535507360.00918115, -0.0334899430.0259830260.037477348, -0.019638410.034445020.03194153, -0.061439980.045849923, -0.0343112830.0427997750.023144597, -0.01235297, -0.0723703, -0.0171361080.007224321, -0.0083739795, -0.01820453, -0.0114215970.008455473, -0.0161296910.014105691, -0.004665744, -0.0109706060.032578386, -0.020768770.0136699425, -0.0101013975, -0.0323760770.07448448, -0.033549950.035129014, -0.0034162682, -0.0195082070.0090468660.0062084193, -0.115421490.00070306750.0158937390.0094046710.0039272520.020145050.009974044, -0.028185304, -0.01277153750.049312390.037725440.0278068910.030862588, -0.0037065197, -0.04931788, -0.0078249913.4293641e-09, -0.00517615630.025381971, -0.0086639220.015905118, -0.04079911, -0.0195752980.06621208, -0.00356832080.000257380070.044816070.054077428, -0.00722478660.009435948, -0.023140049, -0.054129170.008451867, -0.010966122, -0.0345318950.004752035, -0.04602555, -0.00232361840.009561207, -0.0233786370.022030020.0170964360.03657697, -0.000846201550.0048413950.055314980.0161930430.0120972760.0531891920.025982248, -0.032557730.009982884, -0.036482878, -0.00749707180.0913243440.0009932049, -0.0061439960.0119158905, -0.036132270.047933377, -0.008942387, -0.0264367650.01502274350.0200036840.070584180.0660807040.028907618, -0.0449253840.01017655, -0.02693322, -0.033815520.0032376520.012430394, -0.0149997260.026932973, -0.03142256, -0.023699204, -0.0191756380.0345980340.0006931077, -0.014283599, -0.0086997370.06428559, -0.0110425150.0381282160.022812221, -0.025963280.0021123243, -0.026327409, -0.0008933892, -0.0259740910.03291074, -0.00223262560.0081920970.013533811, -0.027154690.01269901450.0249645650.05334236, -0.0239780730.018099934, -0.01582323, -0.065131080.009603815, -0.042491097, -0.0066699567, -0.007812225, -0.0098942190.00089794875, -0.02697370.079909176, -0.043051008, -0.040161613, -0.012302384, -0.0093709620.00968951, -0.0035169220.02560884, -0.0031629042, -1.8169522e-270.0190836120.055952460.081102450.0099144960.0452916850.077853344, -0.05095552, -0.023066273, -0.013523827, -0.030841304, -0.0457395650.03553436, -0.001272520.011472, -0.00259805680.030634630.00188155410.0367098150.04182390.00489876650.0517057330.0276158680.0242998190.0338089, -0.0341416150.00121123310.045954734, -0.0501977130.00186326350.0381852840.0435424260.0031776570.026232130.0471260440.0326858650.0183181540.00599909530.0140305630.064932710.06270943, -0.034145653, -0.0241609890.01995237, -0.082519315, -0.026380774, -0.00084809550.006037592, -0.0273525710.00014761026, -0.020925090.0243933550.0108879460.014051744, -0.050117142, -0.0265659750.022883210.0013072105, -0.0517328460.0298373050.0086420020.00398086760.03630373, -0.00364538240.045337310.049669450.0109707860.010568572, -0.009174333, -0.0308761840.028453270.0035904122, -0.0042679270.020078829, -0.052125897, -0.0478401670.044615194, -0.012507692, -0.037026410.0200368950.0070881178, -0.0178864840.023605479, -0.01983722, -0.016965086, -0.0165531230.0574271120.02306789, -0.028244715, -0.000891977160.0067641498, -0.01275369, -0.0065226373, -0.009160864, -0.00670518170.021619145, -0.034704223, -0.007742388, -0.019153567, -0.00311012780.023993352, -6.7284605e-240.0192298130.0366782170.0260837320.0077469390.005390164, -0.024226930.0133789250.00647079670.0222719160.00479050260.027713606, -0.0590182840.066957220.00463208460.020561788, -0.054861706, -0.0132441520.0241659250.016604742]}, 'metadata': {}}, {'content''观天之道,执天之行,尽矣。\n故天有五贼,见之者昌。\n五贼在心,施行于天。\n宇宙在乎手,万化生乎身。\n天性人也,人心机也。立天之道,以定人也。\n天发杀机,移星易宿;地发杀机,龙蛇起陆;人发杀机,天地反覆;天人合发,万化定基。\n性有巧拙,可以伏藏。九窍之邪,在乎三要,可以动静。\n火生于木,祸发必克;奸生于国,时动必溃。知之修炼,谓之圣人。\n天生天杀,道之理也。天地万物之盗,万物人之盗,人万物之盗。三盗既宜,三才既安。\n故曰食其时,百骸理;动其机,万化安。人知其神之神,不知不神之所以神也。\n日月有数,大小有定,圣功生焉,神明出焉。\n其盗机也,天下莫能见,莫能知。君子得之固躬,小人得之轻命。\n瞽者善听,聋者善视。绝利一源,用师十倍。三返昼夜,用师万倍。\n心生于物,死于物,机在目。\n天之无恩而大恩生。迅雷烈风莫不蠢然。\n至乐性余,至静性廉。天之至私,用之至公。\n禽之制在气。生者死之根,死者生之根。恩生于害,害生于恩。\n愚人以天地文理圣,我以时物文理哲。''embedding': {'__default__': [0.00415541140.0075854566, -0.0070576540.0452912750.048963680.028331915, -0.059418550.0239051450.041470468, -0.0485254530.041954085, -0.009402919, -0.026280737, -0.00597984670.03535848, -0.01118226, -0.0344579220.05115644, -0.0008446922, -0.026550785, -0.0078633250.027618073, -0.011360486, -0.039742425, -0.004911823, -0.04183471, -0.0058986960.044848423, -0.030583072, -0.013616860.038467087, -0.04660302, -0.023572920.040485580.03306541, -0.0273096750.0480688330.088752195, -0.0094700590.0145056370.00196088180.024321485, -0.0624449, -0.050921675, -0.009048667, -0.014897456, -0.020900780.0161172, -0.031017896, -0.0062501850.00340549280.0173257010.050560320.007071267, -0.0085730530.0057591763, -0.0322242940.015077854, -0.0215005860.050998785, -0.012885331, -0.00151668120.0168224740.010382974, -0.01868565, -0.029895348, -0.035598777, -0.0179124060.003902434, -0.0192689710.025545066, -0.0016119897, -0.07398274, -0.0090994835, -0.015249873, -0.053859940.0414562340.0250985440.0112120950.0236218070.028716890.0155634, -0.017000291, -0.00572490040.0412926670.014244264, -0.0204873530.0097962850.00177686990.0029904752, -0.0216769860.0505752860.058154880.0410909430.038526070.00074691770.0146834655, -0.019205064, -0.02501764, -0.014589938, -0.0254961050.00298303460.009922189, -0.014940715, -0.059948150.018909076, -0.00246426930.04501117, -0.047053687, -0.014784821, -0.00209208350.0351939280.0429507640.048697557, -0.015930817, -0.018142601, -0.0211261660.00134881040.02504943, -0.007454921, -0.0262052620.032244360.063204230.0017354549, -0.0139075780.0530081840.013806328, -0.040066086, -0.011262496, -0.00783606, -0.0328097340.0422659260.003031213, -0.0116117540.00383814960.0269667040.0428001770.0329155330.0161082190.0195685270.06046945, -0.0104208910.03388435, -0.00097125750.032708555, -0.039891865, -0.029775867, -0.0144998640.0223186740.0294431370.0105198940.057602942, -2.1037191e-13, -0.040184353, -0.0237295980.059133578, -0.060146023, -0.0099975310.00714687630.02307546, -0.0084451980.025550805, -0.0628794661.1355897e-170.0052545960.0399596840.0085199350.0289871430.069844805, -0.014054391, -0.0357022140.034481020.035850395, -0.026240302, -0.060955, -0.0156162910.005105854, -0.036994290.013898516, -0.011044874, -0.0082502170.0177635250.0145686780.011624834, -0.016273065, -0.016237382, -0.030793475, -0.014173828, -0.004470327, -0.00031035003, -0.047902904, -0.033007123, -0.0370312220.03659820.0301889670.0307254340.023961129, -0.02410339, -0.0292267280.03464168, -0.024372779, -0.021495208, -0.0570384970.073897764, -0.01413330.0198944360.05053698, -0.0124599840.030188244, -0.0424957, -0.009643966, -0.0019589110.0200656480.0141953040.05589650.01836145, -0.031815484, -0.031508720.06952970.0313457440.0084211230.0014973548, -0.033643615, -0.0146294250.022442328, -0.0281675980.015380408, -0.0143780520.012282112, -0.0531930330.0199170910.019215647, -0.0338223460.0027193974, -0.0149810520.00055290776, -0.004483305, -0.0067553650.0286789520.0234755070.020434262, -0.03490839, -0.0274360050.0109724080.079654360.0011874165, -0.0454132670.028245784, -0.00632814920.009295957, -0.037302530.06491051, -0.045857605, -0.06690101, -0.0045730066, -0.0249911880.034862872, -0.0157394910.05290460.031739050.0703093, -0.00260891840.0543196240.007398687, -5.9605067e-050.017072119, -0.0088521190.00266149920.0384290670.0161595210.026893796, -0.0355622470.0380959320.00656836760.033248410.016334219, -0.023613118, -0.040397674, -0.009194677, -0.008144042, -0.016446576, -0.01151405, -0.0109283540.038340032, -0.027145639, -0.0465936550.0151692810.050768223, -0.032948480.019519920.0137970780.0353492540.0084812410.031466227, -0.003075734, -0.029220875, -0.07521815, -0.00437198530.0486723070.0058878683, -0.019860264, -0.038804630.016394598, -0.0122021, -1.4963505e-050.0443209220.0078900180.0054832306, -0.01232922450.03318079, -0.015232731, -0.013402485, -0.023898360.0055185272, -0.0281065160.022771437, -0.0302895270.026778290.083992176, -0.012977583, -0.0202571470.07025292, -0.00543805860.00113687, -0.003930704, -0.0591194480.00053930990.006615847, -0.008725304, -0.0117021070.075331910.0287739670.0292886770.048617944, -0.012453482, -0.02434141, -0.026455658, -0.0387550850.010074069, -0.043186586, -0.008087229, -0.0306954590.00285794910.010711575, -0.0278864470.0245411270.011004065, -0.065237560.0430404130.0259651910.0130716750.0230303350.0293493050.0217833850.0181284680.0068191870.0065158660.029666856, -0.063564910.039198253, -0.0110717190.023561964, -0.031980620.0235169420.0598178950.004185616, -0.008066767, -0.0090410660.0158036820.00670589230.0413373560.0052649537, -0.027125185, -0.0382131750.0145534580.042263320.070982724, -0.11455366, -0.0325853560.01022672650.011878967, -0.0333855860.0010560753, -0.015395409, -0.00441643, -0.01337264, -0.0113903250.03452036, -0.00434787060.0213694050.022401562, -0.0222051780.01836901, -0.040310632, -0.0141554010.022052731, -0.0333518720.0235399730.0066144154, -0.02636437, -0.0490858670.0212613290.025078895, -0.038480870.031188102, -0.0482616050.019504659, -0.005102396, -0.018362310.021106460.0252337120.0310626760.0010754078, -0.017802356, -0.0139149740.0145223870.0569262170.00532300770.0285019190.0487973240.0231924750.0279424620.0073998393, -0.034990810.022158591, -0.0195830170.0058571290.046547387, -0.0389320850.0404600870.0423640650.04290830.009138595, -0.0121838610.058576614, -0.030963775, -0.034653503, -0.025639888, -0.0112842330.047425315, -0.029514484, -0.036847457, -0.0197713820.0082590310.022998115, -0.0170953720.0066413945, -0.054879910.002815856, -0.033681728, -0.0141983090.026634654, -0.00822199, -0.00530929, -0.011223847, -0.011880681, -0.034348324, -0.042124510.067977875, -0.0341037740.05572993, -0.00628870070.010488381, -0.0204596910.012848595, -0.025240324, -0.068493070.025805129, -0.0111565020.05313631, -0.024320655, -0.0289160160.010576907, -0.00159990460.024717461, -0.0144396275, -0.0557156470.0300351360.054834420.010892865, -0.00754235730.03750252, -0.0053609840.061075144, -0.05138248, -0.018550297, -0.068491280.038323350.0511218230.07625229, -0.02237382, -0.025327010.005352677, -0.0081112060.073260285, -0.050452314, -0.03776045, -0.0493333940.044611394, -0.0260057130.0390772750.0220915, -0.029445095, -0.0064717205, -0.027871978, -0.0358758830.00242230060.0185164550.05704868, -0.038983510.006290837, -0.06967245, -0.0219571480.0308877230.019454552, -0.0166945010.013290308, -0.0031141401, -0.013080929, -0.016133493, -0.014472164, -0.01712892, -0.0373032730.027202567, -0.0209928040.0428680.013083941, -0.02203780.004116772, -0.014082884, -0.03672434, -0.025468608, -0.004967429, -0.043549094, -0.059921604, -0.02675990.020867042, -0.024328588, -0.009694849, -0.0086630980.0298301540.015654892, -0.01019365, -0.005135016, -0.0057884980.0056523810.018197127, -0.0482583570.0186976210.0092515980.013933649, -0.057649180.017524255, -0.0559978220.00016845911, -0.027514470.0327938350.0019559327, -0.0075224875, -0.01907267, -0.008904949, -0.0305482480.0138705090.0069685555, -0.028905436, -0.02268364, -0.020575576, -0.00301705420.015130529, -0.00084239285, -0.0295873270.037500374, -0.027868606, -0.0066606830.0312050970.012189993, -0.0001014739760.012176301, -0.0123042690.016419522, -0.031622086, -0.031915437, -0.074471430.0442598570.034456633, -0.0301440120.006155522, -0.047044933, -0.053129446, -0.003883295, -0.02238673, -0.0367940780.0189630760.0384877030.0435472650.0205300190.0135251530.03025931, -0.0129358890.00069878650.0011709720.049213756, -0.021247994, -0.010136263, -0.0089339420.040884793, -0.0144727140.04323616, -0.0245790030.044722065, -0.000441780230.022410542, -0.0436172560.0222419850.04859234, -0.0348334760.0138608750.0230024120.00644085, -0.01903299, -0.030982777, -0.022519950.0254863610.013646234, -0.0254011770.028737532, -0.00213977550.0058395150.031907808, -0.0179335250.00547676670.0119159330.04648308, -0.0016207868, -0.02563286, -0.0445124130.038957560.0260000490.0286056160.00518856760.00205580560.00133449060.0187228710.01694392, -0.048954893, -0.0022045183, -0.04172797, -0.0212445930.0136394340.0023175476, -0.057096064, -0.017740823, -0.041350298, -0.0043968866, -0.03799386, -0.0042910026, -0.09530829, -0.038248345, -0.030624647, -0.0110857375, -0.0102443770.0464642870.116509560.015119723, -0.0692813840.013058120.012854464, -0.00551326340.018360458, -0.0165773, -0.00327162650.031324998, -0.0477364140.0145194010.048135165, -0.035682708, -0.03659960.03389063, -0.0043802285, -0.018778740.048362360.020456195, -0.0567870070.0101327820.0004068694, -0.0474744850.03445135, -0.02724392, -0.03328944, -0.0029203526, -0.026272407, -0.000298901050.0134303690.0138348250.0606702160.0128826290.064323470.015768427, -0.0100938490.0249287650.012602784, -0.064383924, -0.0050611207, -0.044316787, -0.041941397, -0.043223140.0064864780.0071116807, -0.012460606, -0.00741647560.029274957, -0.0245304690.0185980460.03159384, -0.0019515440.005234270.010126147, -0.017179206, -0.018108787, -0.0071695890.0095025270.008932048, -0.0693974940.044141974, -0.005750729, -0.0457500740.0235083830.033682480.027255893, -0.0154478860.0563654860.0082779880.008696923, -0.059466016, -0.00712242860.0205474760.028926386, -0.0310889990.0034343835, -0.0097110870.0011468836, -0.013077483, -0.025331382, -0.0162293260.025458390.0119943190.008591205, -0.00150803420.02231805, -0.043377070.0403358640.026538009, -0.00621745270.0505926760.029670224, -0.0134184610.0727519, -0.0377692950.043682870.03542585, -0.020641856, -0.05900921, -0.0277325230.0072757383, -0.019184208, -0.0131694880.022744527, -0.0107969510.013560294, -0.0355951820.0239652880.0105518660.024181299, -0.03136934, -0.044932388, -0.020812577, -0.0547827740.04152298, -0.0111019990.0241593050.019186165, -0.0622833330.042109832, -0.0031904012, -0.0566089560.0308126020.0202767030.03021552, -0.0396117980.03263928, -0.008093682, -0.029934445, -0.0033287740.040175250.057420370.0657779950.011823178, -0.022649916, -0.030095868, -0.0296027333.6471364e-090.0198887460.042886823, -0.0130497260.037731685, -0.036016267, -0.0467683340.0604997050.000125725060.0160426530.0246105770.0334028270.004141988, -0.0006018001, -0.055084515, -0.0397757070.008617055, -0.01653752, -0.0131559070.016346993, -0.0235892480.024627982, -0.012017998, -0.0537845380.0378205250.035188040.0464519630.0165663510.0196178090.030327220.0517686570.0039736170.0118900010.030245746, -0.009497901, -0.001179198, -0.0268562160.0151336920.042342340.007812807, -0.025032690.017157003, -0.0612246880.049746420.033100616, -0.01518695150.0245703050.0186433050.060400340.0389015940.04535261, -0.045480450.0020181928, -0.034227967, -0.0013509532, -0.030452356, -0.018487962, -0.0238557590.05625316, -0.00090770074, -0.00640004, -0.0147777760.0279411040.016304482, -0.0387407730.0113376580.07173977, -0.035503070.023911772, -0.0016779553, -0.033899304, -0.022336006, -0.0154193010.0018633390.00087603490.009762301, -0.028453615, -0.0113620940.04275605, -0.0435218070.029829519, -0.024161170.0335993, -0.0064212680.04060249, -0.032474253, -0.076009410.004471598, -0.04126713, -0.024537350.0117643050.0153723520.0028955622, -0.02914780.05877516, -0.009823992, -0.0542679950.00678443440.03217804, -0.011592191, -0.000357149930.0287786870.0087011475, -1.7158678e-270.00179327860.035696760.054138917, -0.0086282110.0434321870.023903232, -0.0067582014, -0.03050816, -0.028248005, -0.02057966, -0.0576662120.031576764, -0.0151130445, -0.0279072730.0196042140.019528171, -0.0307403370.0131422590.04407093, -0.00295331030.00639103170.0171762460.0321320740.022815445, -0.050552312, -0.0208009560.018182818, -0.0373494360.014202859, -0.000870276770.0588792560.0056527536, -0.0055391460.0322385580.0220458770.039430930.00776169030.025640120.064691820.020434259, -0.028960383, -0.019539170.016642196, -0.04865382, -0.0097854350.000709767160.005543073, -0.000531075640.019826816, -0.0468174780.023284270.00382577860.041383106, -0.049317617, -0.0121916910.040143386, -0.020945452, -0.0159416370.014165448, -0.0141232770.00488133640.009677354, -0.0015811210.032514740.042997185, -0.015337093, -0.021302974, -0.03482498, -0.028146130.0144239160.030641912, -0.00334747280.019615268, -0.046341933, -0.0075507640.014528103, -0.024804426, -0.04388465, -0.000383028640.0181125120.0169459530.022862429, -0.064071950.02999205, -0.0293779650.038923685, -0.003280456, -0.0140948305, -0.016372854, -0.0024543288, -0.0193219020.029282054, -0.0075866296, -0.0183505680.014755779, -0.050756894, -0.038549382, -0.0260984770.00164590710.023259882, -6.1448135e-24, -0.0229526930.0220255460.0147446880.0099269590.0447780080.0182531250.04506746, -0.0197531840.0464123230.000152340290.060078166, -0.0605116820.0552750270.021075070.0074826907, -0.026740791, -0.0126987390.0170566480.050418142]}, 'metadata': {}}, {'content''天命之谓性,率性之谓道,修道之谓教。\n道也者,不可须臾(yú)离也;可离,非道也。是故君子戒慎乎其所不睹,恐惧乎其所不闻。莫见(xian)乎隐,莫显乎微,故君子慎其独也。\n喜怒哀乐之未发,谓之中;发而皆中节,谓之和。中也者,天下之大本也;和也者,天下之达道也。致中和,天地位焉,万物育焉。\n\n仲尼曰:“君子中庸,小人反中庸。君子之中庸也,君子而时中;小人之反中庸也,小人而无忌惮也。”\n\n子曰:“中庸其至矣乎!民鲜能久矣!”\n\n子曰:“道之不行也,我知之矣:知者过之,愚者不及也。道之不明也,我知之矣:贤者过之,不肖者不及也。人莫不饮食也,鲜能知味也。”''embedding': {'__default__': [0.0254262130.034025285, -0.0313208180.0165559430.063011270.0053874906, -0.038512820.05617112, -0.026647208, -0.044091930.07942905, -0.0160347040.00203014980.0094656880.022152705, -0.02026594, -0.054200440.0484576340.029878676, -0.0235493520.0164393040.0348424280.020412622, -0.00257550760.0027205113, -0.0225896760.0202918830.039224480.009154288, -0.0070374860.018945666, -0.038242333, -0.00591773170.0352700650.023194453, -0.0246634040.0455182050.078890085, -0.0049626785, -0.0013358546, -0.0208471790.008833306, -0.057070132, -0.0467806050.00708164740.009134139, -0.019501934, -0.034442246, -0.035160720.006713291, -0.0069988740.0321052040.0524663250.02138813, -0.009691537, -0.009268791, -0.04773306, -0.01108876, -0.037286460.06281455, -0.0155326770.0122684860.005829601, -0.00135778680.006930408, -0.05296581, -0.023193905, -0.0466430630.0118098760.000239750570.050946303, -0.03136553, -0.063904070.03504418, -0.01805315, -0.043444430.006776730.0137354690.0450642670.0290021640.039383190.0346837380.006368947, -0.009062640.0352923830.058860265, -0.028741615, -0.00369603960.012889149, -0.008738161, -0.0120726810.0448484870.0479138340.00254411650.0335418660.00775994970.025643757, -0.0161671280.02875357, -0.012898535, -0.03224147, -0.021762878, -0.015384127, -0.022707786, -0.0061772997, -0.0200731740.025290430.014178167, -0.018244589, -0.0082396360.021264840.00345607610.0518803260.063630230.01969195, -0.0113070830.005372883, -0.0196294910.00179456810.011739278, -0.0236688670.0097399180.0598299580.034342274, -0.00191691060.039751260.0023158423, -0.044209726, -0.00279104310.00031879256, -0.0408498160.047765710.00203811470.019105890.00277105630.0111464670.0200078430.00424841230.0051769380.0214347780.0777557, -0.00385330130.0473060420.00348373530.042881086, -0.06315707, -0.052171838, -0.048290330.0190414820.0311253370.0459593650.04621173, -1.540408e-13, -0.006997954, -0.0209213590.037427846, -0.070515640.000418915880.0263100830.024995630.00179746370.002034413, -0.0236065939.611427e-18, -0.009685626, -0.0193067580.0337063450.03567120.039736345, -0.012066647, -0.00591785930.0315036960.023679575, -0.011107136, -0.0484891270.0053288713, -0.0123187, -0.0492431630.000418088050.00033940620.015456655, -0.00284653040.014629242, -0.011859338, -0.029523129, -0.026531836, -0.004875002, -0.0096351270.0076378540.016192164, -0.0279066120.0043651653, -0.0112990370.0097028590.0208387580.00272668080.013553497, -0.020599183, -0.0525823720.047934312, -0.0211804, -0.014517992, -0.066614310.057032656, -0.0484243970.0114997520.0056307977, -0.0164093240.044111975, -0.04115124, -0.0265491180.017972990.022250125, -0.0206124990.060236566, -0.006571854, -0.00393553, -0.0311843530.0642153840.046786495, -0.00046514318, -0.00701734, -0.020175270.0087582730.0016055728, -0.026604260.009220425, -0.0147248870.010843108, -0.0442774040.0108157440.034662224, -0.026707007, -0.005213057, -0.011781672, -0.019246181, -0.019320516, -0.00165980160.0052094620.0285005740.013622352, -0.026014416, -0.03194754, -0.0106263470.070779, -0.02930651, -0.0497350540.019265305, -0.017812809, -0.03864494, -0.0570577050.047812644, -0.021030407, -0.06017914, -0.008436314, -0.005657660.040196173, -0.00154370130.033597020.071300420.0194664520.0308421180.030507375, -0.007607868, -0.014626105, -0.011076136, -2.4094108e-05, -0.00240546020.0075302810.0296317410.031458937, -0.0068964580.014952638, -0.00361523290.0278773890.016777638, -0.021922763, -0.03586302, -0.0010009537, -0.0164284950.0089247840.023603037, -0.0201151280.00865001, -0.026730474, -0.0335285630.0131497460.030888822, -0.005040427, -0.00141516320.0209470430.0289813690.007657130.032147344, -0.023203162, -0.0044152536, -0.0454228260.0168762370.0366115270.0131629750.026166268, -0.0277879780.034219008, -0.013293831, -0.0190050.0550619920.007630160.0320726560.0142395420.032104347, -0.037449952, -0.02217896, -0.0323208460.017871961, -0.039706230.026835883, -0.054085482, -0.00270092650.07634273, -0.024354596, -0.00295453980.059289258, -0.0275412740.019158483, -0.02926486, -0.059198033, -0.000454535240.03309172, -0.0585005470.038473180.066862440.03590780.0377159270.026004314, -0.010441379, -0.044468235, -0.04735834, -0.054638885, -0.003099882, -0.0168808120.00814928, -0.010210958, -0.0223010930.0029216122, -0.0124617570.0302584880.00953299, -0.031898320.0151311030.00124984420.0148358260.054116830.0207602380.0170313880.0209081380.0591723620.033446126, -0.0024910853, -0.0605189, -0.0029194264, -0.001088384, -0.0428319950.0081372150.0208135280.037613075, -0.005095976, -0.01884459, -0.0059237140.023792582, -0.0284749230.037573950.024344636, -0.035215013, -0.0234571210.0528759170.0260366830.10764514, -0.051340614, -0.0188219080.0161084290.024424875, -0.053446505, -0.0054197996, -0.0060087196, -0.0041229264, -0.0060346360.007538079, -0.018776668, -0.0133801940.0028277775, -0.00483817370.004624616, -0.030349467, -0.041400860.009394284, -0.008468681, -0.036008462, -0.011696614, -0.020296633, -0.030620858, -0.0632594750.038780067, -0.010312165, -0.053780414, -0.011514602, -0.04333725, -0.0107686980.0016200787, -0.0297524020.0073862460.0185862090.0090539510.000368540580.0229062420.0283030330.0198046420.0364450.0050071120.0149269940.029018547, -0.0150115820.00658849070.029475305, -0.0163821670.007041045, -0.0240089430.0465473530.027779775, -0.00717339060.0124063450.0113136830.03918696, -0.056599338, -0.0341519860.047533363, -0.01459687, -0.0679629, -0.0111756690.0301526320.03822690.0041881935, -0.007858538, -0.0180004030.022335991, -0.003267302, -0.0486223850.023882171, -0.034625404, -0.035592694, -0.042968493, -0.0041530310.019842803, -0.0016533697, -0.004881279, -0.045086186, -0.00111753750.0021369208, -0.0272178580.042248532, -0.0302939250.064989135, -0.0296127140.024552464, -0.033408698, -0.0101805655, -0.0029412394, -0.09153470.041386790.0130302760.06705817, -0.004188915, -0.0062264856, -0.051195238, -0.027746830.00731135, -0.013503835, -0.0524482880.0214964580.019841682, -0.0236708190.009778803, -0.00143203880.0140629960.07120368, -0.04988873, -0.00012608718, -0.040318850.0182060280.06408130.070187114, -0.00155296950.0206455380.0052610333, -0.0435957280.061265957, -0.042493634, -0.04516448, -0.0050894490.010595472, -0.0156048160.039518710.03183545, -0.0168430120.0235672, -0.040382978, -0.030780440.014075101, -0.00147679480.0424456260.0267244130.017799338, -0.08596669, -0.0185002240.051851522, -0.009828736, -0.0292041560.0114744760.0055204323, -0.004064649, -0.0333487840.005237490.022435933, -0.0466522540.024559079, -0.0164805780.040265355, -0.03305009, -0.031315356, -0.010046829, -0.01731933, -0.020532912, -0.00299099580.011842756, -0.048824314, -0.012646131, -0.0553670260.00315665940.014152518, -0.01904883, -0.0249331030.041893740.0281844570.0258122660.024073790.022633042, -0.007829883, -0.014562866, -0.0324463020.0086517780.0261884720.04299915, -0.025443240.021127068, -0.018766753, -0.00636322240.0253888610.0063838237, -0.02366371, -0.0171256440.0014911906, -0.0224124580.009111594, -0.008172784, -0.008471129, -0.023930583, -0.059438370.00966703, -0.0064070530.012494917, -0.012754937, -0.0263575130.0407052, -0.024343831, -0.00375482980.029267209, -0.008125634, -0.0106308780.02786125, -0.0073681230.0062804264, -0.032700405, -0.020568395, -0.0432762880.0158652250.0311623, -0.0434713030.055800557, -0.011932295, -0.045096148, -0.0050109215, -0.011723784, -0.0050008130.0030974280.08114780.0401643440.00364582360.0124835470.0100602360.00230184360.0065840190.0249820.035250570.0041822596, -0.0155560740.00346143990.038330406, -0.008897070.032217030.039597026.4116706e-050.00368503270.038901724, -0.0280864980.0104059110.021698542, -0.0159494490.0059260450.0334398520.0038825811, -0.025638554, -0.0238005630.01334790.0346594, -0.00953549, -0.0471495540.0245431120.017636258, -0.0131939990.0170175920.028411236, -0.00251037070.0182695070.0518458450.062203504, -0.025815979, -0.0519459430.00337211160.079987440.0254226250.009253838, -0.03604736, -0.0423371340.0145618260.04507746, -0.020416573, -0.015352677, -0.036153104, -0.0279902740.01924466, -0.045471456, -0.008539538, -0.040453684, -0.05723972, -0.01498072, -0.030530391, -0.007754218, -0.039093077, -0.03979653, -0.022286240.00084550010.00713567760.0259432420.119816860.014268126, -0.058694110.008343074, -0.040158387, -0.0165525360.018982463, -0.03898772, -0.0079805560.022687193, -0.0516582240.0253676060.019983290.021392373, -0.04342030.021215780.0185561630.0032452110.0187741720.0005500793, -0.034797590.0358574060.00040770217, -0.0359997560.028827438, -0.029466469, -0.0479308140.0265652, -0.05008721, -0.0010059318, -0.012516980.0034471190.058980020.057443540.068406570.0070246580.0065668520.030507237, -0.041550875, -0.041536470.0010598813, -0.0794358, -0.041427277, -0.06114106, -0.019902522, -0.012520727, -0.011630480.00079438320.028543016, -0.0099564910.0256552290.0437092330.0011159213, -0.0300777630.039286032, -0.032976220.0095170215, -0.0396227170.0024015250.021634275, -0.0576860230.04041061, -0.03395288, -0.044939250.0285879930.0465894970.03245823, -0.024442730.0363532940.0086188790.0035412086, -0.0462545680.01484673750.0166348780.03069350.011313652, -0.0310876540.0097682240.007366226, -0.0245210420.013069051, -0.0368039460.0459548640.008253860.013556772, -0.0283805450.01906237, -0.0173735450.02040689, -0.004991811, -0.0113232190.038479466, -0.010787252, -0.0198266260.06550503, -0.0215124860.043110030.063991840.01899978, -0.06393204, -0.0528577530.0026758595, -0.0232972220.00633986670.0291478840.00763348440.029746521, -0.016748957, -0.00216336250.0212566610.022264387, -0.022884088, -0.016388448, -0.033683803, -0.022389290.055876125, -0.00529252270.013481841, -0.0209584, -0.035067280.0162239630.0012435996, -0.046723302, -0.00053879660.0125559970.04083436, -0.012232332, -0.0160817950.0062284134, -0.027968781, -0.0223763440.021191330.0346315020.023184470.0446683130.0023679107, -0.04467385, -0.017894422.5628348e-09, -0.0108428550.0264042020.018929183, -0.008009062, -0.03902315, -0.035394460.0451428440.0016768258, -0.00595710.0489177030.051160040.00053710610.004794815, -0.016625853, -0.062219560.0022883292, -0.037917495, -0.026748385, -0.01792442, -0.00450905920.0344877020.029278198, -0.0353418920.0191855080.0239903070.0407818560.033953443, -0.0128681480.0526921640.005785138, -0.005816314, -0.0177702050.052668773, -0.011497479, -0.0042478456, -0.0411633440.0177274180.0195242690.00286726230.0272069690.0098760445, -0.018392580.064081036, -0.018610574, -0.031679490.0281122030.0380094460.0270180460.0349643530.06139204, -0.000934609, -0.013590096, -0.0112873, -0.016028268, -0.065215790.009233209, -0.0109752240.014953974, -0.013712477, -0.035961337, -0.012009760.016888669, -0.0254347, -0.03294505, -0.0110858230.074079536, -0.0412309060.0080667810.016872587, -0.0095884380.0151293940.004597649, -0.012082546, -0.0205048, -0.011996115, -0.042583340.010901120.025189904, -0.0265451730.011398977, -0.0353429060.011842086, -0.0466537550.0023188903, -0.027082337, -0.0530236920.009047406, -0.04184604, -0.00271260370.022172572, -0.0019665314, -0.020220164, -0.00550295650.059257552, -0.038126208, -0.035578760.011252940.024858668, -0.00564227770.00490456350.0115539890.026200697, -1.432319e-27, -0.00108609460.092685140.06090386, -0.0057963610.0251798280.06804877, -0.0507071880.02777679, -0.031054588, -0.019260377, -0.048696230.04939678, -0.024753766, -0.00725078860.062012840.038341094, -0.023940781, -0.00448048530.06028650.0129556380.0413284670.0122152660.0418577830.05578706, -0.04423121, -0.0300890640.039828192, -0.0254883780.0445665870.016367070.05779351, -0.0294815130.00535006450.0513234550.034585970.013696377, -0.0351920430.043303550.0582196560.049582005, -0.02016202, -0.0366385540.020224864, -0.039496087, -0.0150625040.00164610680.015035624, -0.03048053, -0.027442321, -0.0197957080.061282612, -0.00357668290.049931448, -0.044324502, -0.034937955, -0.055052124, -0.0023345670.0011809984, -0.00053580320.014579384, -0.0067273750.01961919, -0.0242301240.0222609790.0416386350.015936956, -0.0129846865, -0.028698431, -0.0233113130.020681119, -0.0220048130.00453420170.10454673, -0.009072320.0180192720.027693322, -0.0456784180.008820441, -0.0182479150.0022897322, -0.00451689540.0330645, -0.046881303, -0.02397729, -0.036530850.03674319, -0.027837344, -0.02493074, -0.025111457, -0.02385346, -0.0262388980.019991593, -0.00067811715, -0.0147816320.024634153, -0.016004443, -0.00521948470.0246270170.00321425640.013624659, -5.867911e-24, -9.8801356e-050.076493440.020020690.0291811540.030946780.0213314480.029769164, -0.011064590.0358678180.00590473740.044848107, -0.0344377120.0567312430.00321589460.018998314, -0.070217706, -0.0297422910.0346440520.024905458]}, 'metadata': {}}]

结果是以 dict 的形式输出的,之前没配置时是以 Node 的形式输出的。这个时候对 join 要么不设置,要么设置为 False对结果没有影响,不能设置为 True 或字符串否则会报错。

ppl = Retriever(documents, group_name="sentences", similarity="cosine", similarity_cut_off=0.003, topk=3, output_format="dict", join=True)

结果如下所示:

AssertionErrorOnly content output can be joined

然后当我们把 output_format 设置为 content:

ppl = Retriever(documents, group_name="sentences", similarity="cosine", similarity_cut_off=0.003, topk=3, output_format="content")

运行上面的代码,结果输出如下:

nodes: ['观天之道,执天之行,尽矣。\n故天有五贼,见之者昌。\n五贼在心,施行于天。\n宇宙在乎手,万化生乎身。\n天性人也,人心机也。立天之道,以定人也。\n天发杀机,移星易宿;地发杀机,龙蛇起陆;人发杀机,天地反覆;天人合发,万化定基。\n性有巧拙,可以伏藏。九窍之邪,在乎三要,可以动静。\n火生于木,祸发必克;奸生于国,时动必溃。知之修炼,谓之圣人。\n天生天杀,道之理也。天地万物之盗,万物人之盗,人万物之盗。三盗既宜,三才既安。\n故曰食其时,百骸理;动其机,万化安。人知其神之神,不知不神之所以神也。\n日月有数,大小有定,圣功生焉,神明出焉。\n其盗机也,天下莫能见,莫能知。君子得之固躬,小人得之轻命。\n瞽者善听,聋者善视。绝利一源,用师十倍。三返昼夜,用师万倍。\n心生于物,死于物,机在目。\n天之无恩而大恩生。迅雷烈风莫不蠢然。\n至乐性余,至静性廉。天之至私,用之至公。\n禽之制在气。生者死之根,死者生之根。恩生于害,害生于恩。\n愚人以天地文理圣,我以时物文理哲。''天命之谓性,率性之谓道,修道之谓教。\n道也者,不可须臾(yú)离也;可离,非道也。是故君子戒慎乎其所不睹,恐惧乎其所不闻。莫见(xian)乎隐,莫显乎微,故君子慎其独也。\n喜怒哀乐之未发,谓之中;发而皆中节,谓之和。中也者,天下之大本也;和也者,天下之达道也。致中和,天地位焉,万物育焉。\n\n仲尼曰:“君子中庸,小人反中庸。君子之中庸也,君子而时中;小人之反中庸也,小人而无忌惮也。”\n\n子曰:“中庸其至矣乎!民鲜能久矣!”\n\n子曰:“道之不行也,我知之矣:知者过之,愚者不及也。道之不明也,我知之矣:贤者过之,不肖者不及也。人莫不饮食也,鲜能知味也。”''第一章\n道可道,非常道。\n名可名,非常名。\n无,名天地之始;有,名万物之母。\n故常无,欲以观其妙;常有,欲以观其徼。\n此两者同出而异名,同谓之玄,玄之又玄,众妙之门。\n第二章\n天下皆知美之为美,斯恶已;\n皆知善之为善,斯不善已。\n故有无相生,难易相成,长短相形,高下相盈,音声相和,前后相随,恒也。\n是以圣人处无为之事,行不言之教。\n万物作而弗始,生而不有,为而不恃,功成而弗居。\n夫唯弗居,是以不去。\n第三章\n不尚贤,使民不争。\n不贵难得之货,使民不为盗。\n不见可欲,使民心不乱。\n是以圣人之治:\n虚其心,实其腹;弱其志,强其骨。\n常使民无知、无欲,使夫知者不敢为也。\n为无为,则无不治。\n第四章\n道冲而用之或不盈。\n渊兮似万物之宗,湛兮似若存。\n吾不知谁之子,象帝之先。\n第五章\n天地不仁,以万物为刍狗。\n圣人不仁,以百姓为刍狗。\n天地之间,其犹橐龠乎?\n虚而不屈,动而愈出。\n多言数穷,不如守中。\n第六章\n谷神不死,是谓玄牝。\n玄牝之门,是谓天地根。\n绵绵若存,用之不勤。\n第七章\n天长地久。\n天地所以能长且久者,以其不自生,故能长生。\n是以圣人后其身而身先,外其身而身存。\n非以其无私邪!\n故能成其私。\n第八章\n上善若水。\n水善利万物而不争,\n处众人之所恶,\n故几于道。\n居善地,心善渊,与善仁,言善信,政善治,事善能,动善时。\n夫唯不争,故无尤。\n第九章\n持而盈之,不如其已;揣而锐之,不可长保。\n金玉满堂,莫之能守;富贵而骄,自遗其咎。\n功遂身退,天之道也。\n第十章\n载营魄抱一,能无离乎。\n专气致柔,能如婴儿乎。\n涤除玄览,能无疵乎。\n爱民治国,能无为乎。\n天门开阖,能为雌乎。\n明白四达,能无知乎。\n第十一章\n三十辐共一毂,当其无,有车之用。\n埏埴以为器,当其无,有器之用。\n凿户牖以为室,当其无,有室之用。\n故有之以为利,无之以为用。']

因为没有配置 join,它的默认值是 False,所以是以 List[str]格式输出的。

这时候如果把 join设置为 True呢?

ppl = Retriever(documents, group_name="sentences", similarity="cosine", similarity_cut_off=0.003, topk=3, output_format="content", join=True)

结果输出如下:​​​​​​​

  •  
nodes: 天命之谓性,率性之谓道,修道之谓教。道也者,不可须臾(yú)离也;可离,非道也。是故君子戒慎乎其所不睹,恐惧乎其所不闻。莫见(xian)乎隐,莫显乎微,故君子慎其独也。喜怒哀乐之未发,谓之中;发而皆中节,谓之和。中也者,天下之大本也;和也者,天下之达道也。致中和,天地位焉,万物育焉。仲尼曰:“君子中庸,小人反中庸。君子之中庸也,君子而时中;小人之反中庸也,小人而无忌惮也。”子曰:“中庸其至矣乎!民鲜能久矣!”子曰:“道之不行也,我知之矣:知者过之,愚者不及也。道之不明也,我知之矣:贤者过之,不肖者不及也。人莫不饮食也,鲜能知味也。”第一章道可道,非常道。名可名,非常名。无,名天地之始;有,名万物之母。故常无,欲以观其妙;常有,欲以观其徼。此两者同出而异名,同谓之玄,玄之又玄,众妙之门。第二章天下皆知美之为美,斯恶已;皆知善之为善,斯不善已。故有无相生,难易相成,长短相形,高下相盈,音声相和,前后相随,恒也。是以圣人处无为之事,行不言之教。万物作而弗始,生而不有,为而不恃,功成而弗居。夫唯弗居,是以不去。第三章不尚贤,使民不争。不贵难得之货,使民不为盗。不见可欲,使民心不乱。是以圣人之治:虚其心,实其腹;弱其志,强其骨。常使民无知、无欲,使夫知者不敢为也。为无为,则无不治。第四章道冲而用之或不盈。渊兮似万物之宗,湛兮似若存。吾不知谁之子,象帝之先。第五章天地不仁,以万物为刍狗。圣人不仁,以百姓为刍狗。天地之间,其犹橐龠乎?虚而不屈,动而愈出。多言数穷,不如守中。第六章谷神不死,是谓玄牝。玄牝之门,是谓天地根。绵绵若存,用之不勤。第七章天长地久。天地所以能长且久者,以其不自生,故能长生。是以圣人后其身而身先,外其身而身存。非以其无私邪!故能成其私。第八章上善若水。水善利万物而不争,处众人之所恶,故几于道。居善地,心善渊,与善仁,言善信,政善治,事善能,动善时。夫唯不争,故无尤。第九章持而盈之,不如其已;揣而锐之,不可长保。金玉满堂,莫之能守;富贵而骄,自遗其咎。功遂身退,天之道也。第十章载营魄抱一,能无离乎。专气致柔,能如婴儿乎。涤除玄览,能无疵乎。爱民治国,能无为乎。天门开阖,能为雌乎。明白四达,能无知乎。第十一章三十辐共一毂,当其无,有车之用。埏埴以为器,当其无,有器之用。凿户牖以为室,当其无,有室之用。故有之以为利,无之以为用。观天之道,执天之行,尽矣。故天有五贼,见之者昌。五贼在心,施行于天。宇宙在乎手,万化生乎身。天性人也,人心机也。立天之道,以定人也。天发杀机,移星易宿;地发杀机,龙蛇起陆;人发杀机,天地反覆;天人合发,万化定基。性有巧拙,可以伏藏。九窍之邪,在乎三要,可以动静。火生于木,祸发必克;奸生于国,时动必溃。知之修炼,谓之圣人。天生天杀,道之理也。天地万物之盗,万物人之盗,人万物之盗。三盗既宜,三才既安。故曰食其时,百骸理;动其机,万化安。人知其神之神,不知不神之所以神也。日月有数,大小有定,圣功生焉,神明出焉。其盗机也,天下莫能见,莫能知。君子得之固躬,小人得之轻命。瞽者善听,聋者善视。绝利一源,用师十倍。三返昼夜,用师万倍。心生于物,死于物,机在目。天之无恩而大恩生。迅雷烈风莫不蠢然。至乐性余,至静性廉。天之至私,用之至公。禽之制在气。生者死之根,死者生之根。恩生于害,害生于恩。愚人以天地文理圣,我以时物文理哲。

这时候结果是以空字符串拼接成的文本,如果把 join 设置为 '11111111111111111111111111111' 呢?

 

结果输出如下:

 

  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
nodes: 第一章道可道,非常道。名可名,非常名。无,名天地之始;有,名万物之母。故常无,欲以观其妙;常有,欲以观其徼。此两者同出而异名,同谓之玄,玄之又玄,众妙之门。第二章天下皆知美之为美,斯恶已;皆知善之为善,斯不善已。故有无相生,难易相成,长短相形,高下相盈,音声相和,前后相随,恒也。是以圣人处无为之事,行不言之教。万物作而弗始,生而不有,为而不恃,功成而弗居。夫唯弗居,是以不去。第三章不尚贤,使民不争。不贵难得之货,使民不为盗。不见可欲,使民心不乱。是以圣人之治:虚其心,实其腹;弱其志,强其骨。常使民无知、无欲,使夫知者不敢为也。为无为,则无不治。第四章道冲而用之或不盈。渊兮似万物之宗,湛兮似若存。吾不知谁之子,象帝之先。第五章天地不仁,以万物为刍狗。圣人不仁,以百姓为刍狗。天地之间,其犹橐龠乎?虚而不屈,动而愈出。多言数穷,不如守中。第六章谷神不死,是谓玄牝。玄牝之门,是谓天地根。绵绵若存,用之不勤。第七章天长地久。天地所以能长且久者,以其不自生,故能长生。是以圣人后其身而身先,外其身而身存。非以其无私邪!故能成其私。第八章上善若水。水善利万物而不争,处众人之所恶,故几于道。居善地,心善渊,与善仁,言善信,政善治,事善能,动善时。夫唯不争,故无尤。第九章持而盈之,不如其已;揣而锐之,不可长保。金玉满堂,莫之能守;富贵而骄,自遗其咎。功遂身退,天之道也。第十章载营魄抱一,能无离乎。专气致柔,能如婴儿乎。涤除玄览,能无疵乎。爱民治国,能无为乎。天门开阖,能为雌乎。明白四达,能无知乎。第十一章三十辐共一毂,当其无,有车之用。埏埴以为器,当其无,有器之用。凿户牖以为室,当其无,有室之用。故有之以为利,无之以为用。11111111111111111111111111111观天之道,执天之行,尽矣。故天有五贼,见之者昌。五贼在心,施行于天。宇宙在乎手,万化生乎身。天性人也,人心机也。立天之道,以定人也。天发杀机,移星易宿;地发杀机,龙蛇起陆;人发杀机,天地反覆;天人合发,万化定基。性有巧拙,可以伏藏。九窍之邪,在乎三要,可以动静。火生于木,祸发必克;奸生于国,时动必溃。知之修炼,谓之圣人。天生天杀,道之理也。天地万物之盗,万物人之盗,人万物之盗。三盗既宜,三才既安。故曰食其时,百骸理;动其机,万化安。人知其神之神,不知不神之所以神也。日月有数,大小有定,圣功生焉,神明出焉。其盗机也,天下莫能见,莫能知。君子得之固躬,小人得之轻命。瞽者善听,聋者善视。绝利一源,用师十倍。三返昼夜,用师万倍。心生于物,死于物,机在目。天之无恩而大恩生。迅雷烈风莫不蠢然。至乐性余,至静性廉。天之至私,用之至公。禽之制在气。生者死之根,死者生之根。恩生于害,害生于恩。愚人以天地文理圣,我以时物文理哲。11111111111111111111111111111天命之谓性,率性之谓道,修道之谓教。道也者,不可须臾(yú)离也;可离,非道也。是故君子戒慎乎其所不睹,恐惧乎其所不闻。莫见(xian)乎隐,莫显乎微,故君子慎其独也。喜怒哀乐之未发,谓之中;发而皆中节,谓之和。中也者,天下之大本也;和也者,天下之达道也。致中和,天地位焉,万物育焉。仲尼曰:“君子中庸,小人反中庸。君子之中庸也,君子而时中;小人之反中庸也,小人而无忌惮也。”子曰:“中庸其至矣乎!民鲜能久矣!”子曰:“道之不行也,我知之矣:知者过之,愚者不及也。道之不明也,我知之矣:贤者过之,不肖者不及也。人莫不饮食也,鲜能知味也。”

 

可以看出结果是由三段文本拼接而成的字符串。

 

3.Reranker参数详解

(完整GitHub代码链接:

https://github.com/LazyAGI/Tutorial/blob/7abc91dbb82a007a78731845dd8c360ac0cc1e75/rag/codes/chapter8/reranker_parameters_explanation.py#L1

 

Reranker 中的这两个参数值和上面是一样的,这里我们就不多说了,直接看代码。我们让 Reranker输出分数最高的两个结果。​​​​​

import lazyllmfrom lazyllm import pipeline, bind, OnlineEmbeddingModule, SentenceSplitter, Retriever, Rerankerfrom lazyllm.tools.rag import Documentprompt = 'You will play the role of an AI Q&A assistant and complete a dialogue task. In this task, you need to provide your answer based on the given context and question.'documents = Document(dataset_path="rag_master", embed=OnlineEmbeddingModule(source="glm", embed_model_name="embedding-2"), manager=False)documents.create_node_group(name="sentences", transform=SentenceSplitter, chunk_size=1024, chunk_overlap=100)with pipeline() as ppl:    ppl.retriever = Retriever(documents, group_name="sentences", similarity="cosine", similarity_cut_off=0.003, topk=3)    ppl.reranker = Reranker(name="ModuleReranker", model=lazyllm.OnlineEmbeddingModule(type="rerank", source="glm", embed_model_name="rerank"), topk=2) | bind(query=ppl.input)nodes = ppl("何为天道")print(f"nodes: {nodes}")

结果输出如下:

nodes: [<Node id=a8713a16-fc5b-4904-a06f-9e8c55d11988><Node id=058767c8-de0e-4d17-addc-23816ae8e2d2>]

可以看到什么都不设置的话,结果是以 Node 的形式输出的。

接下来把 output_format 设置为 dict:

ppl.reranker = Reranker(name="ModuleReranker", model=lazyllm.OnlineEmbeddingModule(type="rerank"source="glm", embed_model_name="rerank"), topk=2, output_format="dict") | bind(query=ppl.input)

 

结果输出如下:

nodes: [{'content''观天之道,执天之行,尽矣。\n故天有五贼,见之者昌。\n五贼在心,施行于天。\n宇宙在乎手,万化生乎身。\n天性人也,人心机也。立天之道,以定人也。\n天发杀机,移星易宿;地发杀机,龙蛇起陆;人发杀机,天地反覆;天人合发,万化定基。\n性有巧拙,可以伏藏。九窍之邪,在乎三要,可以动静。\n火生于木,祸发必克;奸生于国,时动必溃。知之修炼,谓之圣人。\n天生天杀,道之理也。天地万物之盗,万物人之盗,人万物之盗。三盗既宜,三才既安。\n故曰食其时,百骸理;动其机,万化安。人知其神之神,不知不神之所以神也。\n日月有数,大小有定,圣功生焉,神明出焉。\n其盗机也,天下莫能见,莫能知。君子得之固躬,小人得之轻命。\n瞽者善听,聋者善视。绝利一源,用师十倍。三返昼夜,用师万倍。\n心生于物,死于物,机在目。\n天之无恩而大恩生。迅雷烈风莫不蠢然。\n至乐性余,至静性廉。天之至私,用之至公。\n禽之制在气。生者死之根,死者生之根。恩生于害,害生于恩。\n愚人以天地文理圣,我以时物文理哲。''embedding': {'__default__': [0.00415541140.0075854566, -0.0070576540.0452912750.048963680.028331915, -0.059418550.0239051450.041470468, -0.0485254530.041954085, -0.009402919, -0.026280737, -0.00597984670.03535848, -0.01118226, -0.0344579220.05115644, -0.0008446922, -0.026550785, -0.0078633250.027618073, -0.011360486, -0.039742425, -0.004911823, -0.04183471, -0.0058986960.044848423, -0.030583072, -0.013616860.038467087, -0.04660302, -0.023572920.040485580.03306541, -0.0273096750.0480688330.088752195, -0.0094700590.0145056370.00196088180.024321485, -0.0624449, -0.050921675, -0.009048667, -0.014897456, -0.020900780.0161172, -0.031017896, -0.0062501850.00340549280.0173257010.050560320.007071267, -0.0085730530.0057591763, -0.0322242940.015077854, -0.0215005860.050998785, -0.012885331, -0.00151668120.0168224740.010382974, -0.01868565, -0.029895348, -0.035598777, -0.0179124060.003902434, -0.0192689710.025545066, -0.0016119897, -0.07398274, -0.0090994835, -0.015249873, -0.053859940.0414562340.0250985440.0112120950.0236218070.028716890.0155634, -0.017000291, -0.00572490040.0412926670.014244264, -0.0204873530.0097962850.00177686990.0029904752, -0.0216769860.0505752860.058154880.0410909430.038526070.00074691770.0146834655, -0.019205064, -0.02501764, -0.014589938, -0.0254961050.00298303460.009922189, -0.014940715, -0.059948150.018909076, -0.00246426930.04501117, -0.047053687, -0.014784821, -0.00209208350.0351939280.0429507640.048697557, -0.015930817, -0.018142601, -0.0211261660.00134881040.02504943, -0.007454921, -0.0262052620.032244360.063204230.0017354549, -0.0139075780.0530081840.013806328, -0.040066086, -0.011262496, -0.00783606, -0.0328097340.0422659260.003031213, -0.0116117540.00383814960.0269667040.0428001770.0329155330.0161082190.0195685270.06046945, -0.0104208910.03388435, -0.00097125750.032708555, -0.039891865, -0.029775867, -0.0144998640.0223186740.0294431370.0105198940.057602942, -2.1037191e-13, -0.040184353, -0.0237295980.059133578, -0.060146023, -0.0099975310.00714687630.02307546, -0.0084451980.025550805, -0.0628794661.1355897e-170.0052545960.0399596840.0085199350.0289871430.069844805, -0.014054391, -0.0357022140.034481020.035850395, -0.026240302, -0.060955, -0.0156162910.005105854, -0.036994290.013898516, -0.011044874, -0.0082502170.0177635250.0145686780.011624834, -0.016273065, -0.016237382, -0.030793475, -0.014173828, -0.004470327, -0.00031035003, -0.047902904, -0.033007123, -0.0370312220.03659820.0301889670.0307254340.023961129, -0.02410339, -0.0292267280.03464168, -0.024372779, -0.021495208, -0.0570384970.073897764, -0.01413330.0198944360.05053698, -0.0124599840.030188244, -0.0424957, -0.009643966, -0.0019589110.0200656480.0141953040.05589650.01836145, -0.031815484, -0.031508720.06952970.0313457440.0084211230.0014973548, -0.033643615, -0.0146294250.022442328, -0.0281675980.015380408, -0.0143780520.012282112, -0.0531930330.0199170910.019215647, -0.0338223460.0027193974, -0.0149810520.00055290776, -0.004483305, -0.0067553650.0286789520.0234755070.020434262, -0.03490839, -0.0274360050.0109724080.079654360.0011874165, -0.0454132670.028245784, -0.00632814920.009295957, -0.037302530.06491051, -0.045857605, -0.06690101, -0.0045730066, -0.0249911880.034862872, -0.0157394910.05290460.031739050.0703093, -0.00260891840.0543196240.007398687, -5.9605067e-050.017072119, -0.0088521190.00266149920.0384290670.0161595210.026893796, -0.0355622470.0380959320.00656836760.033248410.016334219, -0.023613118, -0.040397674, -0.009194677, -0.008144042, -0.016446576, -0.01151405, -0.0109283540.038340032, -0.027145639, -0.0465936550.0151692810.050768223, -0.032948480.019519920.0137970780.0353492540.0084812410.031466227, -0.003075734, -0.029220875, -0.07521815, -0.00437198530.0486723070.0058878683, -0.019860264, -0.038804630.016394598, -0.0122021, -1.4963505e-050.0443209220.0078900180.0054832306, -0.01232922450.03318079, -0.015232731, -0.013402485, -0.023898360.0055185272, -0.0281065160.022771437, -0.0302895270.026778290.083992176, -0.012977583, -0.0202571470.07025292, -0.00543805860.00113687, -0.003930704, -0.0591194480.00053930990.006615847, -0.008725304, -0.0117021070.075331910.0287739670.0292886770.048617944, -0.012453482, -0.02434141, -0.026455658, -0.0387550850.010074069, -0.043186586, -0.008087229, -0.0306954590.00285794910.010711575, -0.0278864470.0245411270.011004065, -0.065237560.0430404130.0259651910.0130716750.0230303350.0293493050.0217833850.0181284680.0068191870.0065158660.029666856, -0.063564910.039198253, -0.0110717190.023561964, -0.031980620.0235169420.0598178950.004185616, -0.008066767, -0.0090410660.0158036820.00670589230.0413373560.0052649537, -0.027125185, -0.0382131750.0145534580.042263320.070982724, -0.11455366, -0.0325853560.01022672650.011878967, -0.0333855860.0010560753, -0.015395409, -0.00441643, -0.01337264, -0.0113903250.03452036, -0.00434787060.0213694050.022401562, -0.0222051780.01836901, -0.040310632, -0.0141554010.022052731, -0.0333518720.0235399730.0066144154, -0.02636437, -0.0490858670.0212613290.025078895, -0.038480870.031188102, -0.0482616050.019504659, -0.005102396, -0.018362310.021106460.0252337120.0310626760.0010754078, -0.017802356, -0.0139149740.0145223870.0569262170.00532300770.0285019190.0487973240.0231924750.0279424620.0073998393, -0.034990810.022158591, -0.0195830170.0058571290.046547387, -0.0389320850.0404600870.0423640650.04290830.009138595, -0.0121838610.058576614, -0.030963775, -0.034653503, -0.025639888, -0.0112842330.047425315, -0.029514484, -0.036847457, -0.0197713820.0082590310.022998115, -0.0170953720.0066413945, -0.054879910.002815856, -0.033681728, -0.0141983090.026634654, -0.00822199, -0.00530929, -0.011223847, -0.011880681, -0.034348324, -0.042124510.067977875, -0.0341037740.05572993, -0.00628870070.010488381, -0.0204596910.012848595, -0.025240324, -0.068493070.025805129, -0.0111565020.05313631, -0.024320655, -0.0289160160.010576907, -0.00159990460.024717461, -0.0144396275, -0.0557156470.0300351360.054834420.010892865, -0.00754235730.03750252, -0.0053609840.061075144, -0.05138248, -0.018550297, -0.068491280.038323350.0511218230.07625229, -0.02237382, -0.025327010.005352677, -0.0081112060.073260285, -0.050452314, -0.03776045, -0.0493333940.044611394, -0.0260057130.0390772750.0220915, -0.029445095, -0.0064717205, -0.027871978, -0.0358758830.00242230060.0185164550.05704868, -0.038983510.006290837, -0.06967245, -0.0219571480.0308877230.019454552, -0.0166945010.013290308, -0.0031141401, -0.013080929, -0.016133493, -0.014472164, -0.01712892, -0.0373032730.027202567, -0.0209928040.0428680.013083941, -0.02203780.004116772, -0.014082884, -0.03672434, -0.025468608, -0.004967429, -0.043549094, -0.059921604, -0.02675990.020867042, -0.024328588, -0.009694849, -0.0086630980.0298301540.015654892, -0.01019365, -0.005135016, -0.0057884980.0056523810.018197127, -0.0482583570.0186976210.0092515980.013933649, -0.057649180.017524255, -0.0559978220.00016845911, -0.027514470.0327938350.0019559327, -0.0075224875, -0.01907267, -0.008904949, -0.0305482480.0138705090.0069685555, -0.028905436, -0.02268364, -0.020575576, -0.00301705420.015130529, -0.00084239285, -0.0295873270.037500374, -0.027868606, -0.0066606830.0312050970.012189993, -0.0001014739760.012176301, -0.0123042690.016419522, -0.031622086, -0.031915437, -0.074471430.0442598570.034456633, -0.0301440120.006155522, -0.047044933, -0.053129446, -0.003883295, -0.02238673, -0.0367940780.0189630760.0384877030.0435472650.0205300190.0135251530.03025931, -0.0129358890.00069878650.0011709720.049213756, -0.021247994, -0.010136263, -0.0089339420.040884793, -0.0144727140.04323616, -0.0245790030.044722065, -0.000441780230.022410542, -0.0436172560.0222419850.04859234, -0.0348334760.0138608750.0230024120.00644085, -0.01903299, -0.030982777, -0.022519950.0254863610.013646234, -0.0254011770.028737532, -0.00213977550.0058395150.031907808, -0.0179335250.00547676670.0119159330.04648308, -0.0016207868, -0.02563286, -0.0445124130.038957560.0260000490.0286056160.00518856760.00205580560.00133449060.0187228710.01694392, -0.048954893, -0.0022045183, -0.04172797, -0.0212445930.0136394340.0023175476, -0.057096064, -0.017740823, -0.041350298, -0.0043968866, -0.03799386, -0.0042910026, -0.09530829, -0.038248345, -0.030624647, -0.0110857375, -0.0102443770.0464642870.116509560.015119723, -0.0692813840.013058120.012854464, -0.00551326340.018360458, -0.0165773, -0.00327162650.031324998, -0.0477364140.0145194010.048135165, -0.035682708, -0.03659960.03389063, -0.0043802285, -0.018778740.048362360.020456195, -0.0567870070.0101327820.0004068694, -0.0474744850.03445135, -0.02724392, -0.03328944, -0.0029203526, -0.026272407, -0.000298901050.0134303690.0138348250.0606702160.0128826290.064323470.015768427, -0.0100938490.0249287650.012602784, -0.064383924, -0.0050611207, -0.044316787, -0.041941397, -0.043223140.0064864780.0071116807, -0.012460606, -0.00741647560.029274957, -0.0245304690.0185980460.03159384, -0.0019515440.005234270.010126147, -0.017179206, -0.018108787, -0.0071695890.0095025270.008932048, -0.0693974940.044141974, -0.005750729, -0.0457500740.0235083830.033682480.027255893, -0.0154478860.0563654860.0082779880.008696923, -0.059466016, -0.00712242860.0205474760.028926386, -0.0310889990.0034343835, -0.0097110870.0011468836, -0.013077483, -0.025331382, -0.0162293260.025458390.0119943190.008591205, -0.00150803420.02231805, -0.043377070.0403358640.026538009, -0.00621745270.0505926760.029670224, -0.0134184610.0727519, -0.0377692950.043682870.03542585, -0.020641856, -0.05900921, -0.0277325230.0072757383, -0.019184208, -0.0131694880.022744527, -0.0107969510.013560294, -0.0355951820.0239652880.0105518660.024181299, -0.03136934, -0.044932388, -0.020812577, -0.0547827740.04152298, -0.0111019990.0241593050.019186165, -0.0622833330.042109832, -0.0031904012, -0.0566089560.0308126020.0202767030.03021552, -0.0396117980.03263928, -0.008093682, -0.029934445, -0.0033287740.040175250.057420370.0657779950.011823178, -0.022649916, -0.030095868, -0.0296027333.6471364e-090.0198887460.042886823, -0.0130497260.037731685, -0.036016267, -0.0467683340.0604997050.000125725060.0160426530.0246105770.0334028270.004141988, -0.0006018001, -0.055084515, -0.0397757070.008617055, -0.01653752, -0.0131559070.016346993, -0.0235892480.024627982, -0.012017998, -0.0537845380.0378205250.035188040.0464519630.0165663510.0196178090.030327220.0517686570.0039736170.0118900010.030245746, -0.009497901, -0.001179198, -0.0268562160.0151336920.042342340.007812807, -0.025032690.017157003, -0.0612246880.049746420.033100616, -0.01518695150.0245703050.0186433050.060400340.0389015940.04535261, -0.045480450.0020181928, -0.034227967, -0.0013509532, -0.030452356, -0.018487962, -0.0238557590.05625316, -0.00090770074, -0.00640004, -0.0147777760.0279411040.016304482, -0.0387407730.0113376580.07173977, -0.035503070.023911772, -0.0016779553, -0.033899304, -0.022336006, -0.0154193010.0018633390.00087603490.009762301, -0.028453615, -0.0113620940.04275605, -0.0435218070.029829519, -0.024161170.0335993, -0.0064212680.04060249, -0.032474253, -0.076009410.004471598, -0.04126713, -0.024537350.0117643050.0153723520.0028955622, -0.02914780.05877516, -0.009823992, -0.0542679950.00678443440.03217804, -0.011592191, -0.000357149930.0287786870.0087011475, -1.7158678e-270.00179327860.035696760.054138917, -0.0086282110.0434321870.023903232, -0.0067582014, -0.03050816, -0.028248005, -0.02057966, -0.0576662120.031576764, -0.0151130445, -0.0279072730.0196042140.019528171, -0.0307403370.0131422590.04407093, -0.00295331030.00639103170.0171762460.0321320740.022815445, -0.050552312, -0.0208009560.018182818, -0.0373494360.014202859, -0.000870276770.0588792560.0056527536, -0.0055391460.0322385580.0220458770.039430930.00776169030.025640120.064691820.020434259, -0.028960383, -0.019539170.016642196, -0.04865382, -0.0097854350.000709767160.005543073, -0.000531075640.019826816, -0.0468174780.023284270.00382577860.041383106, -0.049317617, -0.0121916910.040143386, -0.020945452, -0.0159416370.014165448, -0.0141232770.00488133640.009677354, -0.0015811210.032514740.042997185, -0.015337093, -0.021302974, -0.03482498, -0.028146130.0144239160.030641912, -0.00334747280.019615268, -0.046341933, -0.0075507640.014528103, -0.024804426, -0.04388465, -0.000383028640.0181125120.0169459530.022862429, -0.064071950.02999205, -0.0293779650.038923685, -0.003280456, -0.0140948305, -0.016372854, -0.0024543288, -0.0193219020.029282054, -0.0075866296, -0.0183505680.014755779, -0.050756894, -0.038549382, -0.0260984770.00164590710.023259882, -6.1448135e-24, -0.0229526930.0220255460.0147446880.0099269590.0447780080.0182531250.04506746, -0.0197531840.0464123230.000152340290.060078166, -0.0605116820.0552750270.021075070.0074826907, -0.026740791, -0.0126987390.0170566480.050418142]}, 'metadata': {}}, {'content''天命之谓性,率性之谓道,修道之谓教。\n道也者,不可须臾(yú)离也;可离,非道也。是故君子戒慎乎其所不睹,恐惧乎其所不闻。莫见(xian)乎隐,莫显乎微,故君子慎其独也。\n喜怒哀乐之未发,谓之中;发而皆中节,谓之和。中也者,天下之大本也;和也者,天下之达道也。致中和,天地位焉,万物育焉。\n\n仲尼曰:“君子中庸,小人反中庸。君子之中庸也,君子而时中;小人之反中庸也,小人而无忌惮也。”\n\n子曰:“中庸其至矣乎!民鲜能久矣!”\n\n子曰:“道之不行也,我知之矣:知者过之,愚者不及也。道之不明也,我知之矣:贤者过之,不肖者不及也。人莫不饮食也,鲜能知味也。”''embedding': {'__default__': [0.0254262130.034025285, -0.0313208180.0165559430.063011270.0053874906, -0.038512820.05617112, -0.026647208, -0.044091930.07942905, -0.0160347040.00203014980.0094656880.022152705, -0.02026594, -0.054200440.0484576340.029878676, -0.0235493520.0164393040.0348424280.020412622, -0.00257550760.0027205113, -0.0225896760.0202918830.039224480.009154288, -0.0070374860.018945666, -0.038242333, -0.00591773170.0352700650.023194453, -0.0246634040.0455182050.078890085, -0.0049626785, -0.0013358546, -0.0208471790.008833306, -0.057070132, -0.0467806050.00708164740.009134139, -0.019501934, -0.034442246, -0.035160720.006713291, -0.0069988740.0321052040.0524663250.02138813, -0.009691537, -0.009268791, -0.04773306, -0.01108876, -0.037286460.06281455, -0.0155326770.0122684860.005829601, -0.00135778680.006930408, -0.05296581, -0.023193905, -0.0466430630.0118098760.000239750570.050946303, -0.03136553, -0.063904070.03504418, -0.01805315, -0.043444430.006776730.0137354690.0450642670.0290021640.039383190.0346837380.006368947, -0.009062640.0352923830.058860265, -0.028741615, -0.00369603960.012889149, -0.008738161, -0.0120726810.0448484870.0479138340.00254411650.0335418660.00775994970.025643757, -0.0161671280.02875357, -0.012898535, -0.03224147, -0.021762878, -0.015384127, -0.022707786, -0.0061772997, -0.0200731740.025290430.014178167, -0.018244589, -0.0082396360.021264840.00345607610.0518803260.063630230.01969195, -0.0113070830.005372883, -0.0196294910.00179456810.011739278, -0.0236688670.0097399180.0598299580.034342274, -0.00191691060.039751260.0023158423, -0.044209726, -0.00279104310.00031879256, -0.0408498160.047765710.00203811470.019105890.00277105630.0111464670.0200078430.00424841230.0051769380.0214347780.0777557, -0.00385330130.0473060420.00348373530.042881086, -0.06315707, -0.052171838, -0.048290330.0190414820.0311253370.0459593650.04621173, -1.540408e-13, -0.006997954, -0.0209213590.037427846, -0.070515640.000418915880.0263100830.024995630.00179746370.002034413, -0.0236065939.611427e-18, -0.009685626, -0.0193067580.0337063450.03567120.039736345, -0.012066647, -0.00591785930.0315036960.023679575, -0.011107136, -0.0484891270.0053288713, -0.0123187, -0.0492431630.000418088050.00033940620.015456655, -0.00284653040.014629242, -0.011859338, -0.029523129, -0.026531836, -0.004875002, -0.0096351270.0076378540.016192164, -0.0279066120.0043651653, -0.0112990370.0097028590.0208387580.00272668080.013553497, -0.020599183, -0.0525823720.047934312, -0.0211804, -0.014517992, -0.066614310.057032656, -0.0484243970.0114997520.0056307977, -0.0164093240.044111975, -0.04115124, -0.0265491180.017972990.022250125, -0.0206124990.060236566, -0.006571854, -0.00393553, -0.0311843530.0642153840.046786495, -0.00046514318, -0.00701734, -0.020175270.0087582730.0016055728, -0.026604260.009220425, -0.0147248870.010843108, -0.0442774040.0108157440.034662224, -0.026707007, -0.005213057, -0.011781672, -0.019246181, -0.019320516, -0.00165980160.0052094620.0285005740.013622352, -0.026014416, -0.03194754, -0.0106263470.070779, -0.02930651, -0.0497350540.019265305, -0.017812809, -0.03864494, -0.0570577050.047812644, -0.021030407, -0.06017914, -0.008436314, -0.005657660.040196173, -0.00154370130.033597020.071300420.0194664520.0308421180.030507375, -0.007607868, -0.014626105, -0.011076136, -2.4094108e-05, -0.00240546020.0075302810.0296317410.031458937, -0.0068964580.014952638, -0.00361523290.0278773890.016777638, -0.021922763, -0.03586302, -0.0010009537, -0.0164284950.0089247840.023603037, -0.0201151280.00865001, -0.026730474, -0.0335285630.0131497460.030888822, -0.005040427, -0.00141516320.0209470430.0289813690.007657130.032147344, -0.023203162, -0.0044152536, -0.0454228260.0168762370.0366115270.0131629750.026166268, -0.0277879780.034219008, -0.013293831, -0.0190050.0550619920.007630160.0320726560.0142395420.032104347, -0.037449952, -0.02217896, -0.0323208460.017871961, -0.039706230.026835883, -0.054085482, -0.00270092650.07634273, -0.024354596, -0.00295453980.059289258, -0.0275412740.019158483, -0.02926486, -0.059198033, -0.000454535240.03309172, -0.0585005470.038473180.066862440.03590780.0377159270.026004314, -0.010441379, -0.044468235, -0.04735834, -0.054638885, -0.003099882, -0.0168808120.00814928, -0.010210958, -0.0223010930.0029216122, -0.0124617570.0302584880.00953299, -0.031898320.0151311030.00124984420.0148358260.054116830.0207602380.0170313880.0209081380.0591723620.033446126, -0.0024910853, -0.0605189, -0.0029194264, -0.001088384, -0.0428319950.0081372150.0208135280.037613075, -0.005095976, -0.01884459, -0.0059237140.023792582, -0.0284749230.037573950.024344636, -0.035215013, -0.0234571210.0528759170.0260366830.10764514, -0.051340614, -0.0188219080.0161084290.024424875, -0.053446505, -0.0054197996, -0.0060087196, -0.0041229264, -0.0060346360.007538079, -0.018776668, -0.0133801940.0028277775, -0.00483817370.004624616, -0.030349467, -0.041400860.009394284, -0.008468681, -0.036008462, -0.011696614, -0.020296633, -0.030620858, -0.0632594750.038780067, -0.010312165, -0.053780414, -0.011514602, -0.04333725, -0.0107686980.0016200787, -0.0297524020.0073862460.0185862090.0090539510.000368540580.0229062420.0283030330.0198046420.0364450.0050071120.0149269940.029018547, -0.0150115820.00658849070.029475305, -0.0163821670.007041045, -0.0240089430.0465473530.027779775, -0.00717339060.0124063450.0113136830.03918696, -0.056599338, -0.0341519860.047533363, -0.01459687, -0.0679629, -0.0111756690.0301526320.03822690.0041881935, -0.007858538, -0.0180004030.022335991, -0.003267302, -0.0486223850.023882171, -0.034625404, -0.035592694, -0.042968493, -0.0041530310.019842803, -0.0016533697, -0.004881279, -0.045086186, -0.00111753750.0021369208, -0.0272178580.042248532, -0.0302939250.064989135, -0.0296127140.024552464, -0.033408698, -0.0101805655, -0.0029412394, -0.09153470.041386790.0130302760.06705817, -0.004188915, -0.0062264856, -0.051195238, -0.027746830.00731135, -0.013503835, -0.0524482880.0214964580.019841682, -0.0236708190.009778803, -0.00143203880.0140629960.07120368, -0.04988873, -0.00012608718, -0.040318850.0182060280.06408130.070187114, -0.00155296950.0206455380.0052610333, -0.0435957280.061265957, -0.042493634, -0.04516448, -0.0050894490.010595472, -0.0156048160.039518710.03183545, -0.0168430120.0235672, -0.040382978, -0.030780440.014075101, -0.00147679480.0424456260.0267244130.017799338, -0.08596669, -0.0185002240.051851522, -0.009828736, -0.0292041560.0114744760.0055204323, -0.004064649, -0.0333487840.005237490.022435933, -0.0466522540.024559079, -0.0164805780.040265355, -0.03305009, -0.031315356, -0.010046829, -0.01731933, -0.020532912, -0.00299099580.011842756, -0.048824314, -0.012646131, -0.0553670260.00315665940.014152518, -0.01904883, -0.0249331030.041893740.0281844570.0258122660.024073790.022633042, -0.007829883, -0.014562866, -0.0324463020.0086517780.0261884720.04299915, -0.025443240.021127068, -0.018766753, -0.00636322240.0253888610.0063838237, -0.02366371, -0.0171256440.0014911906, -0.0224124580.009111594, -0.008172784, -0.008471129, -0.023930583, -0.059438370.00966703, -0.0064070530.012494917, -0.012754937, -0.0263575130.0407052, -0.024343831, -0.00375482980.029267209, -0.008125634, -0.0106308780.02786125, -0.0073681230.0062804264, -0.032700405, -0.020568395, -0.0432762880.0158652250.0311623, -0.0434713030.055800557, -0.011932295, -0.045096148, -0.0050109215, -0.011723784, -0.0050008130.0030974280.08114780.0401643440.00364582360.0124835470.0100602360.00230184360.0065840190.0249820.035250570.0041822596, -0.0155560740.00346143990.038330406, -0.008897070.032217030.039597026.4116706e-050.00368503270.038901724, -0.0280864980.0104059110.021698542, -0.0159494490.0059260450.0334398520.0038825811, -0.025638554, -0.0238005630.01334790.0346594, -0.00953549, -0.0471495540.0245431120.017636258, -0.0131939990.0170175920.028411236, -0.00251037070.0182695070.0518458450.062203504, -0.025815979, -0.0519459430.00337211160.079987440.0254226250.009253838, -0.03604736, -0.0423371340.0145618260.04507746, -0.020416573, -0.015352677, -0.036153104, -0.0279902740.01924466, -0.045471456, -0.008539538, -0.040453684, -0.05723972, -0.01498072, -0.030530391, -0.007754218, -0.039093077, -0.03979653, -0.022286240.00084550010.00713567760.0259432420.119816860.014268126, -0.058694110.008343074, -0.040158387, -0.0165525360.018982463, -0.03898772, -0.0079805560.022687193, -0.0516582240.0253676060.019983290.021392373, -0.04342030.021215780.0185561630.0032452110.0187741720.0005500793, -0.034797590.0358574060.00040770217, -0.0359997560.028827438, -0.029466469, -0.0479308140.0265652, -0.05008721, -0.0010059318, -0.012516980.0034471190.058980020.057443540.068406570.0070246580.0065668520.030507237, -0.041550875, -0.041536470.0010598813, -0.0794358, -0.041427277, -0.06114106, -0.019902522, -0.012520727, -0.011630480.00079438320.028543016, -0.0099564910.0256552290.0437092330.0011159213, -0.0300777630.039286032, -0.032976220.0095170215, -0.0396227170.0024015250.021634275, -0.0576860230.04041061, -0.03395288, -0.044939250.0285879930.0465894970.03245823, -0.024442730.0363532940.0086188790.0035412086, -0.0462545680.01484673750.0166348780.03069350.011313652, -0.0310876540.0097682240.007366226, -0.0245210420.013069051, -0.0368039460.0459548640.008253860.013556772, -0.0283805450.01906237, -0.0173735450.02040689, -0.004991811, -0.0113232190.038479466, -0.010787252, -0.0198266260.06550503, -0.0215124860.043110030.063991840.01899978, -0.06393204, -0.0528577530.0026758595, -0.0232972220.00633986670.0291478840.00763348440.029746521, -0.016748957, -0.00216336250.0212566610.022264387, -0.022884088, -0.016388448, -0.033683803, -0.022389290.055876125, -0.00529252270.013481841, -0.0209584, -0.035067280.0162239630.0012435996, -0.046723302, -0.00053879660.0125559970.04083436, -0.012232332, -0.0160817950.0062284134, -0.027968781, -0.0223763440.021191330.0346315020.023184470.0446683130.0023679107, -0.04467385, -0.017894422.5628348e-09, -0.0108428550.0264042020.018929183, -0.008009062, -0.03902315, -0.035394460.0451428440.0016768258, -0.00595710.0489177030.051160040.00053710610.004794815, -0.016625853, -0.062219560.0022883292, -0.037917495, -0.026748385, -0.01792442, -0.00450905920.0344877020.029278198, -0.0353418920.0191855080.0239903070.0407818560.033953443, -0.0128681480.0526921640.005785138, -0.005816314, -0.0177702050.052668773, -0.011497479, -0.0042478456, -0.0411633440.0177274180.0195242690.00286726230.0272069690.0098760445, -0.018392580.064081036, -0.018610574, -0.031679490.0281122030.0380094460.0270180460.0349643530.06139204, -0.000934609, -0.013590096, -0.0112873, -0.016028268, -0.065215790.009233209, -0.0109752240.014953974, -0.013712477, -0.035961337, -0.012009760.016888669, -0.0254347, -0.03294505, -0.0110858230.074079536, -0.0412309060.0080667810.016872587, -0.0095884380.0151293940.004597649, -0.012082546, -0.0205048, -0.011996115, -0.042583340.010901120.025189904, -0.0265451730.011398977, -0.0353429060.011842086, -0.0466537550.0023188903, -0.027082337, -0.0530236920.009047406, -0.04184604, -0.00271260370.022172572, -0.0019665314, -0.020220164, -0.00550295650.059257552, -0.038126208, -0.035578760.011252940.024858668, -0.00564227770.00490456350.0115539890.026200697, -1.432319e-27, -0.00108609460.092685140.06090386, -0.0057963610.0251798280.06804877, -0.0507071880.02777679, -0.031054588, -0.019260377, -0.048696230.04939678, -0.024753766, -0.00725078860.062012840.038341094, -0.023940781, -0.00448048530.06028650.0129556380.0413284670.0122152660.0418577830.05578706, -0.04423121, -0.0300890640.039828192, -0.0254883780.0445665870.016367070.05779351, -0.0294815130.00535006450.0513234550.034585970.013696377, -0.0351920430.043303550.0582196560.049582005, -0.02016202, -0.0366385540.020224864, -0.039496087, -0.0150625040.00164610680.015035624, -0.03048053, -0.027442321, -0.0197957080.061282612, -0.00357668290.049931448, -0.044324502, -0.034937955, -0.055052124, -0.0023345670.0011809984, -0.00053580320.014579384, -0.0067273750.01961919, -0.0242301240.0222609790.0416386350.015936956, -0.0129846865, -0.028698431, -0.0233113130.020681119, -0.0220048130.00453420170.10454673, -0.009072320.0180192720.027693322, -0.0456784180.008820441, -0.0182479150.0022897322, -0.00451689540.0330645, -0.046881303, -0.02397729, -0.036530850.03674319, -0.027837344, -0.02493074, -0.025111457, -0.02385346, -0.0262388980.019991593, -0.00067811715, -0.0147816320.024634153, -0.016004443, -0.00521948470.0246270170.00321425640.013624659, -5.867911e-24, -9.8801356e-050.076493440.020020690.0291811540.030946780.0213314480.029769164, -0.011064590.0358678180.00590473740.044848107, -0.0344377120.0567312430.00321589460.018998314, -0.070217706, -0.0297422910.0346440520.024905458]}, 'metadata': {}}]

可以看到结果是以 dict的形式输出的。同样的,这个时候对 join 要么不设置,要么设置为 False对结果没有影响,不能设置为 True 或字符串否则会报错。

ppl.reranker = Reranker(name="ModuleReranker", model=lazyllm.OnlineEmbeddingModule(type="rerank"source="glm", embed_model_name="rerank"), topk=2, output_format="dict"join=True) | bind(query=ppl.input)

 

结果报错如下:

AssertionError: Only content output can be joined

接下来我们把 output_format 设置为 content:

ppl.reranker = Reranker(name="ModuleReranker", model=lazyllm.OnlineEmbeddingModule(type="rerank"source="glm", embed_model_name="rerank"), topk=2, output_format="content" ) | bind(query=ppl.input)

 

结果如下:

nodes: ['观天之道,执天之行,尽矣。\n故天有五贼,见之者昌。\n五贼在心,施行于天。\n宇宙在乎手,万化生乎身。\n天性人也,人心机也。立天之道,以定人也。\n天发杀机,移星易宿;地发杀机,龙蛇起陆;人发杀机,天地反覆;天人合发,万化定基。\n性有巧拙,可以伏藏。九窍之邪,在乎三要,可以动静。\n火生于木,祸发必克;奸生于国,时动必溃。知之修炼,谓之圣人。\n天生天杀,道之理也。天地万物之盗,万物人之盗,人万物之盗。三盗既宜,三才既安。\n故曰食其时,百骸理;动其机,万化安。人知其神之神,不知不神之所以神也。\n日月有数,大小有定,圣功生焉,神明出焉。\n其盗机也,天下莫能见,莫能知。君子得之固躬,小人得之轻命。\n瞽者善听,聋者善视。绝利一源,用师十倍。三返昼夜,用师万倍。\n心生于物,死于物,机在目。\n天之无恩而大恩生。迅雷烈风莫不蠢然。\n至乐性余,至静性廉。天之至私,用之至公。\n禽之制在气。生者死之根,死者生之根。恩生于害,害生于恩。\n愚人以天地文理圣,我以时物文理哲。''天命之谓性,率性之谓道,修道之谓教。\n道也者,不可须臾(yú)离也;可离,非道也。是故君子戒慎乎其所不睹,恐惧乎其所不闻。莫见(xian)乎隐,莫显乎微,故君子慎其独也。\n喜怒哀乐之未发,谓之中;发而皆中节,谓之和。中也者,天下之大本也;和也者,天下之达道也。致中和,天地位焉,万物育焉。\n\n仲尼曰:“君子中庸,小人反中庸。君子之中庸也,君子而时中;小人之反中庸也,小人而无忌惮也。”\n\n子曰:“中庸其至矣乎!民鲜能久矣!”\n\n子曰:“道之不行也,我知之矣:知者过之,愚者不及也。道之不明也,我知之矣:贤者过之,不肖者不及也。人莫不饮食也,鲜能知味也。”']

 

结果是以 List[str]的形式输出的。

如果把 join设置为True呢?

ppl.reranker = Reranker(name="ModuleReranker", model=lazyllm.OnlineEmbeddingModule(type="rerank"source="glm", embed_model_name="rerank"), topk=2, output_format="content"join=True) | bind(query=ppl.input)

 

结果如下:​​​​​​​

nodes: 观天之道,执天之行,尽矣。故天有五贼,见之者昌。五贼在心,施行于天。宇宙在乎手,万化生乎身。天性人也,人心机也。立天之道,以定人也。天发杀机,移星易宿;地发杀机,龙蛇起陆;人发杀机,天地反覆;天人合发,万化定基。性有巧拙,可以伏藏。九窍之邪,在乎三要,可以动静。火生于木,祸发必克;奸生于国,时动必溃。知之修炼,谓之圣人。天生天杀,道之理也。天地万物之盗,万物人之盗,人万物之盗。三盗既宜,三才既安。故曰食其时,百骸理;动其机,万化安。人知其神之神,不知不神之所以神也。日月有数,大小有定,圣功生焉,神明出焉。其盗机也,天下莫能见,莫能知。君子得之固躬,小人得之轻命。瞽者善听,聋者善视。绝利一源,用师十倍。三返昼夜,用师万倍。心生于物,死于物,机在目。天之无恩而大恩生。迅雷烈风莫不蠢然。至乐性余,至静性廉。天之至私,用之至公。禽之制在气。生者死之根,死者生之根。恩生于害,害生于恩。愚人以天地文理圣,我以时物文理哲。天命之谓性,率性之谓道,修道之谓教。道也者,不可须臾(yú)离也;可离,非道也。是故君子戒慎乎其所不睹,恐惧乎其所不闻。莫见(xian)乎隐,莫显乎微,故君子慎其独也。喜怒哀乐之未发,谓之中;发而皆中节,谓之和。中也者,天下之大本也;和也者,天下之达道也。致中和,天地位焉,万物育焉。仲尼曰:“君子中庸,小人反中庸。君子之中庸也,君子而时中;小人之反中庸也,小人而无忌惮也。”子曰:“中庸其至矣乎!民鲜能久矣!”子曰:“道之不行也,我知之矣:知者过之,愚者不及也。道之不明也,我知之矣:贤者过之,不肖者不及也。人莫不饮食也,鲜能知味也。”

 

可以看到结果是以字符串的形式输出的,如果把 join 设置为 '11111111111111111111111111111' 呢?

ppl.reranker = Reranker(name="ModuleReranker", model=lazyllm.OnlineEmbeddingModule(type="rerank"source="glm", embed_model_name="rerank"), topk=2, output_format="content"join='11111111111111111111111111111') | bind(query=ppl.input)

结果如下所示:​​​​​​​

nodes: 观天之道,执天之行,尽矣。故天有五贼,见之者昌。五贼在心,施行于天。宇宙在乎手,万化生乎身。天性人也,人心机也。立天之道,以定人也。天发杀机,移星易宿;地发杀机,龙蛇起陆;人发杀机,天地反覆;天人合发,万化定基。性有巧拙,可以伏藏。九窍之邪,在乎三要,可以动静。火生于木,祸发必克;奸生于国,时动必溃。知之修炼,谓之圣人。天生天杀,道之理也。天地万物之盗,万物人之盗,人万物之盗。三盗既宜,三才既安。故曰食其时,百骸理;动其机,万化安。人知其神之神,不知不神之所以神也。日月有数,大小有定,圣功生焉,神明出焉。其盗机也,天下莫能见,莫能知。君子得之固躬,小人得之轻命。瞽者善听,聋者善视。绝利一源,用师十倍。三返昼夜,用师万倍。心生于物,死于物,机在目。天之无恩而大恩生。迅雷烈风莫不蠢然。至乐性余,至静性廉。天之至私,用之至公。禽之制在气。生者死之根,死者生之根。恩生于害,害生于恩。愚人以天地文理圣,我以时物文理哲。11111111111111111111111111111天命之谓性,率性之谓道,修道之谓教。道也者,不可须臾(yú)离也;可离,非道也。是故君子戒慎乎其所不睹,恐惧乎其所不闻。莫见(xian)乎隐,莫显乎微,故君子慎其独也。喜怒哀乐之未发,谓之中;发而皆中节,谓之和。中也者,天下之大本也;和也者,天下之达道也。致中和,天地位焉,万物育焉。仲尼曰:“君子中庸,小人反中庸。君子之中庸也,君子而时中;小人之反中庸也,小人而无忌惮也。”子曰:“中庸其至矣乎!民鲜能久矣!”子曰:“道之不行也,我知之矣:知者过之,愚者不及也。道之不明也,我知之矣:贤者过之,不肖者不及也。人莫不饮食也,鲜能知味也。”

可以看出结果是由两个 文本段拼接而成。可以看出 Retriever和Reranker的后处理是一样的。

(数据集的下载方式:

https://huggingface.co/datasets/LazyAGI/Chinese_Classics_Articles/tree/main)

更多技术细节,欢迎移步“LazyLLM”gzh~

原文链接:https://my.oschina.net/u/8690838/blog/18691162
关注公众号

低调大师中文资讯倾力打造互联网数据资讯、行业资源、电子商务、移动互联网、网络营销平台。

持续更新报道IT业界、互联网、市场资讯、驱动更新,是最及时权威的产业资讯及硬件资讯报道平台。

转载内容版权归作者及来源网站所有,本站原创内容转载请注明来源。

文章评论

共有0条评论来说两句吧...

文章二维码

扫描即可查看该文章

点击排行

推荐阅读

最新文章