TruLens Langchain 集成示例代码
本示例代码将指导您创建简单 LLM 链,并学习如何记录它并获得 LLM 响应的反馈。
1. 安装依赖
pip install trulens_eval==0.18.3 openai==1.3.7
2. 设置 API 密钥
此示例代码需要 OpenAI 和 Huggingface 密钥。
import os
import openai
from dotenv import load_dotenv, find_dotenv
_ = load_dotenv(find_dotenv())
openai.api_key = os.environ['OPENAI_API_KEY']
openai.base_url = os.environ['OPENAI_BASE_URL']
os.environ["HUGGINGFACE_API_KEY"] = os.environ['HUGGINGFACE_API_KEY']
3. 导入 LangChain 和 TruLens
from IPython.display import JSON
# Imports main tools:
from trulens_eval import TruChain, Feedback, Huggingface, Tru
from trulens_eval.schema import FeedbackResult
tru = Tru()
tru.reset_database()
# Imports from langchain to build app. You may need to install langchain first
# with the following:
# ! pip install langchain>=0.0.170
from langchain.chains import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import ChatPromptTemplate, PromptTemplate
from langchain.prompts import HumanMessagePromptTemplate
4. 创建简单 LLM 应用程序
此示例使用 LangChain 框架和 OpenAI LLM。
full_prompt = HumanMessagePromptTemplate(
prompt=PromptTemplate(
template=
"Provide a helpful response with relevant background information for the following: {prompt}",
input_variables=["prompt"],
)
)
chat_prompt_template = ChatPromptTemplate.from_messages([full_prompt])
llm = OpenAI(temperature=0.9, max_tokens=128)
chain = LLMChain(llm=llm, prompt=chat_prompt_template, verbose=True)
5. 发送您的第一个请求
prompt_input = '现在几点了?'
llm_response = chain(prompt_input)
display(llm_response)
6. 初始化反馈函数
使用 HuggingFace 定义语言匹配反馈函数。
# Initialize Huggingface-based feedback function collection class:
hugs = Huggingface()
# Define a language match feedback function using HuggingFace.
f_lang_match = Feedback(hugs.language_match).on_input_output()
# By default this will check language match on the main app input and main app
# output.
7. 使用 TruLens 链式工具记录日志
tru_recorder = TruChain(chain,
app_id='Chain1_ChatApplication',
feedbacks=[f_lang_match])
with tru_recorder as recording:
llm_response = chain(prompt_input)
display(llm_response)
8. 检索记录和反馈
# The record of the ap invocation can be retrieved from the `recording`:
rec = recording.get() # use .get if only one record
# recs = recording.records # use .records if multiple
display(rec)
# The results of the feedback functions can be rertireved from the record. These
# are `Future` instances (see `concurrent.futures`). You can use `as_completed`
# to wait until they have finished evaluating.
from concurrent.futures import as_completed
for feedback_future in as_completed(rec.feedback_results):
feedback, feedback_result = feedback_future.result()
feedback: Feedback
feedbac_result: FeedbackResult
display(feedback.name, feedback_result.result)
9. 在仪表板中探索
tru.run_dashboard() # open a local streamlit app to explore
# tru.stop_dashboard() # stop if needed
10. 命令行启动仪表板
或者,您可以在同一文件夹中从命令行运行 trulens-eval 来启动仪表板。
trulens-eval
11. (Optional) 直接在 NoteBook 中查看结果
tru.get_records_and_feedback(app_ids=[])[0] # pass an empty list of app_ids to get all
完结!