ragas官方文档中文版（六十七）

操作指南

本节中的每个指南都针对您作为有经验的用户在使用 Ragas 时可能遇到的实际问题提供了专注的解决方案。这些指南设计得简洁直接，为您的问题提供快速解决方案。我们假设您对 Ragas 的概念有基本了解且能够熟练使用。如果不是，请先浏览 快速入门 （Get Started）部分。

使用 Llama 4 评估 LlamaStack Web Search 接地性

在本教程中，我们将测量 LlamaStack 网页搜索智能体生成响应的接地性。LlamaStack 是由 Meta 维护的开源框架，用于简化大语言模型驱动应用的开发和部署。评估将使用 Ragas 指标和 Meta Llama 4 Maverick 作为评判模型进行。

设置并运行 LlamaStack 服务器

此命令安装使用 Together 推理提供程序的 LlamaStack 服务器所需的所有依赖项。

使用 conda 命令

!pipinstallragas langchain-together uv!uv run--withllama-stack llama stack build--templatetogether --image-type conda

使用 venv 命令

!pipinstallragas langchain-together uv!uv run--withllama-stack llama stack build--templatetogether --image-type venv

importosimportsubprocessdefrun_llama_stack_server_background():log_file=open("llama_stack_server.log","w")process=subprocess.Popen("uv run --with llama-stack llama stack run together --image-type venv",shell=True,stdout=log_file,stderr=log_file,text=True,)print(f"Starting LlamaStack server with PID:{process.pid}")returnprocessdefwait_for_server_to_start():importrequestsfromrequests.exceptionsimportConnectionErrorimporttime url="http://0.0.0.0:8321/v1/health"max_retries=30retry_interval=1print("Waiting for server to start",end="")for_inrange(max_retries):try:response=requests.get(url)ifresponse.status_code==200:print("\nServer is ready!")returnTrueexceptConnectionError:print(".",end="",flush=True)time.sleep(retry_interval)print("\nServer failed to start after",max_retries*retry_interval,"seconds")returnFalse# use this helper if needed to kill the serverdefkill_llama_stack_server():# Kill any existing llama stack server processesos.system("ps aux | grep -v grep | grep llama_stack.distribution.server.server | awk '{print $2}' | xargs kill -9")

启动 LlamaStack 服务器

server_process=run_llama_stack_server_background()assertwait_for_server_to_start()

Starting LlamaStack serverwithPID:95508Waitingforserver to start....Serverisready!

构建搜索智能体

fromllama_stack_clientimportLlamaStackClient,Agent,AgentEventLogger client=LlamaStackClient(base_url="http://0.0.0.0:8321",)agent=Agent(client,model="meta-llama/Llama-3.1-8B-Instruct",instructions="You are a helpful assistant. Use web search tool to answer the questions.",tools=["builtin::websearch"],)user_prompts=["In which major did Demis Hassabis complete his undergraduate degree? Search the web for the answer.","Ilya Sutskever is one of the key figures in AI. From which institution did he earn his PhD in machine learning? Search the web for the answer.","Sam Altman, widely known for his role at OpenAI, was born in which American city? Search the web for the answer.",]session_id=agent.create_session("test-session")forpromptinuser_prompts:response=agent.create_turn(messages=[{"role":"user","content":prompt,}],session_id=session_id,)forloginAgentEventLogger().log(response):log.print()

现在，让我们深入了解智能体的执行步骤，看看我们的智能体表现如何。

session_response=client.agents.session.retrieve(session_id=session_id,agent_id=agent.agent_id,)

评估智能体响应

我们希望测量 LlamaStack 网页搜索智能体生成响应的接地性。为此，我们需要 EvaluationDataset 和指标来评估接地响应。Ragas 提供了一系列现成的指标，可用于测量检索和生成的各个方面。

用于测量响应接地性的指标：

1. 忠实度（Faithfulness）
2. 响应接地性（Response Groundedness）

构建 Ragas 评估数据集

要使用 Ragas 进行评估，我们将创建一个 EvaluationDataset 。

importjson# This function extracts the search results for the trace of each querydefextract_retrieved_contexts(turn_object):results=[]forstepinturn_object.steps:ifstep.step_type=="tool_execution":tool_responses=step.tool_responsesforresponseintool_responses:content=response.contentifcontent:try:parsed_result=json.loads(content)results.append(parsed_result)exceptjson.JSONDecodeError:print("Warning: Unable to parse tool response content as JSON.")continueretrieved_context=[]forresultinresults:top_content_list=[item["content"]foriteminresult["top_k"]]retrieved_context.extend(top_content_list)returnretrieved_context

fromragas.dataset_schemaimportEvaluationDataset samples=[]references=["Demis Hassabis completed his undergraduate degree in Computer Science.","Ilya Sutskever earned his PhD from the University of Toronto.","Sam Altman was born in Chicago, Illinois.",]fori,turninenumerate(session_response.turns):samples.append({"user_input":turn.input_messages[0].content,"response":turn.output_message.content,"reference":references[i],"retrieved_contexts":extract_retrieved_contexts(turn),})ragas_eval_dataset=EvaluationDataset.from_list(samples)

ragas_eval_dataset.to_pandas()

设置 Ragas 指标

fromragas.metricsimportAnswerAccuracy,Faithfulness,ResponseGroundednessfromlangchain_togetherimportChatTogetherfromragas.llmsimportLangchainLLMWrapper llm=ChatTogether(model="meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8",)evaluator_llm=LangchainLLMWrapper(llm)ragas_metrics=[AnswerAccuracy(llm=evaluator_llm),Faithfulness(llm=evaluator_llm),ResponseGroundedness(llm=evaluator_llm),]

评估

最后，让我们运行评估。

fromragasimportevaluate results=evaluate(dataset=ragas_eval_dataset,metrics=ragas_metrics)results.to_pandas()

fromragasimportevaluate results=evaluate(dataset=ragas_eval_dataset,metrics=ragas_metrics)results.to_pandas()

kill_llama_stack_server()