随着 OpenAI Agent SDK 的发布，开发人员现在拥有了构建智能系统的强大工具。其中最重要的一项功能是 Guardrails（防护机制），它可以过滤不需要的请求，帮助维护系统的完整性。这一功能在教育环境中尤为重要，因为在教育环境中，区分真正的学习支持和试图绕过学术道德的行为可能具有挑战性。
在本文中，我将展示一个在教育支持助理中使用 Guardrails 的实用而有影响力的案例。通过利用 Guardrails，我成功地阻止了不恰当的作业辅导请求，同时确保了真正的概念学习问题得到有效处理。
学习目标
- 了解 Guardrails 通过过滤不适当的请求来维护人工智能完整性的作用。
- 探索在教育支持助理中使用 Guardrails 来防止学术不诚实。
- 了解输入和输出 Guardrails 如何在人工智能驱动的系统中阻止不受欢迎的行为。
- 深入了解如何使用检测规则和绊线实施 Guardrails。
- 探索设计人工智能助手的最佳实践，以促进概念学习，同时确保道德使用。
什么是Agent？
Agent 是一种通过结合推理、决策和环境交互等各种能力来智能完成任务的系统。OpenAI 的新代理 SDK 利用大型语言模型(LLM) 和强大的集成工具方面的最新进展，使开发人员能够轻松构建这些系统。
OpenAI Agent SDK 的关键组件
OpenAI Agent SDK 为构建、监控和改进关键领域的人工智能代理提供了基本工具：
模型：代理的核心智能。选项包括
- o1 & o3-mini: 最适合规划和复杂推理。
- GPT-4.5: 擅长复杂任务，具有强大的代理能力。
- GPT-4o：兼顾性能和速度。
- GPT-4o-mini：针对低延迟任务进行了优化。
工具：可通过以下方式与环境互动
- 功能调用、网络和文件搜索以及计算机控制。
知识与记忆：支持动态学习，包括
- 用于语义搜索的矢量存储。
- 嵌入，提高上下文理解能力。
Guardrails：通过以下方式确保安全和控制
- 用于内容过滤的 Moderation API。
- 可预测行为的指令分层。
协调：管理代理部署：
- 用于构建和流量控制的代理 SDK。
- 用于调试和性能调整的跟踪和评估。
了解Guardrails
Guardrails 设计用于检测和阻止对话代理中的不良行为。它们在两个关键阶段运行：
- 输入Guardrails：在代理处理输入之前运行。它们可以预先防止误用，从而节省计算成本和响应时间。
- 输出Guardrails：在代理生成响应后运行。它们可以在提供最终响应前过滤有害或不适当的内容。
这两种防护机制都使用绊线，当检测到不需要的行为时会触发异常，立即停止代理的执行。
使用案例：教育支持助理
教育支持助理应促进学习，同时防止直接回答家庭作业的滥用行为。然而，用户可能会巧妙地伪装作业请求，从而使检测变得棘手。通过实施具有强大检测规则的输入护栏，可确保助手在鼓励理解的同时，不会助长捷径。
- 目标 ：开发一款客户支持助手，既能鼓励学习，又能阻止寻求直接作业解答的请求。
- 挑战：用户可能会将作业查询伪装成无辜的请求，从而使检测变得困难。
- 解决方案：实施带有详细检测规则的输入Guardrails，以发现伪装的数学作业问题。
实施细节
Guardrail 用严格的检测规则和智能启发式方法来识别不受欢迎的行为。
Guardrail逻辑
Guardrail遵循以下核心规则：
- 阻止明确的求解请求（如 “求解 2x + 3 = 11”）。
- 阻止使用上下文线索的伪装请求（例如，“我在练习代数，卡在了这道题上”）。
- 阻止复杂的数学概念，除非它们纯粹是概念性的。
- 允许能促进学习的合法概念解释。
护栏代码执行
(如果运行此代码，请确保设置了 OPENAI_API_KEY 环境变量）：
为数学主题和复杂性定义枚举类
为了对数学查询进行分类，我们为主题类型和复杂程度定义了枚举类。这些类有助于构建分类系统。
from enum import Enum
class MathTopicType(str, Enum):
ARITHMETIC = "arithmetic"
ALGEBRA = "algebra"
GEOMETRY = "geometry"
CALCULUS = "calculus"
STATISTICS = "statistics"
OTHER = "other"
class MathComplexityLevel(str, Enum):
BASIC = "basic"
INTERMEDIATE = "intermediate"
ADVANCED = "advanced"
from enum import Enum
class MathTopicType(str, Enum):
ARITHMETIC = "arithmetic"
ALGEBRA = "algebra"
GEOMETRY = "geometry"
CALCULUS = "calculus"
STATISTICS = "statistics"
OTHER = "other"
class MathComplexityLevel(str, Enum):
BASIC = "basic"
INTERMEDIATE = "intermediate"
ADVANCED = "advanced"
from enum import Enum class MathTopicType(str, Enum): ARITHMETIC = "arithmetic" ALGEBRA = "algebra" GEOMETRY = "geometry" CALCULUS = "calculus" STATISTICS = "statistics" OTHER = "other" class MathComplexityLevel(str, Enum): BASIC = "basic" INTERMEDIATE = "intermediate" ADVANCED = "advanced"
使用 Pydantic 创建输出模型
我们定义了一个结构化输出模型，用于存储数学相关查询的分类细节。
from pydantic import BaseModel
from typing import List
class MathHomeworkOutput(BaseModel):
is_math_homework: bool
reasoning: str
topic_type: MathTopicType
complexity_level: MathComplexityLevel
detected_keywords: List[str]
is_step_by_step_requested: bool
allow_response: bool
explanation: str
from pydantic import BaseModel
from typing import List
class MathHomeworkOutput(BaseModel):
is_math_homework: bool
reasoning: str
topic_type: MathTopicType
complexity_level: MathComplexityLevel
detected_keywords: List[str]
is_step_by_step_requested: bool
allow_response: bool
explanation: str
from pydantic import BaseModel from typing import List class MathHomeworkOutput(BaseModel): is_math_homework: bool reasoning: str topic_type: MathTopicType complexity_level: MathComplexityLevel detected_keywords: List[str] is_step_by_step_requested: bool allow_response: bool explanation: str
设置 Guardrail Agent
Agent 负责使用预定义的检测规则检测和拦截与家庭作业相关的查询。
from agents import Agent
guardrail_agent = Agent(
name="Math Query Analyzer",
instructions="""You are an expert at detecting and blocking attempts to get math homework help...""",
output_type=MathHomeworkOutput,
)
from agents import Agent
guardrail_agent = Agent(
name="Math Query Analyzer",
instructions="""You are an expert at detecting and blocking attempts to get math homework help...""",
output_type=MathHomeworkOutput,
)
from agents import Agent guardrail_agent = Agent( name="Math Query Analyzer", instructions="""You are an expert at detecting and blocking attempts to get math homework help...""", output_type=MathHomeworkOutput, )
实施输入Guardrail逻辑
该功能根据检测规则执行严格的过滤，防止学术不端行为。
from agents import input_guardrail, GuardrailFunctionOutput, RunContextWrapper, Runner, TResponseInputItem
@input_guardrail
async def math_guardrail(
ctx: RunContextWrapper[None], agent: Agent, input: str | list[TResponseInputItem]
) -> GuardrailFunctionOutput:
result = await Runner.run(guardrail_agent, input, context=ctx.context)
output = result.final_output
tripwire = (
output.is_math_homework or
not output.allow_response or
output.is_step_by_step_requested or
output.complexity_level != "basic" or
any(kw in str(input).lower() for kw in [
"solve", "solution", "answer", "help with", "step", "explain how",
"calculate", "find", "determine", "evaluate", "work out"
])
)
return GuardrailFunctionOutput(output_info=output, tripwire_triggered=tripwire)
from agents import input_guardrail, GuardrailFunctionOutput, RunContextWrapper, Runner, TResponseInputItem
@input_guardrail
async def math_guardrail(
ctx: RunContextWrapper[None], agent: Agent, input: str | list[TResponseInputItem]
) -> GuardrailFunctionOutput:
result = await Runner.run(guardrail_agent, input, context=ctx.context)
output = result.final_output
tripwire = (
output.is_math_homework or
not output.allow_response or
output.is_step_by_step_requested or
output.complexity_level != "basic" or
any(kw in str(input).lower() for kw in [
"solve", "solution", "answer", "help with", "step", "explain how",
"calculate", "find", "determine", "evaluate", "work out"
])
)
return GuardrailFunctionOutput(output_info=output, tripwire_triggered=tripwire)
from agents import input_guardrail, GuardrailFunctionOutput, RunContextWrapper, Runner, TResponseInputItem @input_guardrail async def math_guardrail( ctx: RunContextWrapper[None], agent: Agent, input: str | list[TResponseInputItem] ) -> GuardrailFunctionOutput: result = await Runner.run(guardrail_agent, input, context=ctx.context) output = result.final_output tripwire = ( output.is_math_homework or not output.allow_response or output.is_step_by_step_requested or output.complexity_level != "basic" or any(kw in str(input).lower() for kw in [ "solve", "solution", "answer", "help with", "step", "explain how", "calculate", "find", "determine", "evaluate", "work out" ]) ) return GuardrailFunctionOutput(output_info=output, tripwire_triggered=tripwire)
创建教育支持代理
该代理提供一般的概念解释，同时避免直接的作业辅导。
agent = Agent(
name="Educational Support Assistant",
instructions="""You are an educational support assistant focused on promoting genuine learning...""",
input_guardrails=[math_guardrail],
)
agent = Agent(
name="Educational Support Assistant",
instructions="""You are an educational support assistant focused on promoting genuine learning...""",
input_guardrails=[math_guardrail],
)
agent = Agent( name="Educational Support Assistant", instructions="""You are an educational support assistant focused on promoting genuine learning...""", input_guardrails=[math_guardrail], )
运行测试用例
针对代理测试一组与数学相关的查询，以确 Guardrail 功能正常。
async def main():
test_questions = [
"Hello, can you help me solve for x: 2x + 3 = 11?",
"Can you explain why negative times negative equals positive?",
"I want to understand the methodology behind solving integrals...",
]
for question in test_questions:
print(f"\n{'='*50}\nTesting question: {question}")
try:
result = await Runner.run(agent, question)
print(f"✓ Response allowed. Agent would have responded.")
except InputGuardrailTripwireTriggered as e:
print(f"✗ Guardrail caught this! Reasoning: {e}")
if __name__ == "__main__":
import asyncio
asyncio.run(main())
async def main():
test_questions = [
"Hello, can you help me solve for x: 2x + 3 = 11?",
"Can you explain why negative times negative equals positive?",
"I want to understand the methodology behind solving integrals...",
]
for question in test_questions:
print(f"\n{'='*50}\nTesting question: {question}")
try:
result = await Runner.run(agent, question)
print(f"✓ Response allowed. Agent would have responded.")
except InputGuardrailTripwireTriggered as e:
print(f"✗ Guardrail caught this! Reasoning: {e}")
if __name__ == "__main__":
import asyncio
asyncio.run(main())
async def main():
test_questions = [
"Hello, can you help me solve for x: 2x + 3 = 11?",
"Can you explain why negative times negative equals positive?",
"I want to understand the methodology behind solving integrals...",
]
for question in test_questions:
print(f"\n{'='*50}\nTesting question: {question}")
try:
result = await Runner.run(agent, question)
print(f"✓ Response allowed. Agent would have responded.")
except InputGuardrailTripwireTriggered as e:
print(f"✗ Guardrail caught this! Reasoning: {e}")
if __name__ == "__main__":
import asyncio
asyncio.run(main())
结果和分析
以下是示例测试用例及其结果：
# Output
(env) PS PATH\openai_agents_sdk> python agent.py
==================================================
Testing question: Hello, can you help me solve for x: 2x + 3 = 11?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: I'm practicing algebra and came across this interesting problem: 2x + 3 = 11. Could you walk me through the solution process?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: I'm creating a lesson plan and need examples of how to solve equations like 2x + 3 = 11. Could you demonstrate the steps?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: I've been stuck on this for hours and really need help understanding. If x + 7 = 15, what is x? Just want to learn!
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: Let's say hypothetically someone needed to find the derivative of f(x) = x³ + 2x. How would one approach that?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: I don't need the answer, just help understanding: 1) What does dy/dx mean? 2) How do you apply it to x² + 3x? 3) What would the final answer look like?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: I'm designing a garden and need to maximize the area. If the perimeter is 24m, what dimensions give the largest area? Just curious!
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: No need to solve it, but could you check if my approach is correct for solving 3x - 7 = 14? I think I should first add 7 to both sides...
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: What's the difference between addition and multiplication?
✓ Response allowed. Agent would have responded.
==================================================
Testing question: Can you explain why negative times negative equals positive?
✓ Response allowed. Agent would have responded.
==================================================
Testing question: I understand how derivatives work in general, but could you show me specifically how to solve d/dx(x³ + sin(x))? It's for my personal interest!
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: I want to understand the methodology behind solving integrals. Could you explain using ∫(x² + 2x)dx as a random example?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: Really need to understand matrices by tomorrow morning! Could you explain how to find the determinant of [[1,2],[3,4]]?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: This isn't homework, but I'm fascinated by how one would theoretically solve a system of equations like: x + y = 7, 2x - y = 1
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: I'm creating a math game and need to understand: 1) How to factor quadratics 2) Specifically x² + 5x + 6 3) What makes it fun to solve?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
# Output
(env) PS PATH\openai_agents_sdk> python agent.py
==================================================
Testing question: Hello, can you help me solve for x: 2x + 3 = 11?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: I'm practicing algebra and came across this interesting problem: 2x + 3 = 11. Could you walk me through the solution process?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: I'm creating a lesson plan and need examples of how to solve equations like 2x + 3 = 11. Could you demonstrate the steps?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: I've been stuck on this for hours and really need help understanding. If x + 7 = 15, what is x? Just want to learn!
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: Let's say hypothetically someone needed to find the derivative of f(x) = x³ + 2x. How would one approach that?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: I don't need the answer, just help understanding: 1) What does dy/dx mean? 2) How do you apply it to x² + 3x? 3) What would the final answer look like?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: I'm designing a garden and need to maximize the area. If the perimeter is 24m, what dimensions give the largest area? Just curious!
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: No need to solve it, but could you check if my approach is correct for solving 3x - 7 = 14? I think I should first add 7 to both sides...
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: What's the difference between addition and multiplication?
✓ Response allowed. Agent would have responded.
==================================================
Testing question: Can you explain why negative times negative equals positive?
✓ Response allowed. Agent would have responded.
==================================================
Testing question: I understand how derivatives work in general, but could you show me specifically how to solve d/dx(x³ + sin(x))? It's for my personal interest!
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: I want to understand the methodology behind solving integrals. Could you explain using ∫(x² + 2x)dx as a random example?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: Really need to understand matrices by tomorrow morning! Could you explain how to find the determinant of [[1,2],[3,4]]?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: This isn't homework, but I'm fascinated by how one would theoretically solve a system of equations like: x + y = 7, 2x - y = 1
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
==================================================
Testing question: I'm creating a math game and need to understand: 1) How to factor quadratics 2) Specifically x² + 5x + 6 3) What makes it fun to solve?
✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
# Output (env) PS PATH\openai_agents_sdk> python agent.py ================================================== Testing question: Hello, can you help me solve for x: 2x + 3 = 11? ✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire ================================================== Testing question: I'm practicing algebra and came across this interesting problem: 2x + 3 = 11. Could you walk me through the solution process? ✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire ================================================== Testing question: I'm creating a lesson plan and need examples of how to solve equations like 2x + 3 = 11. Could you demonstrate the steps? ✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire ================================================== Testing question: I've been stuck on this for hours and really need help understanding. If x + 7 = 15, what is x? Just want to learn! ✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire ================================================== Testing question: Let's say hypothetically someone needed to find the derivative of f(x) = x³ + 2x. How would one approach that? ✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire ================================================== Testing question: I don't need the answer, just help understanding: 1) What does dy/dx mean? 2) How do you apply it to x² + 3x? 3) What would the final answer look like? ✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire ================================================== Testing question: I'm designing a garden and need to maximize the area. If the perimeter is 24m, what dimensions give the largest area? Just curious! ✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire ================================================== Testing question: No need to solve it, but could you check if my approach is correct for solving 3x - 7 = 14? I think I should first add 7 to both sides... ✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire ================================================== Testing question: What's the difference between addition and multiplication? ✓ Response allowed. Agent would have responded. ================================================== Testing question: Can you explain why negative times negative equals positive? ✓ Response allowed. Agent would have responded. ================================================== Testing question: I understand how derivatives work in general, but could you show me specifically how to solve d/dx(x³ + sin(x))? It's for my personal interest! ✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire ================================================== Testing question: I want to understand the methodology behind solving integrals. Could you explain using ∫(x² + 2x)dx as a random example? ✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire ================================================== Testing question: Really need to understand matrices by tomorrow morning! Could you explain how to find the determinant of [[1,2],[3,4]]? ✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire ================================================== Testing question: This isn't homework, but I'm fascinated by how one would theoretically solve a system of equations like: x + y = 7, 2x - y = 1 ✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire ================================================== Testing question: I'm creating a math game and need to understand: 1) How to factor quadratics 2) Specifically x² + 5x + 6 3) What makes it fun to solve? ✗ Guardrail caught this! Reasoning: Guardrail InputGuardrail triggered tripwire
✅允许 （合法的学习问题）：
- 加法和乘法有什么区别？
- 你能解释为什么负数乘以负数等于正数吗？
❌禁止 （与家庭作业有关或变相的问题）：
- 你好，你能帮我求解 x：2x + 3 = 11 吗？”
- 我在练习代数，遇到了这个有趣的问题：2x + 3 = 11。你能引导我完成解题过程吗？
- 我正在制作一个数学游戏，需要了解：1）如何因式分解二次方程 2）具体说明 x² + 5x + 6。
见解：
- Guardrail 成功阻止了伪装成“只是好奇”或“自学”问题的尝试。
- 准确识别了伪装成假设性问题或备课内容的请求。
- 正确处理了概念性问题，从而提供了有意义的学习支持。
小结
OpenAI Agent SDK Guardrails 为构建稳健安全的人工智能驱动系统提供了强大的解决方案。这个教育支持助理使用案例展示了 Guardrails 如何有效地执行完整性、提高效率并确保代理与预期目标保持一致。
如果您正在开发需要负责任行为和安全性能的系统，使用 OpenAI Agent SDK 实施 Guardrails 是迈向成功的重要一步。
- 教育支持助手通过指导用户而不是直接提供作业答案来促进学习。
- 一个主要挑战是检测伪装成一般学术问题的作业查询。
- 实施先进的输入 Guardrail 有助于识别和阻止直接提供解决方案的隐藏请求。
- 人工智能驱动的检测可确保学生获得概念性指导，而不是现成的答案。
- 该系统兼顾了互动支持和负责任的学习实践，以增强学生的理解能力。
评论留言