Generate Test Cases

Auto-generate test cases from documents or conversation logs

POST /v1/test-cases/generate

cURL

curl -X POST https://api.playgent.com/v1/test-cases/generate \
  -H "Authorization: Bearer your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "agent_abc123",
    "source_type": "documents",
    "documents": [
      {
        "content": "Our refund policy allows returns within 30 days...",
        "name": "refund_policy.md"
      }
    ],
    "generation_config": {
      "num_test_cases": 10,
      "difficulty_distribution": { "easy": 0.3, "medium": 0.5, "hard": 0.2 },
      "include_edge_cases": true,
      "focus_areas": ["policy_violations", "ambiguous_requests"]
    }
  }'

Response

{
  "generated_test_cases": [
    {
      "name": "Edge Case: Refund after 30 days",
      "turns": [
        {
          "input": { "text": "I bought this 45 days ago, can I return it?" },
          "expected_behavior": "Agent should explain the 30-day policy limit"
        }
      ],
      "difficulty": "hard",
      "rationale": "Tests boundary condition of refund policy"
    }
  ],
  "coverage_analysis": {
    "policy_topics_covered": ["returns", "refunds", "exchanges"],
    "gaps_identified": ["international shipping returns not covered"]
  }
}

Automatically generate test cases from documents, conversation logs, or failure analysis. This endpoint uses AI to create comprehensive test suites including edge cases.

agent_id · string · required

Agent to generate test cases for

source_type · string · required

Source type: documents, conversation_logs, or failure_analysis

documents · array

Documents to generate test cases from (when source_type is documents)

Show document object

content · string · required

Document content

name · string

Document name

conversation_logs · array

Historical conversations (when source_type is conversation_logs)

Show log object

input · string · required

User input

output · string · required

Agent output

feedback · string

User feedback: good, bad, or neutral

generation_config · object

Configuration for test generation

Show properties

num_test_cases · integer

Number of test cases to generate (default: 10)

difficulty_distribution · object

Distribution of difficulty levels

Show properties

easy · number

Proportion of easy tests

medium · number

Proportion of medium tests

hard · number

Proportion of hard tests

include_edge_cases · boolean

Include adversarial edge cases (default: true)

focus_areas · array

Specific areas to focus on (e.g., policy_violations, ambiguous_requests)

generated_test_cases · array · required

Array of generated test cases

Show properties

name · string

Test case name

turns · array

Test turns

difficulty · string

Difficulty level

rationale · string

Why this test was generated

coverage_analysis · object · required

Analysis of topic coverage

Show properties

policy_topics_covered · array

Topics covered by generated tests

gaps_identified · array

Topics not covered

cURL

curl -X POST https://api.playgent.com/v1/test-cases/generate \
  -H "Authorization: Bearer your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "agent_abc123",
    "source_type": "documents",
    "documents": [
      {
        "content": "Our refund policy allows returns within 30 days...",
        "name": "refund_policy.md"
      }
    ],
    "generation_config": {
      "num_test_cases": 10,
      "difficulty_distribution": { "easy": 0.3, "medium": 0.5, "hard": 0.2 },
      "include_edge_cases": true,
      "focus_areas": ["policy_violations", "ambiguous_requests"]
    }
  }'

Response

{
  "generated_test_cases": [
    {
      "name": "Edge Case: Refund after 30 days",
      "turns": [
        {
          "input": { "text": "I bought this 45 days ago, can I return it?" },
          "expected_behavior": "Agent should explain the 30-day policy limit"
        }
      ],
      "difficulty": "hard",
      "rationale": "Tests boundary condition of refund policy"
    }
  ],
  "coverage_analysis": {
    "policy_topics_covered": ["returns", "refunds", "exchanges"],
    "gaps_identified": ["international shipping returns not covered"]
  }
}