By using the MagicPod MCP Server, you can have an AI agent read test specifications, such as Excel files, and create MagicPod test cases based on their contents.
This article introduces recommended formats and writing practices for test specifications to help improve the accuracy of AI-generated test cases.
Table of Contents
- Recommended Test Specification Format
- Tips for Generating More Accurate Test Cases
- How to Use Existing Shared Steps
- Reviewing Generated Test Cases
1. Recommended Test Specification Format
You can use your existing test specifications, such as Excel files, without modifying their format. However, we recommend including the following five types of information to help the AI accurately understand the purpose and details of each test.
| Item | Description |
|---|---|
| Test Case Name | What the test is intended to verify |
| Preconditions | Required conditions or test data before execution |
| Test Steps | Actions to perform in the application under test |
| Expected Results | What should be verified after performing the actions |
| Automation Status | Whether the test is intended to be automated |
Clearly documenting this information helps the AI understand the test requirements and increases the likelihood of generating test cases as intended.
You can download a sample test specification containing these items below. Please use it as a reference when preparing your own test specifications.
Sample Test Specification.xlsx
2. Tips for Generating More Accurate Test Cases
① Avoid Formatting and Writing Practices That May Confuse the AI
To help the AI correctly interpret your test specifications, we recommend avoiding the following practices whenever possible.
| What to Avoid | Recommended Approach |
|---|---|
| Including notes unrelated to test automation | Remove unnecessary notes and information about past releases |
| Placing important information in Excel comments | Enter important instructions directly into cells |
| Using merged cells | Avoid merging cells whenever possible to make the relationships between items clear |
| Using colors alone to indicate whether a test should be automated | Explicitly label tests as "Not for Automation" or similar, rather than relying on colors or grayed-out cells |
| Using column headers that do not match their contents | Ensure that column headers accurately describe the information in each column |
② Provide Specific Test Steps When Necessary
The AI can determine how to perform actions based on information displayed on the screen, so you do not necessarily need to describe every step in detail. However, actions or preparations that cannot be determined from the screen require specific instructions.
For example, if the exact navigation method does not matter, you can simply write "Return to My Page." However, if a specific sequence of actions is required, provide explicit instructions such as "Open the hamburger menu and select 'My Page.'"
③ Specify How Test Cases Should Be Grouped
Creating a separate test case for every row in a test specification can result in too many small test cases, increasing maintenance effort and execution time.
We recommend combining consecutive actions into a single test case when they share the same preconditions. On the other hand, consider separating test cases when their preconditions differ or when they include actions that may affect subsequent tests, such as account deletion or account locking.
It is helpful to give the AI specific instructions, such as "Create one test case for each subcategory" or "Group items that have the same number before the hyphen in their test IDs into a single test case."
You can also ask the AI to propose how the test cases should be grouped before creating them.
④ Avoid Ambiguous Expressions
Ambiguous expressions such as "any value," "an appropriate value," or "works correctly" may prevent the AI from accurately understanding your intentions.
We recommend specifying input values and expected results as clearly as possible.
⑤ Ask the AI to Clarify Unclear Requirements
If the test specification contains unclear information, the AI may make assumptions when generating test cases.
To avoid this, consider instructing the AI to ask questions before creating test cases.
Example Prompt:
Before creating test cases, review the test steps, expected results, and preconditions in the test specification. If anything is unclear or ambiguous, ask questions instead of making assumptions.
⑥ Use Common Rules
For verification requirements that apply to multiple test cases, it can be helpful to define common rules and instruct the AI to apply them, rather than repeating the same information in every row of the specification.
For example, you can define a rule such as "After navigating to a different screen, verify that the destination screen title is correct." This reduces the amount of information needed in the specification while helping prevent missing expected-result checks.
When using Excel, we recommend organizing common rules in a separate worksheet. This makes the specification easier to navigate and helps the AI identify the common rules.
⑦ Create an Index Worksheet
If your Excel file contains multiple worksheets, consider creating an index worksheet that summarizes the contents of each worksheet and identifies which tests should be automated.
This helps the AI understand the overall structure of the test specification.
⑧ Review the Specification or AI Model If the Results Are Not as Expected
The accuracy of generated test cases depends on how specific the test specification is and the capabilities of the AI model being used.
If the generated test cases do not meet your expectations, providing more detailed information in the specification or switching to a more capable AI model may help improve the results.
3. How to Use Existing Shared Steps
The MagicPod MCP Server also allows you to create test cases using existing shared steps.
However, if you want the AI to use existing shared steps, you need to explicitly instruct it to do so.
For example, we recommend including instructions such as the following in your test specification, common rules, or prompts to the AI agent:
- Use the existing shared step "Login Process" for login operations.
- Use the existing shared step "Initialize Test Data" to reset test data.
- Prioritize existing shared steps over creating new steps when the required actions can be performed using those shared steps.
By providing these instructions, you can reuse existing shared steps and manage common operations across test cases more efficiently.
4. Reviewing Generated Test Cases
We recommend reviewing AI-generated test cases based on the following criteria:
- Test Steps: Do the test steps and screen transitions match the intended test flow?
- Expected Results: Are appropriate assertions included to verify the expected results?
- Preconditions: Are the required test data and preparations properly configured?
- Test Case Structure: Are the test cases appropriately grouped, with related steps organized together?
- Shared Steps: Are the specified existing shared steps used correctly?
We also recommend scheduling the test cases to run daily for approximately one week to check whether they consistently pass and to identify potentially flaky tests.