AI engineeringCompleted

Yunxiao-driven Automated Regression Pipeline

One command runs API, web and mini-program cases, writes results back and files defects

Role: Design and implementation (solo)
Scale: 179 cases · three executors · automatic write-back
Period: Sep 2026 — Present

Node.jsPlaywrightWeChat devtools CLIYunxiao MCPshellJUnit XML

What I did

179 cases (92 API / 48 mini program / 39 web) executed automatically and written back case by case
Failed cases automatically create Yunxiao defects with case ID, environment, request/response evidence and report path
Environment pre-checks on all three executors fail fast instead of producing batches of false failures
Every run emits junit.xml + report.html + summary.md, archivable and comparable

Why

Manual execution costs three things: time, consistency, and the results themselves. Cases live in the test platform, a second copy lives in a spreadsheet, outcomes get retyped into defect tickets, and the next regression starts over.

How

read the Yunxiao test plan
   ↓
dispatch by case type
   ├─ API      → shell scripts (independently reproducible curl cases)
   ├─ Web      → Playwright (shared login state, selector-based)
   └─ Mini app → WeChat CLI (automator / debugger, element-level)
   ↓
write status back per case (PASS / FAILURE / POSTPONE)
   ↓
failures → create Yunxiao defects (owner from the case, linked to the case)
   ↓
emit junit.xml / report.html / summary.md

Two decisions

No coordinate clicking or image recognition. RPA-style execution was on the table, but its failure rate drifts with resolution and theme — it “runs” but cannot be maintained. Everything moved to CLI plus coded selectors, so a failure is a real failure.

No 100% coverage target. Cases that cannot be automated are explicitly written back as POSTPONE and listed in the report. Better a visible gap than a flattering pass rate.

Pre-checks

The first run produced a batch of failures that turned out to be the environment not being up. Each executor now pre-checks service reachability, database connectivity and build artifacts, failing fast with a reason instead of reporting environment problems as product defects.

Outcome

One command covers all three targets, results land in the test platform, and defects arrive with evidence. Regression went from “two days for one person” to “run it after the change”.