SmartTestGen 🧪

Auto-generate failing test cases from sprint requirements. Multi-agent. Multi-framework. Multi-language. Left-shift from day one.

What Is SmartTestGen?

SmartTestGen is an open source AI Testing Army — a multi-agent orchestration layer that takes your sprint requirements from Jira, GitHub, or Notion and automatically generates failing test cases before a single line of production code is written.

Tests fail on day one because the app doesn't exist yet. That's intentional. As developers implement each feature, tests go green one by one. 🔴→🟢

Key Differentiators

1. Accepts multiple requirement formats in one run GWT + Figma + OpenAPI + NFR + Security + Integration requirements in a single pipeline.

2. Intentionally left-shift Existing tools (Dredd, swagger_meqa, Autify) generate tests against a running system. SmartTestGen generates tests before the system exists, from specs alone.

3. Framework-agnostic via a plugin architecture The same requirements produce TypeScript (Playwright) or Python (pytest) output with a single config change.

4. Multi-agent parallel dispatch Six specialist agents run concurrently via Promise.all(). UI, API, Performance, Security, Mobile, and Embedded agents each own their domain.

5. Covers IoT + Cloud + Mobile + Embedded scope MQTT integration testing, OWASP ISVS device security, Appium mobile tests, cloud telemetry SLA testing — all in one framework.

6. Integrated CI + reporting GitHub Actions workflows, Slack digests, Allure HTML reports, GitHub PR coverage comments, and a coverage gate that blocks merges if coverage drops.

Agents

Agent	Trigger	Generates	Framework
`AgentUI`	GWT + Figma/website	Page Object Model + UI specs	Playwright TS / playwright-python
`AgentAPI`	OpenAPI spec URL	Endpoint tests for every route	pytest + httpx / Playwright TS
`AgentPerformance`	NFR requirements	SLA tests + k6 load scripts	locust / k6
`AgentSecurity`	OWASP/GDPR requirements	Security test stubs	pytest (OWASP scope)
`AgentMobile`	GWT + mobile context	Appium v2 Python or XCUITest Swift	Appium / XCUITest
`AgentEmbedded`	Integration requirements	MQTT / BLE / REST IoT tests	pytest + paho-mqtt / bleak

All agents run in parallel via MasterTestAgent.runArmy().

Intake Sources

Tests are driven by tickets, not manual input. The intake layer fetches stories from:

Source	MCP (preferred)	REST Fallback
Jira	`jira_search_issues` (JQL)	Jira Cloud REST API v3
GitHub	`list_issues`	GitHub REST API
Notion	`notion_query_database`	—

Stories are normalised to GWTContext (Given/When/Then) via Claude (claude-haiku-4-5) or keyword extraction fallback.

Note: The CLI manages its own MCP connections via mcp.json — independent of any Claude.ai MCP settings.

Skills (Parsers)

Skill	Input	Output
`GWTParserSkill`	GWT text / JSON	`GWTScenario[]`
`FigmaParserSkill`	Figma API	`FigmaScreen[]` (selectors)
`SwaggerParserSkill`	OpenAPI URL / file	`APIContext` (endpoints)
`NFRParserSkill`	NFR JSON	`NFRRequirement[]`
`SecurityRequirementParserSkill`	OWASP JSON	`SecurityRequirement[]`
`IntegrationRequirementParserSkill`	Integration JSON	`IntegrationRequirement[]`
`PageCrawlerSkill`	Website URL	`FigmaScreen[]` (from DOM)
`UserStoryNormalizer`	Raw ticket prose	`GWTContext` (via Claude API)

Supported Plugins

Plugin	Language	UI	API	Perf	Security	Integration
`playwright-ts`	TypeScript	`.spec.ts`	`.spec.ts`	k6 script	`.spec.ts`	`.spec.ts`
`pytest`	Python	pytest + playwright-python	pytest + httpx	locustfile.py	pytest + httpx	pytest + httpx
`jest` (coming soon)	JavaScript	—	—	—	—	—
`cypress` (coming soon)	JavaScript	—	—	—	—	—

Architecture

┌──────────────────────────────────────────────────────────────┐
│                        INTAKE LAYER                          │
│                                                              │
│  IntakeRouter                                                │
│  ├── JiraMCPIntakeSkill  / JiraRESTIntakeSkill               │
│  ├── GitHubMCPIntakeSkill / GitHubRESTIntakeSkill            │
│  └── NotionMCPIntakeSkill                                    │
│                         │                                    │
│                  UserStoryNormalizer (Claude API)            │
│                         │                                    │
│                    GWTContext[]                              │
└──────────────────────────┬───────────────────────────────────┘
                           │
                           ▼
┌──────────────────────────────────────────────────────────────┐
│                  MASTER TEST AGENT                           │
│           MasterTestAgent.runArmy() → Promise.all()          │
│                                                              │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐       │
│  │ AgentUI  │ │ AgentAPI │ │AgentPerf │ │AgentSec  │       │
│  └──────────┘ └──────────┘ └──────────┘ └──────────┘       │
│  ┌──────────────────┐   ┌──────────────────────────┐        │
│  │  AgentMobile     │   │     AgentEmbedded         │        │
│  └──────────────────┘   └──────────────────────────┘        │
│                                                              │
│  Each agent uses Skills (parsers) to interpret input         │
│  and delegates generation to the configured Plugin           │
└──────────────────────────┬───────────────────────────────────┘
                           │
           ┌───────────────┴──────────────┐
           ▼                              ▼
┌──────────────────┐          ┌──────────────────────┐
│  PLUGIN          │          │  PLUGIN               │
│  playwright-ts   │          │  pytest               │
│  .spec.ts files  │          │  test_*.py files      │
│  k6 load script  │          │  locustfile.py        │
└──────────────────┘          └──────────────────────┘
           │                              │
           └──────────────┬───────────────┘
                          ▼
┌──────────────────────────────────────────────────────────────┐
│                    REPORTING LAYER                           │
│                                                              │
│  SlackReporter        → nightly digest + flakiness alerts   │
│  AllurePublisher      → HTML report to GitHub Pages         │
│  GitHubPRCommenter    → coverage table on every PR          │
│  CoverageGate         → blocks merge if < 80% coverage      │
└──────────────────────────────────────────────────────────────┘

CI Workflows

Workflow	Trigger	What It Does
`on_pr.yml`	PR open / update	Intake → Army run → Coverage gate → PR comment
`nightly.yml`	2 AM daily (3× matrix)	Full army + flakiness detection + Allure to Pages
`on_jira_webhook.yml`	Jira issue created	Intake single ticket → Army run → commit to feature branch

Project Structure

smart-test-gen/
│
├── core/                              ← Never changes
│   ├── types.ts                       ← All shared TypeScript interfaces
│   ├── ITestGenerator.ts              ← Plugin contract (interface)
│   └── PluginRegistry.ts             ← Maps plugin names to implementations
│
├── parsers/                           ← Language-agnostic skills
│   ├── GWTParserSkill.ts
│   ├── FigmaParserSkill.ts
│   ├── SwaggerParserSkill.ts
│   ├── NFRParserSkill.ts
│   ├── SecurityRequirementParserSkill.ts
│   ├── IntegrationRequirementParserSkill.ts
│   └── PageCrawlerSkill.ts
│
├── plugins/                           ← One folder per framework
│   ├── playwright-ts/
│   │   └── PlaywrightTSGenerator.ts
│   └── pytest/
│       └── PytestGenerator.ts
│
├── agents/                            ← Army agents
│   ├── MasterTestAgent.ts            ← Orchestrator (runArmy + run)
│   ├── AgentUI.ts                    ← Playwright / playwright-python
│   ├── AgentAPI.ts                   ← API endpoint tests
│   ├── AgentPerformance.ts           ← NFR + k6 / locust
│   ├── AgentSecurity.ts              ← OWASP / GDPR
│   ├── AgentMobile.ts                ← Appium v2 / XCUITest Swift
│   └── AgentEmbedded.ts             ← MQTT / BLE / IoT REST
│
├── intake/                            ← Ticket ingestion
│   ├── MCPClientManager.ts           ← MCP connection manager
│   ├── IntakeRouter.ts               ← Source selector + GWTContext builder
│   ├── UserStoryNormalizer.ts        ← Claude API → GWT extraction
│   ├── JiraMCPIntakeSkill.ts
│   ├── JiraRESTIntakeSkill.ts
│   ├── GitHubMCPIntakeSkill.ts
│   ├── GitHubRESTIntakeSkill.ts
│   └── NotionMCPIntakeSkill.ts
│
├── reporting/                         ← Reporting layer
│   ├── SlackReporter.ts              ← Slack webhook (Block Kit)
│   ├── AllurePublisher.ts            ← Allure HTML report generator
│   ├── GitHubPRCommenter.ts          ← PR comment + commit status
│   └── CoverageGate.ts              ← Coverage gate + flakiness detection
│
├── .github/workflows/
│   ├── on_pr.yml                     ← PR coverage gate
│   ├── nightly.yml                   ← Nightly army + Allure
│   └── on_jira_webhook.yml           ← Jira webhook trigger
│
├── cli/
│   └── index.ts                      ← init | generate | army | intake | report
│
├── config/
│   └── smart-test-gen.config.json
│
├── mcp.json                           ← CLI MCP server registry
├── index.ts                           ← Entry point
├── package.json
├── tsconfig.json
└── .env.example

Quick Start

# 1. Clone and install
git clone https://github.com/stilaye/SmartTestGen
cd SmartTestGen
npm install

# 2. Copy env and fill in your tokens
cp .env.example .env

# 3. Initialize
npx ts-node cli/index.ts init

# 4. Generate tests from sprint requirements
npx ts-node cli/index.ts generate

# 5. Run the full AI Testing Army (fetches from Jira/GitHub/Notion)
npx ts-node cli/index.ts army run

# 6. Intake tickets from a specific source
npx ts-node cli/index.ts intake --source jira
npx ts-node cli/index.ts intake --source github
npx ts-node cli/index.ts intake --source notion

# 7. Post a report
npx ts-node cli/index.ts report --to slack
npx ts-node cli/index.ts report --to allure
npx ts-node cli/index.ts report --to github

Sprint Workflow

Monday
  PO          → Open Jira tickets / GitHub issues
  QA          → npx ts-node cli/index.ts army run
                Intake fetches tickets → 6 agents run in parallel
                All tests generated, all RED 🔴

Tue–Thu       → Developers implement feature
                Tests go GREEN 🟢 as each layer ships

Friday        → CI runs nightly.yml
                Coverage gate checks 80% threshold
                Slack digest posted to #qa-army
                Allure report published to GitHub Pages

PR opened     → on_pr.yml runs army (UI + API + Security)
                Coverage comment posted to PR
                Merge blocked if coverage < gate

Sprint Demo   → Show green tests as evidence of quality ✅

CLI Commands

npx ts-node cli/index.ts init                   Initialize config
npx ts-node cli/index.ts generate               Generate from local index.ts input
npx ts-node cli/index.ts army run               Run all 6 agents (parallel)
npx ts-node cli/index.ts army run --agent ui    Run one agent only
npx ts-node cli/index.ts intake --source jira   Fetch Jira sprint tickets
npx ts-node cli/index.ts intake --source github Fetch GitHub issues
npx ts-node cli/index.ts intake --source notion Fetch Notion database pages
npx ts-node cli/index.ts report --to slack      Post Slack summary
npx ts-node cli/index.ts report --to allure     Generate Allure HTML report
npx ts-node cli/index.ts report --to github     Post GitHub PR comment

Security Standards Supported

Standard	Scope	Coverage
OWASP ISVS	IoT device + ecosystem security	Device auth, firmware, communication
OWASP ASVS	Web + API security	Injection, auth, session, data
OWASP MASVS	Mobile app security	Mobile-specific controls
GDPR	Data privacy (EU)	PII handling, data retention
HIPAA	Healthcare data (US)	PHI protection, audit logs
PCI-DSS	Payment card data	Card data handling, encryption

Plugin Architecture (Strategy Pattern)

// Adding a new framework = implement this interface

interface ITestGenerator {
  readonly language: string;
  readonly framework: string;
  readonly displayName: string;

  generateUITests(gwt, figma): GeneratedTest[];
  generateAPITests(gwt, api): GeneratedTest[];
  generatePerformanceTests(nfr): GeneratedTest[];
  generateSecurityTests(sec): GeneratedTest[];
  generateIntegrationTests(int): GeneratedTest[];
  getSetupInstructions(): string;
  getFrameworkConfig(): string;
}

To add Robot Framework: create plugins/robot-framework/RobotFrameworkGenerator.ts, implement the interface, register in PluginRegistry.ts. Zero changes elsewhere.

Contributing — Adding a New Plugin

Create plugins/your-framework/YourFrameworkGenerator.ts
Implement ITestGenerator
Register in core/PluginRegistry.ts
Add to the plugin list in cli/index.ts

Comparison With Existing Tools

Feature	SmartTestGen	swagger_meqa	Dredd	Autify Genesis	Playwright Agents
GWT input	✅	❌	❌	✅ (chat only)	❌
Figma input	✅	❌	❌	✅ (chat only)	❌
OpenAPI input	✅	✅	✅	❌	❌
NFR / SLA input	✅	❌	❌	❌	❌
Security reqs input	✅	❌	❌	❌	❌
Integration reqs	✅	❌	❌	❌	❌
Multi-agent parallel	✅	❌	❌	❌	❌
Multi-framework output	✅	❌	❌	❌	❌
Left-shift (pre-implementation)	✅	❌	❌	✅	❌
Jira / GitHub / Notion intake	✅	❌	❌	❌	❌
IoT / MQTT / BLE scope	✅	❌	❌	❌	❌
Mobile (Appium / XCUITest)	✅	❌	❌	❌	❌
CI workflows + coverage gate	✅	❌	❌	✅ (SaaS)	❌
Slack + Allure + PR comments	✅	❌	❌	✅ (SaaS)	❌
Open source + owned code	✅	✅	✅	❌ (SaaS)	✅

Author

Built by Swapnil — Senior QA Automation Engineer IoT + Cloud + Mobile App left-shift testing framework.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

SmartTestGen 🧪

What Is SmartTestGen?

Key Differentiators

Agents

Intake Sources

Skills (Parsers)

Supported Plugins

Architecture

CI Workflows

Project Structure

Quick Start

Sprint Workflow

CLI Commands

Security Standards Supported

Plugin Architecture (Strategy Pattern)

Contributing — Adding a New Plugin

Comparison With Existing Tools

Author

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
.github/workflows		.github/workflows
agents		agents
cli		cli
config		config
core		core
docs		docs
intake		intake
parsers		parsers
plugins		plugins
reporting		reporting
tests/unit		tests/unit
.env.example		.env.example
.gitignore		.gitignore
README.md		README.md
eslint.config.mjs		eslint.config.mjs
index.ts		index.ts
mcp.json		mcp.json
package-lock.json		package-lock.json
package.json		package.json
playwright.config.ts		playwright.config.ts
tsconfig.json		tsconfig.json

Folders and files

Latest commit

History

Repository files navigation

SmartTestGen 🧪

What Is SmartTestGen?

Key Differentiators

Agents

Intake Sources

Skills (Parsers)

Supported Plugins

Architecture

CI Workflows

Project Structure

Quick Start

Sprint Workflow

CLI Commands

Security Standards Supported

Plugin Architecture (Strategy Pattern)

Contributing — Adding a New Plugin

Comparison With Existing Tools

Author

About

Resources

Contributing

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages