Available for AI QA Engineer roles

Steven Leon

AI-Focused QA Engineer & Builder

I test, evaluate, and ship AI-powered products — from LLM regression suites to solo projects.

View ProjectsDownload ResumeGet in touch →
0+ years
QA at Vox Media (The Verge)
0+
LLM regression tests shipped
0
Production tools built solo
0
Passing tests in one suite alone
01 — Selected work

What I've built

DropLinx landing page: paste a URL and get a week of content across seven formats

DropLinx

Project

Drop a link. Get a week of content.

A multi-channel AI content engine that turns any URL — or a plain prompt — into a finished week of content: blog post, newsletter, post pack, reel script, YouTube Short, product ad, and slide deck. Two modes (Social Content and Marketing Ads) reuse the same source once and fan it out across every channel, each output shipped finished rather than drafted.

7 output formats • 2 input modes • seconds per run
TypeScriptReactClaude APISupabaseLovable Cloud
Live Demo

Workflow diagram

1Paste URL (auto-detected on paste) or Prompt an Idea → source normalized and summarized
2Mode switch routes the run through Social Content or Marketing Ads prompt sets
3Fan-out to parallel generators for blog, newsletter, post pack, reel script, YouTube Short, product ad, slide deck
4Outputs persisted to Supabase and returned as one browsable content pack

Integrations diagram

DropLinxAnthropic Claude APISupabase (Postgres + A…Lovable CloudURL scraping / metadat…

Workflows

  • Paste URL (auto-detected on paste) or Prompt an Idea → source normalized and summarized
  • Mode switch routes the run through Social Content or Marketing Ads prompt sets
  • Fan-out to parallel generators for blog, newsletter, post pack, reel script, YouTube Short, product ad, slide deck
  • Outputs persisted to Supabase and returned as one browsable content pack

Integrations

  • Anthropic Claude API
  • Supabase (Postgres + Auth)
  • Lovable Cloud
  • URL scraping / metadata extraction

Tools

  • TypeScript
  • React
  • Tailwind
  • Vite
  • Supabase
  • GitHub

Core functionality

  • One input in, seven finished formats out — no per-channel rewriting
  • Dual input modes: paste a live URL or describe an idea from scratch
  • Social Content vs Marketing Ads presets with per-format prompt templates and brand voice tuning
  • Auto paste detection so a copied link kicks off a run with no extra click
  • Auth-gated dashboard with saved history and re-generation of individual formats

Testing

  • Smoke suite across all 7 generators and both input modes before each release
  • Prompt regression checks on tone, length thresholds, and format structure
  • Negative-path coverage: unreachable URLs, paywalled pages, empty scrapes, provider rate limits
  • Cross-browser and responsive checks on the paste flow and output viewer
AI Assistant Eval promptfoo benchmark: 5 test categories and a 94% weighted pass rate

AI Assistant Eval

Project

Systematic evaluation of AI assistant responses for accuracy, tone, and helpfulness.

A Promptfoo-based evaluation suite that benchmarks AI assistant responses against rubric-driven criteria. Uses model-graded assertions, deterministic checks, and custom scoring scripts to measure response quality, factual correctness, and user-helpfulness across multiple test scenarios.

Multi-provider evals • rubric scoring • CI-ready
promptfooOpenAIAnthropicJavaScriptYAMLVS Code
GitHub

Workflow diagram

1Define test cases in YAML with prompt templates, inputs, and expected behaviors
2Run model-graded and deterministic assertions against assistant responses
3Aggregate scores and export results to a human-readable dashboard

Integrations diagram

AI Assistant EvalOpenAI APIAnthropic APIpromptfoo

Workflows

  • Define test cases in YAML with prompt templates, inputs, and expected behaviors
  • Run model-graded and deterministic assertions against assistant responses
  • Aggregate scores and export results to a human-readable dashboard

Integrations

  • OpenAI API
  • Anthropic API
  • promptfoo

Tools

  • JavaScript
  • YAML
  • VS Code
  • Node.js
  • GitHub

Core functionality

  • Rubric-based evaluation of assistant accuracy and helpfulness
  • Side-by-side response comparison across providers and prompts
  • Configurable pass/fail thresholds and regression detection

Testing

  • Model-graded assertions for tone and helpfulness scoring
  • Deterministic assertions for factual correctness and format validation
  • Reusable test suite structure for continuous eval runs

QA Practice Projects

Project

Three intentionally broken bugs, built to teach by breaking.

Self-contained Playwright projects reproducing real production bug patterns: a UTM parameter lost mid-redirect, a legally required disclosure stripped by a templating bug, and a flaky test pattern shown side by side with its fix.

PlaywrightTypeScriptGitHub Actions
GitHub

Workflow diagram

1Each bug ships as an isolated repo folder with reproduction steps and a failing test
2CI runs the failing test to prove the bug, then runs the fixed test to prove the patch
3Flaky vs. stable patterns run side by side to show the delta

Integrations diagram

QA Practice Proj…GitHub Actions CI

Workflows

  • Each bug ships as an isolated repo folder with reproduction steps and a failing test
  • CI runs the failing test to prove the bug, then runs the fixed test to prove the patch
  • Flaky vs. stable patterns run side by side to show the delta

Integrations

  • GitHub Actions CI

Tools

  • Playwright
  • TypeScript
  • Node.js
  • GitHub

Core functionality

  • UTM parameter loss across redirect chains
  • Legally required disclosure stripped by a templating bug
  • Flaky test pattern paired with its deterministic rewrite

Testing

  • Playwright E2E tests with intentional and fixed variants
  • CI-enforced pass/fail on every push
  • Trace + video artifacts on failure for post-mortem review

TokenSavr

Project

A token efficiency dashboard for vibe coders.

Tracks burn rate, compares models, and scores prompt efficiency — built for developers who build with AI tools daily and want to know what they're actually spending.

LovableSupabaseTypeScript
Live Demo

Workflow diagram

1Log token usage per prompt/model → persist to Supabase
2Aggregate by day/model/project → compute burn rate and cost
3Score prompt efficiency (output value ÷ tokens spent) and surface outliers

Integrations diagram

TokenSavrSupabase (Postgres + A…Lovable Cloud

Workflows

  • Log token usage per prompt/model → persist to Supabase
  • Aggregate by day/model/project → compute burn rate and cost
  • Score prompt efficiency (output value ÷ tokens spent) and surface outliers

Integrations

  • Supabase (Postgres + Auth)
  • Lovable Cloud

Tools

  • TypeScript
  • React
  • Tailwind
  • Recharts
  • Vite

Core functionality

  • Live burn-rate dashboard across models and projects
  • Model-to-model cost and efficiency comparison
  • Per-prompt efficiency score with historical trend

Testing

  • Manual QA on data ingestion and aggregation edge cases
  • Auth + RLS checks so users only see their own usage
  • Chart-render smoke tests across empty, sparse, and dense datasets
Seed & Berri Social Media Content OS: how to use Compose to generate, review, and schedule a week of posts

Social Media Content OS

Client project — code private

A fully autonomous social content engine for a real fashion brand.

Scrapes trending topics daily, classifies relevance against brand voice, generates captions and imagery with Claude, plans a 7-day calendar, and publishes to Instagram and Threads through the Meta Graph API. Built first for Seed & Berri, architected to scale.

LovableSupabaseClaudeFirecrawlMeta API
LinkedIn

Workflow diagram

1Daily cron → Firecrawl scrape of trending fashion topics
2Claude classifies relevance vs. brand voice guardrails
3Approved topics → caption + image generation → 7-day calendar
4Scheduled publish to Instagram and Threads via Meta Graph API

Integrations diagram

Social Media Con…Meta Graph API (Instag…Anthropic ClaudeFirecrawlSupabase

Workflows

  • Daily cron → Firecrawl scrape of trending fashion topics
  • Claude classifies relevance vs. brand voice guardrails
  • Approved topics → caption + image generation → 7-day calendar
  • Scheduled publish to Instagram and Threads via Meta Graph API

Integrations

  • Meta Graph API (Instagram + Threads)
  • Anthropic Claude
  • Firecrawl
  • Supabase

Tools

  • Lovable
  • TypeScript
  • Cron / scheduled functions
  • Supabase Storage

Core functionality

  • Autonomous trend discovery scoped to brand voice
  • Caption + imagery generation with per-brand style prompts
  • 7-day content calendar with editable drafts
  • Direct publish + status tracking to Meta platforms

Testing

  • Dry-run mode that generates a full calendar without publishing
  • Guardrail eval on classifier output before any Meta API call
  • Publish-failure retry + alerting on Meta API errors

MockBook QA Suite

Project

A full sportsbook testing sandbox, built to prove automation depth.

A complete Playwright + TypeScript suite simulating a real sportsbook platform, covering API, SQL, and end-to-end layers with a custom test harness. Built specifically to target iGaming QA roles.

43 passing tests across 3 layers
PlaywrightTypeScriptGitHub Actions
GitHub

Workflow diagram

1Custom harness spins up a mock sportsbook environment before each run
2Tests execute across API, SQL, and UI layers with shared fixtures
3GitHub Actions runs the full suite on every push with artifacts on failure

Integrations diagram

MockBook QA SuiteGitHub Actions CISQL test database

Workflows

  • Custom harness spins up a mock sportsbook environment before each run
  • Tests execute across API, SQL, and UI layers with shared fixtures
  • GitHub Actions runs the full suite on every push with artifacts on failure

Integrations

  • GitHub Actions CI
  • SQL test database

Tools

  • Playwright
  • TypeScript
  • Node.js
  • SQL

Core functionality

  • API-layer contract and validation tests
  • SQL-layer data integrity checks (odds, wagers, settlements)
  • End-to-end user flows (bet placement, cashout, settlement)

Testing

  • 43 passing tests across 3 layers (API / SQL / E2E)
  • Deterministic fixtures — no flaky retries in CI
  • Trace + screenshot artifacts uploaded on failure
02 — Tooling

What I work with

AI & LLM

ClaudeOpenAIPromptfooPrompt Engineering

Automation & Testing

PlaywrightGitHub ActionsBrowserStackJira

Building

LovableSupabaseTypeScriptJavaScriptReact

DevOps

CI/CDGitHubRelease Gating
03 — Track record

Where I've worked

  1. Oct 2025 — Present

    Boyce Technologies

    Quality Technician

    Manufacturing QA for MTA transit infrastructure projects. Inspects laser-cut and fabricated metal components using precision tools.

  2. Mar 2025 — Jun 2025

    Creative Circle

    Quality Assurance Analyst

    Executed test cases across devices, conducted cross-browser testing with BrowserStack, and tracked bugs in GitHub with reproduction steps.

  3. Oct 2016 — Dec 2023

    Vox Media / The Verge

    Senior QA Analyst

    7+ years embedded QA across one of tech media's most-read publications.

  4. Mar 2015 — Feb 2016

    Thrillist Media Group

    QA Coordinator

    QA lead on editorial and digital product releases.

Full resume available on request
04 — Education & training

Where I leveled up

Engenious University

AI Test Automation of Large Language Models
2025
Intensive 8-Week Master's-Level Program
  • Designed and executed 150+ structured LLM regression tests spanning safety, ambiguity handling, hallucination detection, and structured output enforcement across multiple model providers.
  • Built Promptfoo evaluation pipelines with rubric-based scoring and release thresholds (≥90% compliance pass rate), functioning as release gates before production deployment.
  • Constructed gold sets and YAML-configured test suites to measure model accuracy and consistency, enabling data-driven release decisions.
  • Conducted 50+ adversarial prompt simulations to detect injection vulnerabilities and unsafe outputs, reducing false negatives in safety-critical scenarios.
  • Tuned model parameters (temperature, top-p) to reduce output variance and improve determinism by ~25%, balancing accuracy against cost and latency.
  • Implemented structured JSON validation checks (≥99% schema integrity) for downstream system reliability, catching malformed outputs before they reached production.
  • Integrated automated evaluation tests into GitHub Actions workflows for pre-release validation and continuous regression detection.

Per Scholas

Certificate, Quality Assurance & Software Testing Principles
New York, NY
  • Learned the fundamentals of software quality assurance and the software development life cycle (SDLC).
  • Learned how to create and execute test cases, document results, and report defects clearly.
  • Developed skills in manual testing, including functional, regression, integration, and user acceptance testing.
  • Learned how to identify, reproduce, and communicate software bugs so developers could troubleshoot and fix them.
  • Built a foundation in QA processes, testing methodologies, and quality standards that I’ve continued to apply throughout my QA career.

Adelphi University

Bachelor of Arts, Liberal Arts & General Studies
Garden City, NY
05 — Live signal

Still building

Latest open-source activity from my GitHub.

06 — Inbox open

Let's talk

Open to AI QA, automation engineering, and AI product roles. Always happy to talk shop.

Contact LinkedIn GitHub