Available for AI QA Engineer roles

Steven Leon

LLM Evaluation and Quality Assurance Engineer

I test, evaluate, and ship AI-powered products — from LLM regression suites to solo projects.

0+ years
QA at Vox Media (The Verge)
0+
LLM regression tests shipped
0
Production tools built solo
0
Passing tests in one suite alone
01 — Selected work

What I've built

DropLinx landing page: paste a URL and get a week of content across seven formats

DropLinx

Project

Drop a link. Get a week of content.

A multi-channel AI content engine that turns any URL — or a plain prompt — into a finished week of content: blog post, newsletter, post pack, reel script, YouTube Short, product ad, and slide deck. Two modes (Social Content and Marketing Ads) reuse the same source once and fan it out across every channel, each output shipped finished rather than drafted.

7 output formats • 2 input modes • seconds per run
TypeScriptReactClaude APISupabaseLovable Cloud
Live Demo

Workflow diagram

1Paste URL (auto-detected on paste) or Prompt an Idea → source normalized and summarized
2Mode switch routes the run through Social Content or Marketing Ads prompt sets
3Fan-out to parallel generators for blog, newsletter, post pack, reel script, YouTube Short, product ad, slide deck
4Outputs persisted to Supabase and returned as one browsable content pack

Integrations diagram

DropLinxAnthropic Claude APISupabase (Postgres + A…Lovable CloudURL scraping / metadat…

Workflows

  • Paste URL (auto-detected on paste) or Prompt an Idea → source normalized and summarized
  • Mode switch routes the run through Social Content or Marketing Ads prompt sets
  • Fan-out to parallel generators for blog, newsletter, post pack, reel script, YouTube Short, product ad, slide deck
  • Outputs persisted to Supabase and returned as one browsable content pack

Integrations

  • Anthropic Claude API
  • Supabase (Postgres + Auth)
  • Lovable Cloud
  • URL scraping / metadata extraction

Tools

  • TypeScript
  • React
  • Tailwind
  • Vite
  • Supabase
  • GitHub

Core functionality

  • One input in, seven finished formats out — no per-channel rewriting
  • Dual input modes: paste a live URL or describe an idea from scratch
  • Social Content vs Marketing Ads presets with per-format prompt templates and brand voice tuning
  • Auto paste detection so a copied link kicks off a run with no extra click
  • Auth-gated dashboard with saved history and re-generation of individual formats

Testing

  • Smoke suite across all 7 generators and both input modes before each release
  • Prompt regression checks on tone, length thresholds, and format structure
  • Negative-path coverage: unreachable URLs, paywalled pages, empty scrapes, provider rate limits
  • Cross-browser and responsive checks on the paste flow and output viewer
Ghost Index landing page: Stop getting ghosted. Hold employers accountable.

Ghost Index by DropLinx

Project

Screenshot-verified employer ghosting reports.

A public index of employers that ghost candidates — every report is backed by screenshot evidence of the application and interview trail, so job seekers can see which companies go silent before they invest hours in a hiring process.

Screenshot-verified reports • public employer index
TypeScriptReactSupabaseLovable Cloud
Live Demo

Workflow diagram

1Candidate submits a ghosting report with employer, role, and timeline
2Screenshot evidence uploaded and attached to the report
3Report reviewed for verification before it appears publicly
4Verified reports roll up into a searchable per-employer ghosting record

Integrations diagram

Ghost Index by D…Supabase (Postgres + A…Lovable Cloud

Workflows

  • Candidate submits a ghosting report with employer, role, and timeline
  • Screenshot evidence uploaded and attached to the report
  • Report reviewed for verification before it appears publicly
  • Verified reports roll up into a searchable per-employer ghosting record

Integrations

  • Supabase (Postgres + Auth + Storage)
  • Lovable Cloud

Tools

  • TypeScript
  • React
  • Tailwind
  • Vite
  • Supabase
  • GitHub

Core functionality

  • Evidence-first reporting: every claim requires screenshot proof
  • Searchable employer index showing ghosting history
  • Verification step that separates proven reports from unverified ones

Testing

  • Upload-path coverage: file types, size limits, failed and duplicate uploads
  • Form validation and negative-path checks on report submission
  • Search and employer page checks across browsers and screen sizes
Seed & Berri landing page: Four lines, one love story — hand-embellished denim collections.

Seed & Berri

Project

Hand-embellished denim storefront with an AI try-on.

Online shop for a small-batch denim brand with four lines: Women, Girls, Mommy & Me, and For My AKA's. It includes Magic Mirror, an AI tool where shoppers upload a photo and see how an outfit would look on them before buying.

4 collections • AI virtual try-on • ships worldwide
TypeScriptReactShopifyLovable CloudAI
In Production

Workflow diagram

1Shopper browses a collection → quick view → add to cart → Shopify checkout
2Magic Mirror: upload a full-body photo → pick an outfit → AI generates a try-on preview
3Free demo try-on for guests; sign in unlocks unlimited try-ons
4Newsletter signup for early access to new drops

Integrations diagram

Seed & BerriShopify Storefront (pr…Lovable Cloud (auth + …AI image generation fo…

Workflows

  • Shopper browses a collection → quick view → add to cart → Shopify checkout
  • Magic Mirror: upload a full-body photo → pick an outfit → AI generates a try-on preview
  • Free demo try-on for guests; sign in unlocks unlimited try-ons
  • Newsletter signup for early access to new drops

Integrations

  • Shopify Storefront (products, cart, checkout)
  • Lovable Cloud (auth + storage)
  • AI image generation for virtual try-on

Tools

  • TypeScript
  • React
  • Tailwind
  • Vite
  • Shopify
  • GitHub

Core functionality

  • Four collection lines with a weekly lookbook and quick-view product cards
  • AI virtual fitting room with a clear disclaimer that results may vary
  • Accounts, search, cart, and social links to Instagram and Facebook
  • Brand pages: Meet the Founder, Berri Ambassadors, As Seen On

Testing

  • End-to-end checks of browse → cart → checkout across devices
  • Try-on upload checks: bad photos, large files, failed AI responses, demo limits
  • Sign-in and account flow checks, plus layout checks on mobile and desktop
AI Assistant Eval promptfoo benchmark: 5 test categories and a 94% weighted pass rate

AI Assistant Eval

Project

Systematic evaluation of AI assistant responses for accuracy, tone, and helpfulness.

A Promptfoo-based evaluation suite that benchmarks AI assistant responses against rubric-driven criteria. Uses model-graded assertions, deterministic checks, and custom scoring scripts to measure response quality, factual correctness, and user-helpfulness across multiple test scenarios.

Multi-provider evals • rubric scoring • CI-ready
promptfooOpenAIAnthropicJavaScriptYAMLVS Code
GitHub

Workflow diagram

1Define test cases in YAML with prompt templates, inputs, and expected behaviors
2Run model-graded and deterministic assertions against assistant responses
3Aggregate scores and export results to a human-readable dashboard

Integrations diagram

AI Assistant EvalOpenAI APIAnthropic APIpromptfoo

Workflows

  • Define test cases in YAML with prompt templates, inputs, and expected behaviors
  • Run model-graded and deterministic assertions against assistant responses
  • Aggregate scores and export results to a human-readable dashboard

Integrations

  • OpenAI API
  • Anthropic API
  • promptfoo

Tools

  • JavaScript
  • YAML
  • VS Code
  • Node.js
  • GitHub

Core functionality

  • Rubric-based evaluation of assistant accuracy and helpfulness
  • Side-by-side response comparison across providers and prompts
  • Configurable pass/fail thresholds and regression detection

Testing

  • Model-graded assertions for tone and helpfulness scoring
  • Deterministic assertions for factual correctness and format validation
  • Reusable test suite structure for continuous eval runs

Gossip Pipeline

Project

Three Claude agents that turn any product into a ready-to-shoot TikTok storytime.

One prompt in: a research pass, a first-person gossip script with a three-second hook, and scene-by-scene visual direction. Each agent hands structured JSON to the next, and an optional paid step generates Seedance clips with ElevenLabs voiceover merged by ffmpeg.

3 agents • JSON handoffs • dry run by default
TypeScriptClaude APIReplicate SeedanceElevenLabsffmpeg
GitHub

Workflow diagram

1Product name or URL → Agent 1 Researcher proposes three gossip-worthy angles
2research.json → Agent 2 Story Writer drafts a first-person script with a 3-second hook
3script.json → Agent 3 Visual Director builds scene prompts with durations and voiceover lines
4Opt-in step: Replicate Seedance clips + ElevenLabs voiceover merged with ffmpeg into output/

Integrations diagram

Gossip PipelineAnthropic Claude APIReplicate Seedance 1.0…ElevenLabs voiceoverffmpegGitHub Actions CI

Workflows

  • Product name or URL → Agent 1 Researcher proposes three gossip-worthy angles
  • research.json → Agent 2 Story Writer drafts a first-person script with a 3-second hook
  • script.json → Agent 3 Visual Director builds scene prompts with durations and voiceover lines
  • Opt-in step: Replicate Seedance clips + ElevenLabs voiceover merged with ffmpeg into output/

Integrations

  • Anthropic Claude API
  • Replicate Seedance 1.0 Lite
  • ElevenLabs voiceover
  • ffmpeg
  • GitHub Actions CI

Tools

  • TypeScript
  • Node.js
  • VS Code
  • GitHub

Core functionality

  • Three-agent pipeline with structured JSON handoffs between every step
  • Dry run by default — research, script, and visuals written with zero paid API calls
  • #ad disclosure enforced in code, not just requested in the prompt
  • Per-step timing logs so each stage's cost in time is visible
  • A failed clip or voiceover lands in the run's warnings field instead of killing the pipeline

Testing

  • Tested the way I'd test any product: dry run first, paid paths strictly opt-in
  • Failure injection on Replicate and ElevenLabs calls to confirm partial runs still complete
  • Disclosure enforcement checked against generated scripts before anything ships
  • CI workflow runs on every push

Mission-Critical Compliance Tests

Project

FDCPA compliance for an AI debt collector, enforced as a hard gate.

A Promptfoo eval suite that runs one prompt against multiple models with strict pass/fail. In debt collection a missed disclosure or an ignored "stop calling me" is a legal problem, so compliance here is a gate rather than a score to optimize.

6 non-negotiable rules • 2 providers • strict pass/fail
promptfooOpenAIAnthropicYAMLGitHub Actions
GitHub

Workflow diagram

1System prompt encodes six non-negotiable FDCPA rules, templated with Nunjucks
2Each test injects user_message plus optional caller_context and time_context
3Same prompt runs against two providers side by side for direct comparison
4GitHub Actions runs the eval on every push with per-test metrics and weights

Integrations diagram

Mission-Critical…OpenAI APIAnthropic APIpromptfooGitHub Actions CI

Workflows

  • System prompt encodes six non-negotiable FDCPA rules, templated with Nunjucks
  • Each test injects user_message plus optional caller_context and time_context
  • Same prompt runs against two providers side by side for direct comparison
  • GitHub Actions runs the eval on every push with per-test metrics and weights

Integrations

  • OpenAI API
  • Anthropic API
  • promptfoo
  • GitHub Actions CI

Tools

  • YAML
  • promptfoo
  • Node.js
  • VS Code
  • GitHub

Core functionality

  • Mini-Miranda disclosure required in full on first contact
  • Cease-communication requests honored immediately, with no salvage attempt
  • Debt details never disclosed to anyone but the confirmed debtor
  • No threats of arrest, jail, or garnishment without a court order
  • Contact refused outside 8:00 AM–9:00 PM local time

Testing

  • icontains / not-icontains assertions for hard string checks like balance leakage
  • llm-rubric assertions for judgment calls like pressure after a cease request
  • Time-window tests at 6:30 AM and 10:45 PM confirm refusal and callback behavior
  • Per-test metrics and weights trace every failure back to one specific rule
Seed & Berri Social Media Content OS: how to use Compose to generate, review, and schedule a week of posts

Social Media Content OS

Client project — code private

A fully autonomous social content engine for a real fashion brand.

Scrapes trending topics daily, classifies relevance against brand voice, generates captions and imagery with Claude, plans a 7-day calendar, and publishes to Instagram and Threads through the Meta Graph API. Built first for Seed & Berri, architected to scale.

LovableSupabaseClaudeFirecrawlMeta API
LinkedIn

Workflow diagram

1Daily cron → Firecrawl scrape of trending fashion topics
2Claude classifies relevance vs. brand voice guardrails
3Approved topics → caption + image generation → 7-day calendar
4Scheduled publish to Instagram and Threads via Meta Graph API

Integrations diagram

Social Media Con…Meta Graph API (Instag…Anthropic ClaudeFirecrawlSupabase

Workflows

  • Daily cron → Firecrawl scrape of trending fashion topics
  • Claude classifies relevance vs. brand voice guardrails
  • Approved topics → caption + image generation → 7-day calendar
  • Scheduled publish to Instagram and Threads via Meta Graph API

Integrations

  • Meta Graph API (Instagram + Threads)
  • Anthropic Claude
  • Firecrawl
  • Supabase

Tools

  • Lovable
  • TypeScript
  • Cron / scheduled functions
  • Supabase Storage

Core functionality

  • Autonomous trend discovery scoped to brand voice
  • Caption + imagery generation with per-brand style prompts
  • 7-day content calendar with editable drafts
  • Direct publish + status tracking to Meta platforms

Testing

  • Dry-run mode that generates a full calendar without publishing
  • Guardrail eval on classifier output before any Meta API call
  • Publish-failure retry + alerting on Meta API errors

MockBook QA Suite

Project

A full sportsbook testing sandbox, built to prove automation depth.

A complete Playwright + TypeScript suite simulating a real sportsbook platform, covering API, SQL, and end-to-end layers with a custom test harness. Built specifically to target iGaming QA roles.

43 passing tests across 3 layers
PlaywrightTypeScriptGitHub Actions
GitHub

Workflow diagram

1Custom harness spins up a mock sportsbook environment before each run
2Tests execute across API, SQL, and UI layers with shared fixtures
3GitHub Actions runs the full suite on every push with artifacts on failure

Integrations diagram

MockBook QA SuiteGitHub Actions CISQL test database

Workflows

  • Custom harness spins up a mock sportsbook environment before each run
  • Tests execute across API, SQL, and UI layers with shared fixtures
  • GitHub Actions runs the full suite on every push with artifacts on failure

Integrations

  • GitHub Actions CI
  • SQL test database

Tools

  • Playwright
  • TypeScript
  • Node.js
  • SQL

Core functionality

  • API-layer contract and validation tests
  • SQL-layer data integrity checks (odds, wagers, settlements)
  • End-to-end user flows (bet placement, cashout, settlement)

Testing

  • 43 passing tests across 3 layers (API / SQL / E2E)
  • Deterministic fixtures — no flaky retries in CI
  • Trace + screenshot artifacts uploaded on failure
02 — Tooling

What I work with

AI & LLM

ClaudeOpenAIPromptfooPrompt Engineering

Automation & Testing

PlaywrightGitHub ActionsBrowserStackJira

Building

LovableSupabaseTypeScriptJavaScriptReact

DevOps

CI/CDGitHubRelease Gating
03 — Track record

Where I've worked

  1. Oct 2025 — Present

    Boyce Technologies

    Quality Technician

    Manufacturing QA for MTA transit infrastructure projects. Inspects laser-cut and fabricated metal components using precision tools.

  2. Mar 2025 — Jun 2025

    Creative Circle

    Quality Assurance Analyst

    Executed test cases across devices, conducted cross-browser testing with BrowserStack, and tracked bugs in GitHub with reproduction steps.

  3. Oct 2016 — Dec 2023

    Vox Media / The Verge

    Senior QA Analyst

    7+ years embedded QA across one of tech media's most-read publications.

  4. Mar 2015 — Feb 2016

    Thrillist Media Group

    QA Coordinator

    QA lead on editorial and digital product releases.

Full resume available on request
04 — Education & training

Where I leveled up

Engenious University

AI Test Automation of Large Language Models
2025
Intensive 8-Week Master's-Level Program
  • Designed and executed 150+ structured LLM regression tests spanning safety, ambiguity handling, hallucination detection, and structured output enforcement across multiple model providers.
  • Built Promptfoo evaluation pipelines with rubric-based scoring and release thresholds (≥90% compliance pass rate), functioning as release gates before production deployment.
  • Constructed gold sets and YAML-configured test suites to measure model accuracy and consistency, enabling data-driven release decisions.
  • Conducted 50+ adversarial prompt simulations to detect injection vulnerabilities and unsafe outputs, reducing false negatives in safety-critical scenarios.
  • Tuned model parameters (temperature, top-p) to reduce output variance and improve determinism by ~25%, balancing accuracy against cost and latency.
  • Implemented structured JSON validation checks (≥99% schema integrity) for downstream system reliability, catching malformed outputs before they reached production.
  • Integrated automated evaluation tests into GitHub Actions workflows for pre-release validation and continuous regression detection.

Per Scholas

Certificate, Quality Assurance & Software Testing Principles
New York, NY
  • Learned the fundamentals of software quality assurance and the software development life cycle (SDLC).
  • Learned how to create and execute test cases, document results, and report defects clearly.
  • Developed skills in manual testing, including functional, regression, integration, and user acceptance testing.
  • Learned how to identify, reproduce, and communicate software bugs so developers could troubleshoot and fix them.
  • Built a foundation in QA processes, testing methodologies, and quality standards that I’ve continued to apply throughout my QA career.

Adelphi University

Bachelor of Arts, Liberal Arts & General Studies
Garden City, NY
05 — Live signal

Still building

Latest open-source activity from my GitHub.

06 — Inbox open

Let's talk

Open to AI QA, automation engineering, and AI product roles. Always happy to talk shop.