SASheharyar Akhtar
GenAI builds · LLM evaluation · Lahore, remote

I build GenAI products, and the evaluation that tells you they work.

Five years across production ML, enterprise data and LLM evaluation. For the last two, I've built the systems that review AI training data at scale for frontier-lab and big-tech AI programs. I take on contract work, fixed-scope builds and evaluation audits.

Book a 20-minute call → See the work
Email LinkedIn GitHub CV
17languages under automated review
1,000+annotators reviewed in production
5–10%revenue lift from production A/B
3public benchmark audits
Work with me

Three ways to bring me in.

Contract

Embedded with your team

Part-time or full-time, by the hour or the month. LLM pipelines, automated review, evaluation and the data work around them.

Fixed-scope build

A GenAI product, start to finish

A defined product or prototype for a set price, from brief to deployed app, with evaluation built in so you know how often it is right.

Audit · Rubric Agents

Find what's broken

Send 20 tasks from your RL environments or eval sets. You get back which ones are broken, and why, each finding reproduced by running it.

Rates are agreed on a short call once the scope is clear.

Selected work · products

Products I've built and shipped.

For an AI content studio · Oct 2026

A complete photoshoot from a few uploads.

Upload a model, the garments and the shoes, and get back a finished photoshoot, consistent from shot to shot.

  • Claude plans the shoot as director and picks the references each shot needs.
  • Gemini renders the shots in parallel, with retries, in about 20 to 40 seconds each.
  • Every run is stored with its prompts, references and timings.
Claude as directorGemini rendersNext.js · Vercel
run · 5 of 5 shots rendered
Four photoshoot frames of the same model in a beige top and burgundy skirt, from front, back, side and a closer upper-body angle
Own product · Oct 2026

From a brief to a factory-ready tech pack.

Design Studio turns a brief, a palette and reference images into a lineup of apparel designs. Shortlist them into a collection and export a tech pack: style numbers, fit, fabric, colorways with hex values, and print technique and placement.

lineup generationtech pack exportNext.js · Vercel
tech pack · Beach Explorer Henley
Tech pack page for a sea-green Henley tee with a beach graphic, listing style number, fit, fabric, colorway and print technique
Selected work · evaluation at scale

The systems that decide what's good enough.

Client work for Turing, described without naming the AI companies it was delivered to.

Turing · big-tech AI program · Aug 2025 to Jan 2026

An automated reviewer in production

As one of two engineers on the design, I built the reviewer that checks annotation quality inside a labeling platform. I defined the quality dimensions, wrote the system and agent prompts, and took it to production.

Earlier roles on the program
  • Pod lead: a team of 5 annotators, with onboarding, training and daily QA reviews.
  • Calibrator: the core-team role that sets the quality bar for the program, so hallucinated or unreasoned annotations don't pass.
quality dimensionsagent promptsproduction
Turing · Jul to Sep 2026

Audit and repair for agentic tasks

A pipeline that finds and repairs broken agentic tasks. Every task faces two runs: the known-good solution must pass, and a run that does nothing must fail. Tasks that slip either way are repaired and run again.

golden trajectoryno-op runauto-repair
Turing · Jan to Jun 2026

Evaluation platforms, merged into one API

Built evaluation platforms for chain-of-thought and ReAct-style annotation projects, then merged them into one service whose API the client's frontend team wired into their workflow.

The UI reviewer

A reviewer for UI annotations that judges each action from the difference between the interface's starting and final state.

CoT and ReActAPI serviceUI state diffs
Turing · frontier-lab coding program · Dec 2024 to May 2025

A staged code-repair pipeline

An Agentless-style pipeline for SWE-bench-style tasks that narrows the problem step by step before attempting a fix. The client adopted it for a five-month campaign, with me as the campaign's tooling lead.

4 and 6 stepsColab · GCP · Kubernetesadopted by client
Smaller builds
Public-health interview study · 2026

Qualitative research coded by three models

Transcribed 21 bilingual Urdu-English interviews with speaker separation, then had Claude, GPT-4o and Gemini code them independently. Their codes were reconciled into 41 codes and 6 themes and compared across two participant groups.

21 interviews3 independent coders41 codes · 6 themes
For a friend's business · 2026

Papas Den: handwritten receipts to clean data

Receipts go in by drag and drop, and a spreadsheet comes out, ready to load into the business's data platform. It replaced reading each receipt by hand and is in daily use, at around 10 receipts a day.

in daily use~10 receipts a day
Ventures

Two things I run alongside client work.

Find the broken tasks before your model trains on them.

Quality control for agentic tasks, RL environments and AI training data. Every task is checked, run and repaired, then routed to ready, needs a human, or failed.

DeepSWE6 of the 30 hardest tasks reject a correct solution
FrontierSWE13 of 13 tasks reviewed have a score-changing finding
Terminal-Bench4 of 5 sampled tasks passed the standard gate but still had a defect
Read the audits → rubricagents.com

Understand risk. Don't predict returns.

A glowing bowl-shaped phase portrait of SPY, with a gold ring marking the 90 percent stability basin and four regime labels around the rim
  • Regime DetectorVolatility and trend regimes with Markov transition probabilities.
  • Phase RegimeAny asset in velocity and acceleration space, with a 90% stability basin.
  • Portfolio OptimizerTrue diversification against MPT and risk parity.
  • Net Worth TrackerA whole portfolio, valued daily across currencies.
Recent research
  • On 10 years of SPY, regimes persist about 91% of days inside and outside the basin alike, but 6 of 16 basin exits came through Accelerating Down, a regime covering only 14% of normal days.
  • On an 8-asset universe, first-factor regime transitions lead asset-level transitions by about 13 days.
NO EXECUTION. NO ADVICE. NO ALERTS.
markovchainfinance.com →
Experience

Five years, three very different rooms.

Sep 2024 to now

Turing client · contract

  • Trainer, then pod lead within two weeks, then calibrator within two months on a frontier-lab agentic coding program.
  • Tooling champion for a five-month Agentless campaign the client adopted.
  • Since then, auto-reviewers, evaluation platforms and task-repair pipelines for big-tech AI clients. Now scoping research-grade annotation sets for labs.
Aug 2023 to Nov 2024

Systems Limited data and AI consultant

  • ESG data ingestion and validation platform, with the app around it, for a global consumer goods company, standardizing data across 50+ business units.
  • Credit scoring and forecasting pipelines with PySpark and MLflow on AWS and Azure, with SHAP explanations for compliance.
  • 20+ enterprise proposals across the US, the Gulf, Australia and Pakistan (pitch decks, RFP responses, bills of quantities), contributing $5M+ in pipeline. Led a bid for a Gulf telecom operator.
Apr 2021 to Aug 2023

Afiniti data scientist I to II

  • Caller-agent pairing for telecom operators: Bayesian IRT models in Stan estimate agent performance by caller segment, feeding a production optimizer.
  • 5 to 10% revenue lift across clients. On one queue, 8%+ revenue per call against the client's own routing algorithm on an 80/20 benchmark split.
  • Led client launches in Iberia, including two pilots that converted to full contracts. Owned the SQL that matches outcomes to calls, and engineered real-time features that improved model validation by 50%+.

BS Mechanical Engineering, GIKI. Came to data science through self-directed study.

How I think

Five rules I build by.

  1. 01Reducing emotionally amplified mistakes beats maximizing returns. Show people where they are, not what will happen.
  2. 02A model that can't explain why it's wrong is more dangerous than no model at all.
  3. 03Regimes matter more than averages. A 2% move in a calm market is not the same as a 2% move in a volatile one.
  4. 04The hardest problems are rarely technical. They're about calibrating human judgment at scale without losing signal.
  5. 05If a tool doesn't change a decision, it's a dashboard. I build systems that force better choices under uncertainty.
Writing and tools

Notes along the way.

Basics of Markov Chain Monte Carlo

How MCMC uses randomness to find high-probability regions in Bayesian inference, with intuition over formulas.

Towards Data Science →

Feature importance in Bayesian regression

Assessing a predictor's real contribution in a Bayesian model, beyond correlation.

Towards Data Science →

Building RAG systems from scratch

Architecture decisions in retrieval-augmented generation, from embeddings to deployment.

LinkedIn →
Modeling

Python, R, Stan, PyMC, SQL, Bayesian and Markov models, experimentation

LLMs

Claude, GPT and Gemini APIs, agent and prompt design, automated review

Apps

Next.js, React, Flask, Vercel, Supabase, Postgres

Data and cloud

PySpark, MLflow, Azure ML, AWS, GCP, Docker

Contact

Have a GenAI product to build, or one you need to trust?

Book a call, email sheharyar7146@gmail.com, or send a message. Based in Lahore (UTC+5), working with teams in any time zone.

Book a 20-minute call → LinkedIn