Ferris Akkad

AI Product Manager & Builder

I like to build the things I wish existed.

Scroll to explore

About

I pick up new domains fast and turn ambiguity into reliable products, whether that's cloud infrastructure, financial services, operations, or compliance. I gravitate toward 0-to-1 work and I'm most useful when the problem isn't well-defined yet.

AI changed how I work. I went from writing PRDs to living in the terminal building products. New tools ship every day and I'm deep in it: LLMs, agents, RAG, knowledge graphs. I've always been technical, but now I can actually build what I'm thinking.

UVA McIntire MSMIT. JMU BS. Based in DC.

Projects

EvalKitBuilding

EvalKit

Eval Infrastructure

CLI-first LLM evaluation framework with G-Eval scoring and structured reporting.

PythonYAMLG-Eval
JudgeBenchBuilding

JudgeBench

Eval Infrastructure

Measures whether your LLM judge can be trusted with statistical rigor.

PythonStatistics
DriftWatchBuilding

DriftWatch

Eval Infrastructure

Automated prompt regression detector using McNemar's test.

PythonSQLiteMcNemar's

The Eval Stack

Three tools that chain together to give LLM quality assurance real statistical power.

JudgeBench
Validate the judge
EvalKit
Score outputs
DriftWatch
Catch regressions

Writing

Thoughts on building AI products, eval infrastructure, and working with LLMs.

Experience

2024–Present

Senior Product Manager · AWS Global Services

  • AI-powered decision systems for enterprise cloud transformation
2021–2024

Senior Product Manager · Amazon Fulfillment

  • Compliance systems governing every package shipped by Amazon globally
2020–2021

Senior Product Manager · Freddie Mac

  • Automated income verification and platform modernization for loan eligibility
2017–2020

Risk Manager · Freddie Mac

  • Loan eligibility system — whether Freddie Mac buys a loan from a bank

Connect

Always interested in talking about AI eval, product building, or interesting problems. Feel free to reach out.