Khashayar Bayati
Khashayar Bayati

Khashayar Bayati

I build production AI systems end to end: agentic platforms for business users, optimization and forecasting models, and the evaluation that decides whether they are ready for daily use.

Data Scientist, Horizon3 AI Labs at Unilever Toronto Email LinkedIn GitHub
01

Selected work

Most of this work starts with a group of people who need a decision made better: buyers planning distribution, a brand team fielding customer questions, forecasters pricing a commodity. I gather the requirements from them, build the system, and measure whether it actually helped, on their terms rather than a benchmark's. That covers agentic platforms, optimization, time-series models, and a full-stack product I built alone.

sources conversion processing to site feedstock A feedstock B intermediate A intermediate B process side input product solving the planbars: capacity used intermediate A capacity is the binding constraint raise it, or route more flow through intermediate B feasible
one request, traced faithfulness0.94 completeness0.89 costper span 41-case regression suite, replayed against production traffic
pass^k reliabilityLLM as judge
as first reported 9% as first reported 9% improvement over baseline by market regime calm regimes 8% shock regimes 41% a single average was standing in for two different models
gapped cutoffsleakage checksregime stratified
1st of 14 on mean CRPS, 1 to 13 weeksskill +0.27 vs naivecoverage 89% vs 80% target
kaivest.ai
Kaivest dashboard for a demo account: monthly budget with bills still due, a financial health score with trend, and spending summary cards.
live, invite-only beta1,200 unit tests, 49 end to end

Horizon3 AI Labs, Unilever

Distribution optimization agent

An agentic platform for multi-hop distribution optimization, used by the procurement team behind a multi-billion-dollar supply chain. Buyers set up runs in plain language and adjust constraints before the solver fires.

It also handles the failure case. When no valid plan exists, the system locates the binding constraint mathematically and names the input to change, which cuts root-cause diagnosis from days or weeks to minutes and simplifies quarterly planning for the buyers who depend on it.

Python, LangGraph, human in the loop, runs on the lab's on-prem GPU server

Problem. The procurement team behind a multi-billion-dollar supply chain needed to run multi-hop distribution optimizations themselves, interpret the results, and understand why a given plan came back infeasible. In practice that meant repeated spreadsheet exchanges across teams, and root-causing an infeasible plan took analysts days or weeks.

What I built. An agentic platform in Python and LangGraph that lets buyers set up runs in natural language, with a human-in-the-loop step where they review and adjust constraints before the solver runs. On top of that, a mathematical infeasibility-analysis capability, built to support Q3 2026 planning, that identifies the binding constraint and points at the exact input changes that would make the plan feasible. It runs on the lab's on-prem Supermicro GPU server.

How I know it worked. Validation came from the business users. Infeasibility diagnosis dropped from days or weeks to minutes and simplified their quarterly planning. Along the way I found and fixed a cost-duplication defect in the optimizer, confirmed against the numbers the users were seeing.

What it taught me. The demo case, where a plan solves cleanly, was never where the value was. The value was in the failure path, and in letting people change inputs before the solve rather than explaining a bad answer afterward.

Horizon3 AI Labs, Unilever

Conversational agent evaluation

I own evaluation and observability for a retail-facing conversational agent. Every request carries a trace with per-span latency, token and cost visibility, and judge-scored faithfulness and completeness.

A 41-case regression suite with pass^k reliability scoring let a major architecture change be measured as a quality against latency trade-off. Replaying real production questions through the traced agent surfaced a 25% failure rate that the original suite had not reached.

FastAPI, Gemini function calling, an internal platform comparable to LangSmith or Langfuse

Problem. A retail-facing conversational product agent, built in Python and FastAPI on Gemini function calling to connect with retailer shopping agents, had no dependable way to tell whether a change made it better or worse. Decisions about architecture were being argued rather than measured.

What I built. I instrumented the agent on an internal LLM observability platform comparable to LangSmith or Langfuse, giving per-request trace, token and cost visibility, with LLM-as-judge scoring for faithfulness and completeness. Then a 41-case regression suite scored with pass^k reliability, so that a case which passes sometimes is not counted the same as one that passes every time.

How I know it worked. The suite let a major architecture change be benchmarked as a quality against latency trade-off with numbers attached. Replaying real production prompts through the traced agent surfaced a 25% failure rate on real customer questions that the existing tests had missed, and led me to a model-migration break in the tool-calling layer, which I diagnosed and fixed to unblock the retirement of an older model.

What it taught me. A test suite written by the team measures what the team imagined users would ask. Replaying what users actually asked is a different measurement, and it was the one that found the real failure.

Horizon3 AI Labs, Unilever

Foundation model benchmarking

I benchmark eight time-series foundation model families against production and statistical baselines, across three commodities, eight cutoff dates and seven horizons.

The evaluation protocol changed the conclusion. Gapped development and holdout splits, block-bootstrap intervals and leakage checks against model release dates moved a headline improvement from 9% to roughly 21%, and stratifying by regime showed 8% in calm markets against 41% in shocks.

Chronos, TimesFM, Moirai, Toto, Lag-Llama, MOMENT, Timer, TTM. LoRA fine-tuning with hyperparameter sweeps. Co-presented at Vector Institute Learn Day.

Problem. Decide whether time-series foundation models should replace or complement the production forecast for three commodities, CPO, PKO and FAL. Eight model families, Chronos, TimesFM, Moirai, Toto, Lag-Llama, MOMENT, Timer and TTM, against production and statistical baselines, over eight cutoff dates and seven horizons. A first pass reported a 9% headline improvement.

What I built. A leakage-aware evaluation protocol: gapped development and holdout cutoff splits so tuning on one window cannot bleed into the next, block-bootstrap confidence intervals that respect autocorrelation, error reporting stratified by market regime rather than pooled, and leakage checks against each model's release date in both directions. LoRA fine-tuning of TimesFM and Chronos with hyperparameter sweeps, to test whether adaptation beat zero-shot use.

How I know it worked. Under the corrected protocol the headline improvement moved from 9% to roughly 21%, and the regime split showed why the pooled figure had been misleading: 8% in calm periods and 41% in shocks. Co-presented at Vector Institute Learn Day in July 2026.

What it taught me. A single average over a series that has regimes is not a summary of two numbers, it is a number that is wrong for both. Reporting by regime changed what the models were for.

shock yearscalm yearsin-samplegrades its own examfixed splithonest, but tests only the calm tailleave-one-outuses the future to chooserolling originhonest, respects timetimechooses the settingreports the errordoes both (the trap)
Why selection and evaluation need disjoint cutoffs. Only rolling origin and a gapped fixed split keep the two jobs apart.
AAAWgmp1bWIAAAAeanVtZGMycGEAEQAQgAAAqgA4m3EDYzJwYQAAABZcanVtYgAAAEdqdW1kYzJtYQARABCAAACqADibcQN1cm46YzJwYToyMmZjNjk0Zi02NTg1LTRiODgtYjM1Mi1iNzU4NGFmMWZiMDQAAAADl2p1bWIAAAApanVtZGMyYXMAEQAQgAAAqgA4m3EDYzJwYS5hc3NlcnRpb25zAAAAALxqdW1iAAAARGp1bWRjYm9yABEAEIAAAKoAOJtxE2MycGEuaW5ncmVkaWVudC52MwAAAAAYYzJzaDZU1xnu6UtCstaHGQ5JvpUAAABwY2JvcqNpZGM6Zm9ybWF0bWltYWdlL3N2Zyt4bWxqaW5zdGFuY2VJRHgseG1wOmlpZDpjNDcwNDhjYy0yOWE0LTQxMTctYjQ0My02ZGQ5MDBmODNlZmVscmVsYXRpb25zaGlwaHBhcmVudE9mAAAB4mp1bWIAAABBanVtZGNib3IAEQAQgAAAqgA4m3ETYzJwYS5hY3Rpb25zLnYyAAAAABhjMnNo514FAlrgAh5RPLMqcTMeigAAAZljYm9yomdhY3Rpb25zgqJmYWN0aW9ua2MycGEub3BlbmVkanBhcmFtZXRlcnOha2luZ3JlZGllbnRzgaJjdXJseC1zZWxmI2p1bWJmPWMycGEuYXNzZXJ0aW9ucy9jMnBhLmluZ3JlZGllbnQudjNkaGFzaFggFk6d1gR4R3mXfElFdv3TYwclbfGam9/mSL4oFp6C9OekZmFjdGlvbngdY29tLmFudGhyb3BpYy5jbGF1ZGUucHJvdmlkZWRqcGFyYW1ldGVyc6F4H2NvbS5hbnRocm9waWMub3JpZ2luLWNvbmZpZGVuY2VndW5rbm93bmtkZXNjcmlwdGlvbnhmQ2xhdWRlIHByb3ZpZGVkIHRoaXMgZmlsZSBhdCB0aGUgcmVxdWVzdCBvZiBhIHVzZXIgYW5kIG1heSBoYXZlIGNyZWF0ZWQgb3IgbW9kaWZpZWQgdGhlIGZpbGUgY29udGVudHMubXNvZnR3YXJlQWdlbnShZG5hbWVmQ2xhdWRlcmFsbEFjdGlvbnNJbmNsdWRlZPUAAADIanVtYgAAAEBqdW1kY2JvcgARABCAAACqADibcRNjMnBhLmhhc2guZGF0YQAAAAAYYzJzaMzYG4nXew6oKhxzni80KsAAAACAY2JvcqVjYWxnZnNoYTI1NmNwYWRMAAAAAAAAAAAAAAAAZGhhc2hYIJ4ofGM8HqUX0hGSVAs9gd2hBQHmawYyTSYRp4hrKIRZZG5hbWVuanVtYmYgbWFuaWZlc3RqZXhjbHVzaW9uc4GiZXN0YXJ0GQF8Zmxlbmd0aBkeBAAAAj5qdW1iAAAAJ2p1bWRjMmNsABEAEIAAAKoAOJtxA2MycGEuY2xhaW0udjIAAAACD2Nib3KlY2FsZ2ZzaGEyNTZpc2lnbmF0dXJleE1zZWxmI2p1bWJmPS9jMnBhL3VybjpjMnBhOjIyZmM2OTRmLTY1ODUtNGI4OC1iMzUyLWI3NTg0YWYxZmIwNC9jMnBhLnNpZ25hdHVyZWppbnN0YW5jZUlEeCx4bXA6aWlkOjdlY2Q0ODRkLWRmNjItNDQzNS04N2VlLWQzOTZhYzdhNGI4Y3JjcmVhdGVkX2Fzc2VydGlvbnODomN1cmx4LXNlbGYjanVtYmY9YzJwYS5hc3NlcnRpb25zL2MycGEuaW5ncmVkaWVudC52M2RoYXNoWCAWTp3WBHhHeZd8SUV2/dNjByVt8Zqb3+ZIvigWnoL056JjdXJseCpzZWxmI2p1bWJmPWMycGEuYXNzZXJ0aW9ucy9jMnBhLmFjdGlvbnMudjJkaGFzaFgg7M0Yp64LS+hytpsAifNgAaY6Ydg/qrUZrdSeYi1wIueiY3VybHgpc2VsZiNqdW1iZj1jMnBhLmFzc2VydGlvbnMvYzJwYS5oYXNoLmRhdGFkaGFzaFggdk5GLsdw1BTbE7zXjJaWdeU8I1utGJvlZ000yNcpZ7p0Y2xhaW1fZ2VuZXJhdG9yX2luZm+jZG5hbWVvQW50aHJvcGljIEZpbGVzZ3ZlcnNpb25lMS4wLjBrc3BlY1ZlcnNpb25lMi40LjAAABA4anVtYgAAAChqdW1kYzJjcwARABCAAACqADibcQNjMnBhLnNpZ25hdHVyZQAAABAIY2JvctKEWQISogEmGCFZAgowggIGMIIBjaADAgECAhRA5aAK7sI50L64g/oGQgU9Z1UTADAKBggqhkjOPQQDAzBJMRcwFQYDVQQKEw5BbnRocm9waWMsIFBCQzEuMCwGA1UEAxMlQW50aHJvcGljIENvbnRlbnQgQ3JlZGVudGlhbHMgUm9vdCBDQTAeFw0yNjA4MDcxODQzNTZaFw0yODA4MDYxOTQzNTZaMEQxFzAVBgNVBAoTDkFudGhyb3BpYywgUEJDMSkwJwYDVQQDEyBBbnRocm9waWMgQ2xhdWRlIENvbnRlbnQgU2lnbmluZzBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABJh6CmvLUBgFFNU0vUKlOVtE6djd17L5SuwX0LemFisBM3dkd/3cyjxFA3Qo5S46fX0/ihY0VZ7mfb9KF703t5OjWDBWMA4GA1UdDwEB/wQEAwIHgDAVBgNVHSUEDjAMBgorBgEEAYPoXgIBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzlHiBIFOZFsj+OPEz5o+nMHXXMIwCgYIKoZIzj0EAwMDZwAwZAIwMXMdFJ4BetLLVY7ORuE9noqbbAZOZn/aArXyTwFAZfKrPzxF2vPoJNf1+UCdg1XGAjBwX1zd9WGqYkqmL5SFqw1QySjr1zJfpJM9+1rdDwSPLMOPOjKuiXjoU/pUUeG9RwmhY3BhZFkNngAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPZYQETE/InvjgURzF+J1B/NqojrWcCRe/JYzyzeKiOF+iqqHi5EsrIMfOezFLuoCJ3UQFxTxIMz/2FRrGTzSGnRwyw= 1score the model atmany forecast dates50 to 140 per commodity 2choose on dev,verify on holdoutdevholdoutgap = one horizon 3intervals byblock bootstrapblock = one horizon 4report per regime,always vs a baselinecalmshockcalm vs shock
The protocol as applied to every number in the benchmark, not just the one that was first corrected.

Vector Institute, Agentic Forecasting Bootcamp 2026

Multi-agent price forecasting

A multi-agent system producing probabilistic commodity price forecasts from news. It ranked first of fourteen methods on mean CRPS across horizons of one to thirteen weeks, with a skill score of +0.27 over the naive baseline.

The comparison set included ETS, AutoARIMA, Kalman filters, LightGBM and Prophet, and coverage came in at 89% against an 80% target.

Cohort teams came from TD, BMO and Telus

  • Forecasts are probabilistic rather than point estimates, which is why the comparison is on CRPS.
  • Coverage of 89% against an 80% nominal target means the intervals were conservative rather than overconfident.
  • Built during the Vector Institute Agentic Forecasting Bootcamp, 2026.

Personal project, built solo

Kaivest

A personal finance platform I built alone and use daily with my own bank data. Next.js and React Native clients over one REST API, Supabase Postgres with row-level security, and Claude through the Vercel AI SDK with about sixty tools routed by query complexity.

Roughly 1,200 unit tests and 49 end-to-end tests gate every merge, and an evaluation harness benchmarks candidate models across the full tool surface to set the production routing.

kaivest.ai, invite-only beta

  • Roughly 57,000 lines of TypeScript across web and mobile, built AI-assisted with Claude Code.
  • About sixty tools across fourteen modules, with automatic model routing by query complexity.
  • Supabase Postgres with row-level security, Stripe billing, Resend email, Sentry across client, server and edge.
  • Users manage expenses, income, budgets, savings goals, credit cards, debt and investments through conversation.
02

Other work at Horizon3 AI Labs

Optimizer ownership

Took over three production optimizers from an outgoing vendor

When the vendor that built the lab's optimization tooling rolled off, I completed the knowledge transfer for three of their solutions: the downstream multi-hop distribution optimizer, a general-purpose optimizer for a second supply chain, and a single-hop optimizer for liquid oil materials. Beyond the handover sessions, I read the codebases, ran each tool locally, wrote up my understanding for the team, and have been the point of contact when the business hits an issue.

  • The downstream optimizer is the one the DSO agent and the infeasibility analysis were later built on, so the takeover was the foundation for both.
  • Documentation lives in the team's internal knowledge base and was written to survive my own absence, not just the vendor's.
  • Ongoing support means diagnosing issues in code I did not originally write, under planning deadlines.
Research agent

Retrieval, ranking and evaluation over arXiv, Semantic Scholar and GitHub

I designed the retrieval, ranking and evaluation architecture: source descriptions chunked and embedded into a Chroma vector store, vector search and relevance ranking on top, and a human-in-the-loop step where users pick sources from LLM-generated relevancy reports. I built the benchmarking framework that evaluates what the agent returns, then handed the code-generation component to a colleague to carry forward.

AAAWgmp1bWIAAAAeanVtZGMycGEAEQAQgAAAqgA4m3EDYzJwYQAAABZcanVtYgAAAEdqdW1kYzJtYQARABCAAACqADibcQN1cm46YzJwYTpmNjRmOWIwZi1hMDgyLTQ4NzEtYjQ0NC02Nzk3NDJlOWU2NmIAAAADl2p1bWIAAAApanVtZGMyYXMAEQAQgAAAqgA4m3EDYzJwYS5hc3NlcnRpb25zAAAAALxqdW1iAAAARGp1bWRjYm9yABEAEIAAAKoAOJtxE2MycGEuaW5ncmVkaWVudC52MwAAAAAYYzJzaIcQW/8tis4ooxOCUOfDi0kAAABwY2JvcqNpZGM6Zm9ybWF0bWltYWdlL3N2Zyt4bWxqaW5zdGFuY2VJRHgseG1wOmlpZDo4OTg5YTMxZi05M2ZlLTQwNzYtOWIzZC01YzgxNzdlN2E0ZGZscmVsYXRpb25zaGlwaHBhcmVudE9mAAAB4mp1bWIAAABBanVtZGNib3IAEQAQgAAAqgA4m3ETYzJwYS5hY3Rpb25zLnYyAAAAABhjMnNoM+ul6/zaI4foQl8ojLbcawAAAZljYm9yomdhY3Rpb25zgqJmYWN0aW9ua2MycGEub3BlbmVkanBhcmFtZXRlcnOha2luZ3JlZGllbnRzgaJjdXJseC1zZWxmI2p1bWJmPWMycGEuYXNzZXJ0aW9ucy9jMnBhLmluZ3JlZGllbnQudjNkaGFzaFgg9PlsBtONMxjWnHhlIQx9s9fUY1YYFWOflsXSoJgen1KkZmFjdGlvbngdY29tLmFudGhyb3BpYy5jbGF1ZGUucHJvdmlkZWRqcGFyYW1ldGVyc6F4H2NvbS5hbnRocm9waWMub3JpZ2luLWNvbmZpZGVuY2VndW5rbm93bmtkZXNjcmlwdGlvbnhmQ2xhdWRlIHByb3ZpZGVkIHRoaXMgZmlsZSBhdCB0aGUgcmVxdWVzdCBvZiBhIHVzZXIgYW5kIG1heSBoYXZlIGNyZWF0ZWQgb3IgbW9kaWZpZWQgdGhlIGZpbGUgY29udGVudHMubXNvZnR3YXJlQWdlbnShZG5hbWVmQ2xhdWRlcmFsbEFjdGlvbnNJbmNsdWRlZPUAAADIanVtYgAAAEBqdW1kY2JvcgARABCAAACqADibcRNjMnBhLmhhc2guZGF0YQAAAAAYYzJzaHEU3MCAwMbtwGEk2RJrMn0AAACAY2JvcqVjYWxnZnNoYTI1NmNwYWRMAAAAAAAAAAAAAAAAZGhhc2hYIHJFbO/aNxYzlKaygFHTVF8rr6ORN+53Npenn0C/2wjGZG5hbWVuanVtYmYgbWFuaWZlc3RqZXhjbHVzaW9uc4GiZXN0YXJ0GQGdZmxlbmd0aBkeBAAAAj5qdW1iAAAAJ2p1bWRjMmNsABEAEIAAAKoAOJtxA2MycGEuY2xhaW0udjIAAAACD2Nib3KlY2FsZ2ZzaGEyNTZpc2lnbmF0dXJleE1zZWxmI2p1bWJmPS9jMnBhL3VybjpjMnBhOmY2NGY5YjBmLWEwODItNDg3MS1iNDQ0LTY3OTc0MmU5ZTY2Yi9jMnBhLnNpZ25hdHVyZWppbnN0YW5jZUlEeCx4bXA6aWlkOjZiZTBlZjA3LTg4NjYtNDRhYS1iMTFlLTcyNTY3OTk3ZGIxNXJjcmVhdGVkX2Fzc2VydGlvbnODomN1cmx4LXNlbGYjanVtYmY9YzJwYS5hc3NlcnRpb25zL2MycGEuaW5ncmVkaWVudC52M2RoYXNoWCD0+WwG040zGNaceGUhDH2z19RjVhgVY5+WxdKgmB6fUqJjdXJseCpzZWxmI2p1bWJmPWMycGEuYXNzZXJ0aW9ucy9jMnBhLmFjdGlvbnMudjJkaGFzaFggtgRT3HM+rZxe6VhX1B+UAt2Tmz7KMFkEXmsofCPuSyuiY3VybHgpc2VsZiNqdW1iZj1jMnBhLmFzc2VydGlvbnMvYzJwYS5oYXNoLmRhdGFkaGFzaFggwpd5fifxrn0ZAfw9niq//MZWp+LSgm+fugQ4P4otP6B0Y2xhaW1fZ2VuZXJhdG9yX2luZm+jZG5hbWVvQW50aHJvcGljIEZpbGVzZ3ZlcnNpb25lMS4wLjBrc3BlY1ZlcnNpb25lMi40LjAAABA4anVtYgAAAChqdW1kYzJjcwARABCAAACqADibcQNjMnBhLnNpZ25hdHVyZQAAABAIY2JvctKEWQISogEmGCFZAgowggIGMIIBjaADAgECAhRA5aAK7sI50L64g/oGQgU9Z1UTADAKBggqhkjOPQQDAzBJMRcwFQYDVQQKEw5BbnRocm9waWMsIFBCQzEuMCwGA1UEAxMlQW50aHJvcGljIENvbnRlbnQgQ3JlZGVudGlhbHMgUm9vdCBDQTAeFw0yNjA4MDcxODQzNTZaFw0yODA4MDYxOTQzNTZaMEQxFzAVBgNVBAoTDkFudGhyb3BpYywgUEJDMSkwJwYDVQQDEyBBbnRocm9waWMgQ2xhdWRlIENvbnRlbnQgU2lnbmluZzBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABJh6CmvLUBgFFNU0vUKlOVtE6djd17L5SuwX0LemFisBM3dkd/3cyjxFA3Qo5S46fX0/ihY0VZ7mfb9KF703t5OjWDBWMA4GA1UdDwEB/wQEAwIHgDAVBgNVHSUEDjAMBgorBgEEAYPoXgIBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzlHiBIFOZFsj+OPEz5o+nMHXXMIwCgYIKoZIzj0EAwMDZwAwZAIwMXMdFJ4BetLLVY7ORuE9noqbbAZOZn/aArXyTwFAZfKrPzxF2vPoJNf1+UCdg1XGAjBwX1zd9WGqYkqmL5SFqw1QySjr1zJfpJM9+1rdDwSPLMOPOjKuiXjoU/pUUeG9RwmhY3BhZFkNngAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPZYQOrsvvrgVmjTOCZZck/Vd6ctpW6WxKwyNEdJmQPMRGkzsoTGxeVFcZDgEKMoX85pWPX2J0R8EUa+4bt93ob6WSE= Query Intent frame Per-source search Rank: judge + vector stage of the data-science workflow, topic, sources User selects Implement method Benchmark on time series human in the loop: relevancy report, user picks scores accumulate across runs, so the comparison builds over time
The benchmark sits after implementation. Its precision, recall and F1 describe the implemented method on standardized datasets, not the retrieval.
  • The loop closes at implementation: retrieved methods are actually built and then benchmarked on standardized datasets, which is how you find out whether the agent's selections fit the task.
  • Benchmarks run on fair datasets from a configurable generator I built, so competing methods are measured on the same footing.
  • Precision, recall and F1 here describe the implemented method's task performance, not the quality of retrieval.
Price forecasting

Commodity price forecasting for buyers

Six-month horizon models with exogenous market drivers, reaching about 9% test MAPE across six product grades. I gathered the requirements directly from buyers and presented the results back to them alongside product owners and technical stakeholders.

grade 1grade 2grade 3grade 4grade 5grade 6
Schematic only. Six product grades, each forecast six months out from its cutoff with an interval. Values are not shown.
  • Each of the six product grades was modelled and evaluated separately rather than pooled into one series.
  • The work ran end to end, from requirements with the buying team through to presenting results to a mixed business and technical audience.
03

Notes

September 2026

Backtesting a foundation model is not the same as backtesting ARIMA

A classical forecaster is fit on your data and evaluated on the part you held out. A time-series foundation model was pretrained on a large public corpus before you arrived, so a cutoff date in your data says nothing about what the model has already seen. If the model's release date falls after your test window, some of that window may be in its pretraining set, and your backtest is measuring recall rather than forecasting. The first check I run now is release date against test window, in both directions, before any error is computed.

It would be nicer to test memorization directly, and it mostly does not work. In a controlled model trained on synthetic series, members and non-members only become distinguishable after three to five passes over the data, and Chronos sees each training window roughly once, so a static membership test on the released model comes back near chance. That is why the release-date check is the practical screen rather than a naive one: it is the check that still works when the model has not memorized anything.

Two more things follow from the same logic. Development and holdout windows that touch each other leak through autocorrelation, so I gap them. And confidence intervals from a plain bootstrap assume independent errors, which forecast errors are not; block bootstrap keeps the dependence structure and gives intervals you can actually defend.

On a commodity benchmark of eight model families, the protocol above moved a reported 9% improvement to roughly 21% and, more usefully, split it into 8% in calm markets and 41% in shocks. The second number is the one that decides where the model belongs in a production stack, and the pooled figure hid it entirely.

September 2026

Pass rate is the wrong number for an agent

A conventional test either passes or fails. An LLM agent run five times on the same input can pass three and fail two, and a pass rate of 60% on that case tells you almost nothing about whether a user will get the right answer. The score I use instead is pass^k: the probability that all k runs pass. It punishes flakiness, which is the failure mode that actually reaches customers, and it makes a stable pass and a lucky pass look different on a dashboard.

The second problem is that a regression suite only covers what its authors thought to ask. On a retail-facing agent, replaying real production prompts through a fully traced version of the system surfaced a 25% failure rate on questions the suite had never contained, and pointed at a model-migration break in the tool-calling layer. The suite was not wrong; it was measuring the team's imagination, and production was measuring the users'. I now treat replay of recent traffic as part of the suite, not a separate exercise.

04

Talks and teaching

Vector Institute Learn Day, July 2026

Benchmarking time-series foundation models with leakage-aware evaluation

Co-presented results from the eight-family commodity benchmark, including the evaluation protocol and the regime-stratified findings above.

Toronto Metropolitan University

Course instructor and mentor

I taught Signals and Systems and Random Processes at the undergraduate and graduate levels, and received a Dean's Teaching Award. I currently mentor two University of Toronto MScAC interns on model evaluation and experimental design.

05

Experience

Jan 2026 to present

Data Scientist, Horizon3 AI Labs, Unilever

Toronto. Agentic systems for supply chain decisions, ownership and support of three production optimizers, time-series foundation model benchmarking, evaluation and observability for production LLM agents, and planning input on future optimization work. Mentor two University of Toronto MScAC interns.

Jun to Dec 2025

Independent projects and study

Between the PhD and the first industry role, I built a set of applied projects to move from research code to production practice: a sales forecasting system, a fraud classifier, and two multi-agent LLM applications. Alongside them I completed the DeepLearning.AI Machine Learning in Production certification and the HackerRank SQL certification.

  • Sales forecasting. Weekly forecasts for 970 material and customer pairs over a 10-week horizon, from about three years of history in which 52% of the target values are zero. Recursive multi-step prediction with engineered features and post-hoc calibration reached a WMAPE of 0.38, a 32% improvement over the naive baseline.
  • Fraud detection. A LightGBM classifier for fintech transaction fraud, evaluated under cross-validation: ROC-AUC 0.91 and PR-AUC 0.52, with feature-importance analysis to explain what the model relies on.
  • Multi-agent blog generator. Built on the OpenAI Agents SDK; separate agents research a topic, write the draft, generate images and optimize the post for search.
  • Summarization battle. Several local models served through Ollama summarize the same fetched web articles, and an OpenAI model judges them on coverage, faithfulness, clarity and conciseness.
Sep 2020 to May 2025

PhD Researcher, Toronto Metropolitan University

PhD in Electrical and Computer Engineering, 2025, focused on machine learning and causal discovery for time series. Course instructor for Signals and Systems and Random Processes, Dean's Teaching Award.

Sep to Dec 2023

AI Research Intern, Vector Institute

Implemented and benchmarked HMC, SVGD, MALA and ULA sampling for deep generative models on Vector's GPU clusters, with Monte Carlo diagnostics and posterior-analysis workflows documented for handoff.

Mar 2019 to Nov 2020

Signal Processing Engineer, Oxymap

Stochastic noise modelling and statistical filtering for physiological signal estimation, benchmarked across firmware iterations in a cross-functional production setting.

Sep 2017 to Aug 2020

Research Assistant, University of Tehran

Supervised time-series classification on noisy biomedical signals, with PCA and ICA feature extraction and permutation-tested cross-validation. MSc in Electrical Engineering, 2020. BSc, 2016.

Certifications

Machine Learning in Production (MLOps), DeepLearning.AI, 2025. SQL, HackerRank, 2025.

06

Stack

Agents and LLMs

LangGraph, LangChain, OpenAI Agents SDK, Gemini function calling, FastAPI, Chroma, LoRA fine-tuning (TimesFM, Chronos), LLM-as-judge evaluation, regression suites with pass^k scoring, per-request tracing on an internal platform comparable to LangSmith or Langfuse

Time series

State-space models, ARIMA, ETS, VAR, variational and Bayesian methods, causal discovery, LightGBM, XGBoost, Chronos, TimesFM, Moirai and other foundation model families, leakage-aware evaluation protocols

ML and data

Python, PyTorch, scikit-learn, Hugging Face Transformers, TensorFlow, Pandas, NumPy, SQL

Infrastructure

Docker, GCP, SLURM GPU clusters, Azure DevOps repositories, Git, Streamlit. TypeScript, Next.js, React Native and Supabase through Kaivest.

Familiar with

LangSmith, Langfuse, OpenAI Agents SDK tracing

07

Papers