Saheb Motiani

Side Projects

• Orbit Wars RL Agent: a self-play RL agent for the Orbit Wars Kaggle competition, a multiplayer planet-capture strategy game. (View)
  • Built and trained an entity-transformer reinforcement learning agent using PPO self-play on a custom GPU-vectorised PyTorch environment (~4k env steps/sec on H100), trained from scratch with no imitation warm-start.
  • Built the full training stack: opponent league with prioritised fictitious self-play (PFSP), KL-anchored self-play to prevent policy collapse, action masking, reward shaping, and stratified 256-game evaluation panels against calibrated leaderboard opponents.
  • Ran one-change-per-run controlled experiments across GCP/JarvisLabs L4/H100 spot instances, profiled the loop end-to-end (PPO-compute-bound, not env-bound), and diagnosed self-play pathologies such as unanchored policy drift; from-scratch self-play reached 98% head-to-head against ~1200-rated planner bots over 100M steps.
• Snake RL: where the RL journey started: climbing the RL stack on Snake, ending in a mechanistic study of board-size generalization; one agent trained only on small boards plays 100×100 zero-shot. (View)
  • Built a size-invariant agent (egocentric head-relative representation + attention pooling) trained on a curriculum of small boards {10, 20} that navigates 100×100 grids zero-shot at 98% of moves toward the goal, a 4.5× extrapolation with no degradation, where single-size agents are no better than random by board 16.
  • Diagnosed the failure with linear probes: the direction-to-food signal was decodable but unused (its magnitude collapsed), and probe-validated architecture fixes made real-play transfer worse. The true bottleneck was state-distribution coverage, not architecture: 71–88% of big-board states are out-of-support for a small-board policy.
  • Earlier phases climbed the RL stack from scratch (tabular/double Q-learning, REINFORCE, PPO, curriculum + imitation learning, and an ICM curiosity ablation that helped PPO under sparse reward but not DQN), and showed a from-pixels ConvNet beating a 24-feature hand-engineered DQN.
• tastebento.com: a social discovery app with a production LLM pipeline generating grounded "Did You Know?" trivia from users' taste profiles. (View)
  • Designed a layered reward pipeline for LLM outputs: verifiable hard gates (claim grounding against retrieved context, non-triviality vs user-visible fields, verbatim quiz answers) with soft quality scores and a bounded LLM-judge tiebreak, so fluency can never outvote factuality. Grading production outputs reframed the problem: 96% passed gates while 84% lacked specificity, shifting optimization from hallucination to blandness.
  • Shipped best-of-N rejection sampling over Llama-3.3-70B (parallel high-temperature candidates, grader+judge pooled selection, near-duplicate suppression); every production generation logs (context, candidates, rewards) tuples, accumulating GRPO-ready training data as a side effect.
  • Built an offline fact-mining worker: a free 8B model scans entire Wikipedia articles in the background and distills trivia nuggets, each programmatically verified against its source passage (extract-then-verify), moving read-time compute offline so serving prompts carry distilled depth for the whole catalog within a fixed token budget.
• Poker RL: replicating poker research to build state of the art poker agents (View)
  • Built CFR and MCCFR blueprint solvers for Leduc and No-Limit Hold'em, training 2B+ iterations against Slumbot with vectorized Cython extensions achieving 180k it/s throughput.
  • Implemented Single Deep CFR with a custom RangeEncoder architecture (1326→64 bottleneck) for range-conditional postflop play, integrating reach-weighted traversal sampling and composition-based synthetic range injection.
• Content Moderation OpenEnv: a deterministic environment for evaluating AI agents on user-generated content moderation, built for the Meta PyTorch OpenEnv Hackathon. (View | Demo)
  • Simulates the sequential trust & safety review workflow (read a post, gather thread context, apply a policy, issue a decision) across four graded task packs, with deterministic graders and dense reward shaping: over-censorship penalties, policy-citation bonuses, and deductions for skipping context on context-critical posts.
  • Benchmarked frontier LLMs on it: models converge on accuracy (~0.82 on the hard pack) but diverge widely on shaped reward (−0.93 to +6.39), separating genuine policy reasoning from correct guessing; live demo and benchmark hosted on Hugging Face Spaces.

Experience

Skills

• ML: Python, PyTorch, JAX, NumPy
• Data & infrastructure: Kafka, Flink, AWS, GCP, Kubernetes, SQL
• Languages & web: Scala, Java, TypeScript, React, GraphQL

Education

• Finding Hidden Messages in DNA (Bioinformatics I) by University of California San Diego (Jan 2026)
• Neural Networks and Deep Learning by DeepLearning.AI (May 2023)