BluffBot
Developed iterative Bounty Hold'em agents culminating in a neural policy that uses card/bet embeddings and action masking; includes a local engine to pit bots against each other.
Context
Developed as part of MIT Pokerbots (6.176) competition course, this project explores how simple heuristics can be scaled into stronger agents through opponent modeling, simulation, and learned policies.
The Problem
Heuristic bots struggled to balance bounty-adjusted EV, bluffing, and dynamic bet sizing across streets, leading to brittle strategies and inconsistent results.
Approach
Shipped bots in weekly tiers: baseline → opponent profiling and pot-odds/EV → Monte Carlo strength and board texture → Deep CFR–inspired neural policy selecting masked actions. Used a socket-driven engine to automate head-to-head evaluations.
Architecture
Python engine (`engine.py`) orchestrating two subprocess bots via a simple socket protocol and `commands.json`. Each bot implements `player.py` over a shared `skeleton/` (actions, states, runner). Final bot (`w4_bot_cristi`) loads a PyTorch `DeepCFRModel` that embeds hole/board cards and recent bet sizes to score actions; training utilities live in `training/` and `training_cristi/`.
Results
Demonstrated clear strength gains across iterations (w1 → w2 → w3 → w4) in local 1000-round matches
Neural policy with legality masking eliminated invalid raises and stabilized postflop decisions
Modular framework enabled rapid A/B testing of strategies and ablations
Produced reproducible logs and head-to-head comparisons via the local engine
What I Learned
Action masking and discrete raise sets simplify policy learning; Monte Carlo strength and simple texture signals provide strong baselines; opponent-type heuristics are helpful but sensitive; socket orchestration and reproducible evaluation loops are critical for fast iteration.
Impact Summary
Built a progression of heads-up poker bots from heuristics to a Deep CFR–guided policy; benchmarked across 1000-round matches with a custom engine and bounty mechanics.