Politikos Shop.
How we used Markov Chain + Multi-Armed Bandit models to redistribute budget across channels and campaigns — maximizing expected profit for a flagship fashion retailer.
Last-click attribution was leaving profit on the table
With the account fully automated and scaled, a new question surfaced: where should the next incremental euro go? The default last-click attribution model — the one baked into most ad platforms — credits the final channel before conversion with the entire sale. That single assumption quietly underweights upper-funnel campaigns that assisted the conversion but never closed it, and it over-credits brand and retargeting channels that would have converted anyway.
For an account already running at scale, this is not a rounding error. It translates directly into real budget decisions: campaigns that appear "inefficient" under last-click get defunded, even when they are the discovery engine feeding the entire funnel. Over time, the account converges toward a narrow slice of bottom-funnel activity and stops growing the top of its own pipeline — an invisible ceiling that manual optimization cannot break.
The goal was to replace the platform default with something closer to reality: a probabilistic model that captures the actual causal contribution of each touchpoint, and a decision layer that continuously reallocates budget to where expected profit is highest. Anything less would lock the account into the same last-click blind spots that limited the previous year.
Markov Chain for attribution, Multi-Armed Bandit for allocation
The solution combines two probability-theory models. Markov Chain treats the customer journey as a stochastic sequence of states — impression, click, visit, conversion — and computes the removal effect of each touchpoint. Pull a channel or campaign out of every historical path and measure how much conversion probability drops: that drop is its true causal contribution. The output is a credible attribution weight for every channel and campaign, grounded in observed data rather than platform default heuristics.
On top of attribution sits the allocation problem, which is a classic Multi-Armed Bandit: given a budget to distribute across a set of choices (channels, campaigns) with uncertain payoffs, what allocation maximizes expected profit? The bandit continuously balances exploration — sending a small share of budget to under-tested campaigns to learn their real ROI — with exploitation — pouring the majority into proven winners. Over time it converges on an allocation that beats any static budget split.
Operationally, the two models form a closed loop. Markov Chain runs weekly on the latest path data, producing updated attribution weights. Those weights feed the Multi-Armed Bandit, which adjusts budgets daily across the account structure. Every Monday the account looks slightly different — because every week the data tells a slightly different story about where expected profit actually lives.
From touchpoints to expected profit
Data Pipeline & Path Extraction
Exported full conversion paths from GA4, Google Ads, and Meta into a dedicated data warehouse. Cleaned and normalized touchpoint sequences so channel and campaign paths could be analyzed together without double-counting. Introduced deduplication logic for cross-device journeys using first-party identifiers where available.
Markov Chain Attribution
Built a Markov Chain model that computes the removal effect of each touchpoint — the drop in conversion probability if that channel or campaign disappeared from every path. Result: attribution weights grounded in observed causal contribution rather than last-click bias. Weights refresh weekly against the latest path data.
Multi-Armed Bandit Allocation
Fed the attribution weights into a Multi-Armed Bandit that continuously reallocates budget across channels and campaigns to maximize expected profit. The bandit balances exploration of under-funded opportunities with exploitation of proven winners, adjusting budgets daily and surfacing the rationale to account managers.
The Results
Revenue Growth
Revenue tripled year-over-year, with the attribution-driven reallocation unlocking upper-funnel campaigns that last-click had been starving. The model surfaced patterns invisible to the platform default and turned them into budget decisions.
Transactions
Transaction volume grew 225% — proving the model correctly identified audiences and campaigns that were contributing to conversions but receiving insufficient credit under default attribution logic.
Ad Spend Redistribution
Total ad spend scaled 230%, but the more interesting number is the distribution shift: significantly more budget went to upper-funnel and assist channels than last-click would have recommended, and the returns justified it.
Production ML Models
Markov Chain + Multi-Armed Bandit run in production continuously — updating weights weekly, feeding budget decisions daily. Both models are owned, not licensed, so they evolve with the business rather than a vendor roadmap.
Ready to move past last-click attribution?
Let's build a probabilistic attribution model tuned to your business — and turn it into continuous budget optimization.
Start a conversation