What is media mix modeling actually good for?

Media mix modeling is a regression of an outcome you care about, usually sales or signups, on spend by channel over time, with the other things that move the outcome held alongside: seasonality, price, promotions, and whatever else matters in your business. It answers a budget question: if I move a dollar from here to there, what happens?

It answers at the level of channels and weeks. It knows nothing about clicks or users. That is its limitation and also the entire point.

The problem it solves

Click attribution credits whoever was standing closest to the sale. That is almost always search, because search is where people go when they have already decided. The channels that made them decide, the video they half-watched on YouTube, the TikTok they didn't click, the billboard on the drive to work, get nothing, because there was nothing to click.

So a click-based view of a media plan will tell you, every time, to put more money into search. Follow that advice long enough and you are paying to catch demand you no longer create.

MMM doesn't need a click. It looks at what happened to signups in the weeks you spent on video versus the weeks you didn't, controlling for everything else that was going on. If video moves the number, the model sees it, click or no click.

What it found at a crypto exchange

I built the first one of these for a large crypto exchange, before I had a name for what I was doing. I only learned later, back at Omnicom, that the thing had a name and a literature. Two things made that first model unusual.

Volatility was a variable. In crypto, the market is the biggest driver of signups, bigger than any campaign. Days when Bitcoin moved sharply, or when an altcoin ran, brought people to the exchange whether we advertised or not. If you leave that out of the model, every campaign that happened to run during a rally looks like a genius. So the model carried the price movement of Bitcoin and the major altcoins alongside the media spend. Once it did, two things fell out: a fair read on each channel, and a map of when advertising worked best, which turned out to be tied to what the market was doing.

Reddit was doing more than Reddit could see. This was the period of WallStreetBets and GameStop, when a great deal of retail interest in trading was forming on Reddit. Reddit's own ad platform was, at the time, not sophisticated. Its reporting showed modest results. But every week we spent on Reddit, the model showed a significant rise in new registrations that nothing else explained. The platform couldn't attribute what it was causing. The model could, and it changed how the budget was split. It makes complete sense in hindsight: that is where the people were. But no dashboard would have told you to spend there.

What it found at a cruise line

Later, at Omnicom, I built one for a cruise line. The media plan was heavy on channels people don't click: video on YouTube and TikTok, connected TV, audio. Until then the team had been reading Google Analytics and click-based attribution, which reported that search drove nearly everything. That is what attribution reports when the plan is built around channels that create demand instead of catching it.

The model, with seasonality included, flipped the script. For the first time the channel numbers looked like what the team expected from experience: the demand-creating channels carried weight that the click view had assigned to search. Not "did anyone click" but "did the weeks with video spend produce more bookings than the weeks without, after accounting for the season and the price." That is the question a brand needs answered before it cuts a channel, and attribution cannot answer it.

Two things only a model can show you

Halo. Spend on one channel lifts another. Brand video makes search cheaper the following week because more people search for you by name. Attribution gives search the credit. A model can separate the two.

Decay. An ad's effect doesn't stop when the campaign does. Interest fades over days or weeks. Modeling that decay (the term is adstock) tells you how long a burst of spend keeps working, which changes how you pace a budget.

What it is bad at

• Small budgets with little variation. If spend never changes, the model has nothing to learn from. It needs weeks that differ.
• Short histories. Two years of weekly data is a reasonable floor. Less than that and the uncertainty swallows the answer.
• Decisions inside a channel. Creative, audiences, bids. The model sees "YouTube," not "which YouTube ad."
• Small channels. The model is confident about your biggest line and vague about everything else.

The version that fits a small business

Not a model. A discipline. Log spend by channel weekly. Log the outcome you care about. Change one thing at a time. After six months, look at the two lines together. That is a media mix model with the statistics removed and the honesty kept. When the budget and the history are big enough, the statistics come back.

Methods and assumptions

Model form. Outcome regressed on channel spend with adstock (carryover) and saturation (diminishing returns) transforms, plus controls. For the exchange, controls included daily price movement of Bitcoin and major altcoins, day of week, and known promotions. For the cruise line, seasonality, pricing, and sailing schedule.

How to check it isn't fitting noise. Hold out the last several weeks, fit on the rest, and see whether the model predicts them. Compare its channel estimates against any experiments you have. A model that disagrees with a clean holdout test is wrong, not the test.

What to be skeptical of. Any channel estimate with a wide interval, any result that flips when you change the adstock assumption, and any model whose recommendations look exactly like what the person who commissioned it wanted.