Why qualitative research gets skipped in CPG innovation

A former CPG strategist on the cost model that rationed qual to the big bets, and what changed when a study took days instead of weeks

In most CPG innovation processes, qualitative research is treated as a large undertaking: weeks of fieldwork, an agency budget, a slot in the insights team's queue. So it gets reserved for the biggest bets of the year, and everything else moves forward on sales data, a competitive scan and a deck. I spent my career in CPG at Blue Bottle Coffee, Bimbo Bakeries USA and Constellation Brands working that way, then ran research myself at Graza and saw how different it could be.

The consumer usually entered the process after the decision

Here is how innovation typically moved when I was in strategy in CPG companies. We built the case for a new product or line extension from syndicated sales data, category trend reports, retailer feedback and competitive benchmarking. The output was a deck. Somewhere in it was a slide labeled "consumer need," and more often than I'd like to admit it drew on a survey fielded for another purpose a year or two earlier.

Qual was in the plan but it wasn't a priority. It showed up as concept testing after the idea was locked as a nice to have. By then the research question had become "will this work" rather than "what should we build." Both are legitimate questions. The second one was the one we rarely got to ask.

Qual got rationed because it was expensive and slow, not because anyone doubted it

At Bimbo and Constellation I had a full insights team I could go to for consumer data. That was a privilege, and it also meant I was far enough from the research that I never touched the tools or worked directly with the agencies. What I saw was the queue. A qual request meant a brief, a budget line, an agency, recruiting, fieldwork, and a readout weeks later. By the time it came back, the strategy project I was working on had already been read out to leadership and insights were "follow on work".

Nobody on those teams thought talking to consumers was a waste of time. The insights professionals who I worked with were some of the sharpest in the building. But when something costs that much and takes that long, you ration it, and rationing has a logic. The flagship launch got focus groups. The line extension, the packaging refresh, the regional test and the flavor rotation got the deck.

The initiatives that skipped qual weren't small. In fact, they were the ones that made up most of the innovation calendar. They just weren't the "big bets".

The launches that skipped qual carried more risk than the model showed

New products fail more often than most innovation plans assume. A 2021 study in Marketing Letters tracked 83,719 new CPG SKUs across 31 US categories and found that one in four were no longer being bought a year after launch, rising to roughly 40 percent after two years. Failure was more likely in higher-revenue categories and for smaller-share parent brands, which describes a lot of line extensions.

The financial models behind those launches were probably fine. What was missing was the part you only get from a conversation: why someone would pick this up, what they'd stop buying to make room for it, and the words they'd use to describe it to a friend. That is exactly the part that gets cut when qual is a luxury.

Meanwhile, the growth in the category has been coming from brands that talk to their consumers constantly. McKinsey's April 2026 State of Food and Beverage report found that small independent brands under $100 million in sales were 13 percent of the US food and beverage market in 2021 and accounted for 35 percent of category growth by 2025. Those brands don't have bigger insights budgets, they're just close to their customers. The founder is answering DMs and someone on the team is standing in the store watching people pick things up.

At an insurgent brand, I finally ran the research myself

At Graza I had no insights team and no agency retainer. I had questions and an innovation deadline. I'd been talking to Priya Krishnan, Strella's co-founder, for about six months about what she was building, and it took an urgent need on our side to finally run the first project. If you've worked at a growing brand, you know the kind of urgent I mean.

The study came back in days. It answered the question we had, and it surfaced things we hadn't thought to ask about. What struck me most was not the technology, though the AI moderator was better than I expected. It was that the research happened at all. Under the model I'd worked in for years, that study would have been priced out before anyone wrote a brief.

That project changed how I think about research. A few months later I joined the Strella team.

What I'd have done differently with a day instead of six weeks

I've thought a lot about what those years in strategy would have looked like if a round of 20 to 30 interviews with real consumers took a day or two instead of six weeks. Not what the insights team should have done differently. What I would have done.

I would have talked to consumers during ideation, when the question was still open, instead of after the read out, when the question was really a request for permission.

I would have run a small round on the flavor rotation and the packaging refresh, not just the flagship. Not the same depth, but the same basic step: hear from real consumers before deciding.

I would have run a dozen small studies a year that never got a brief, a budget line or a place in the queue. And the insights team would have spent its time on the studies that need a human moderator and a proper sample design.

And the "consumer need" slide would have been fifty people I talked to last week, with their words on it, instead of a survey from two years ago.

None of that required anyone to work harder or care more. It required the cost of a conversation to drop far enough that rationing stopped making sense. That's the part that has changed.

Frequently asked questions

Why does qualitative research get skipped in CPG innovation?

Traditional qual has been slow and expensive, typically weeks of fieldwork and an agency budget. Teams reserve it for the biggest bets and move everything else forward on sales data and competitive analysis. The constraint is the cost model, not the value of the method.

When does qualitative research usually happen in the product innovation process?

Most often at concept testing, after the idea is locked and the financials are modeled. Discovery-stage qual, when the question is still "what should we build," is less common because it competes for the same limited research budget.

How many interviews does a directional innovation study need?

For a directional read during ideation, 15 to 25 in-depth interviews is usually enough to see patterns and hear the language consumers use. Larger samples matter more for validation than for discovery.

What percentage of new CPG products fail?

A 2021 study in Marketing Letters tracking more than 83,000 new CPG SKUs found 25 percent were no longer being purchased a year after launch, and about 40 percent after two years. The widely repeated "85 percent of launches fail" figure is not supported by that data.

Can a strategy or brand team run qualitative research without an insights team?

Yes. AI-moderated interview platforms let a strategist field and synthesize a round of interviews in a day or two. In practice the strongest setups pair fast studies run by strategy or brand teams with insights teams advising on design and interpretation.

What is the difference between concept testing and discovery research?

Concept testing asks consumers to react to an idea that already exists. Discovery research asks about their habits and unmet needs before an idea is formed. Innovation processes that only do the first tend to confirm what was already decided.