What Questions Should You Ask in a Concept Test?

Most concept tests tell a team which idea won and nothing about why. So the team ends up arguing about what to change before launch.

When Hatch needed to test messaging concepts with a new growth audience, it ran a new round of AI-moderated interviews with real participants on Strella every week: 11 studies and about 221 interviews in ten weeks. The ratings told the team which messages were ahead. The follow-up questions told it why, and that is what shaped the concepts it took to market.

This guide covers the questions to ask at each stage of a concept test, each with the follow-up that gets to the reason behind the answer.

What concept testing is and why it matters before launch

A concept test puts an idea in front of the people who would buy it before anything gets built. They see a description, a mockup or a prototype and react: do they get it, do they want it, does it feel different from what they already use, and would they pay for it. The point is to find out whether a concept has a problem while it is still cheap to fix. Concept tests run as surveys or as interviews. On Strella they run as AI-moderated interviews, a qualitative method where an AI moderator holds a live, voice-to-voice conversation with each real participant and follows up on what they say. Every question below works in either format.

The concept testing questions to ask, by category

Each category below gives the question to ask, the answer scale, and the follow-up that gets to the reason behind the score.

Screening questions to qualify respondents

Screen on recent category behavior, exclude people too close to the industry, and set quotas so each segment that matters has enough people to read. These screener questions cover most consumer concepts:

On Strella, the panel provider’s fraud prevention works underneath, custom screeners set those quotas, screener fact-checking removes people whose verifiable claims turn out false, post-call checks on every interview review completion, response quality, consistency, location, IP and timing, and human review follows.

Comprehension and clarity questions

Check that people understand the concept before measuring whether they like it, because an appeal score from someone who misread the idea measures the misreading. Three questions cover it:

Teams catch problems with the playback question that a clarity rating misses. Playback shows whether people can restate the key term in the concept in their own words, and whether a phrase like “up to” reads as a promise or a ceiling.

Believability belongs in this block because a claim people cannot picture is a claim they do not trust. If fewer people can play the concept back than rate it clear, rewrite the copy before measuring appeal.

Appeal and liking questions

Appeal is the headline score, and the follow-up decides what it means. Use one rating and two open questions:

Two people can give the same “very appealing” for opposite reasons: one for the price, one for the design. Only the follow-up tells the team which benefit to lead with. If appeal is high but the reasons split, test each benefit as its own claim.

Relevance and problem fit questions

Relevance asks whether the concept solves a problem people already have. These questions test fit with the person’s week, not with the brand’s strategy:

In B2B software, the second question does the most work. Asking which tool or workaround the concept would replace, and who would have to approve the switch, shows whether the concept fits a budget line or competes with a habit. If people name no current workaround, treat the need as unproven.

Uniqueness and differentiation questions

Uniqueness measures how new the concept feels against what people already know. Ask it this way:

A team testing a new pricing tier learns more from “what does it remind you of?” than from the rating, because the answer names the competitor already in the person’s head. Uniqueness on its own does not drive purchase. A highly original concept with low relevance scores well here and poorly on intent, so read the two together.

Purchase intent questions

Purchase intent uses the standard 5-point scale, read as top-two-box: the share who chose either of the top two answers. The scale and follow-ups:

Stated intent runs ahead of what people buy, so treat the scale as a way to rank concepts against each other. The forced-choice cart task gives each person one pick, which mirrors the shelf: one option wins and nobody can rate everything a 4. Pairing the cart pick with an avoid pick shows which concepts attract buyers and which ones push them away. If top-two-box is high but believability is low, rewrite the claim before retesting.

Pricing and value questions

Researchers using the Van Westendorp Price Sensitivity Meter ask four price questions and read the acceptable range from where the answer curves cross. This is the wording Quirk’s gives:

1. “At what price would this product be so cheap that you would doubt its quality and not consider it?”
2. “At what price would this product be a bargain, a great buy for the money?”
3. “At what price would this product seem expensive, but you would still consider buying it?”
4. “At what price would this product be too expensive for you to consider?”

Reading the results takes three steps:

• Plot each question as a curve: the share of people who named each price or lower. Flip the “too cheap” and “expensive” curves so all four can be read together.
• The acceptable price range runs between the two outer crossing points (where “too cheap” meets “expensive,” and where “bargain” meets “too expensive”).
• The optimal price point sits where the “too cheap” and “too expensive” curves cross.

Van Westendorp gives a range, not a forecast. Quirk’s notes it has no competitive context and does not estimate demand, so teams often pair it with one of two other methods:

Whichever method sets the numbers, the follow-up is what explains them. After each price question, the AI moderator asks “What would make that price feel fair?” and “What would you compare it to?” Those two answers show what “too expensive” actually means to the person: the price itself, the length of the commitment, or an alternative they already pay for. Each of those calls for a different fix. Ritual used this approach, running its pricing tests as AI-moderated interviews on Strella, and saw a 24% increase in net revenue per user (Ritual case study).

If the decision is the range to test, Van Westendorp with open follow-ups is enough. If the decision is the price on the shelf, add a conjoint study so competitors are in the choice.

Open-ended improvement questions

Close the concept section by asking what would make the idea better. These questions surface fixes no rating can:

• If you could change one thing about this product, what would it be?
• What is missing that would make you more likely to buy it?
• What would you remove?
• If you told a friend about this, what would you say?

These questions also show when a claim splits the room. A claim that half the sample loves and half avoids can land in the middle of the ratings as if nobody cared, and only the open answers show two groups reacting to the same words in opposite ways. If a claim draws strong reactions in both directions, test it with the segment it is meant for before deciding to keep it.

Worked example: how Hatch tested messaging concepts week by week

Hatch, the sleep and wellness device brand, needed to understand an entirely new growth audience and decide which messaging concepts would reach it. Instead of one large study, the team ran a new round of AI-moderated interviews with real participants on Strella every week: 11 studies and about 221 interviews in about ten weeks. Each week’s ratings showed which concepts were ahead, and the follow-up answers explained what the audience heard in each message and what they missed. Amanda Amyx, Senior Director, Design & Research at Hatch, described the effect: “The fact that we were able to have a new data set every week deepened our understanding of an entirely new growth audience and directly informed the messaging concepts we developed.”

Read the full Hatch case study here.

The pattern is the one this guide is built on: the rating ranks the concepts, and the follow-up tells the team what to change before the next round.

The question bank above covers what to ask. The next three sections cover how to ask it: which answer format to use for each question, how to word questions so they do not lead, and how to decide which concepts each participant sees.

Question formats: rating scales vs open-ended

Every question in the bank above is either a rating or an open question, and the choice depends on what the answer has to do. Use a rating when the decision needs a comparison across concepts or against past tests, and an open question when it needs a reason. From a rating, the team learns that one concept lost. The open answer says whether it lost on price, on a claim people did not believe or on a benefit nobody wanted, and each answer leads to a different next step.

Pair them in that order: rating first, open why second. Asking the open question last, after several ratings, means people have to remember what they were thinking when they scored. More example prompts are in Strella’s library of open-ended questions for surveys.

In a survey, the open box after a rating often gets a few words. In an interview, the AI moderator keeps the follow-up going with a real participant until the reason is specific. On Strella, survey-style blocks (multiple choice, ranking, rating scales) sit inside the interview guide, so the score and the AI moderator’s probing happen in one conversation, and the answer to “why a 3?” is attached to the 3.

If the result will be compared with other concepts, ask a rating. If the result will change the concept, ask why.

How to word questions without leading or biasing consumers

The format decides what kind of answer comes back; the wording decides whether that answer can be trusted. Concept tests carry a particular risk here, because the people writing the questions usually wrote the concept too, and their enthusiasm leaks into the wording. A neutral question describes the concept and asks for a reaction without suggesting the answer. Three rewrites cover the most common mistakes:

Order matters as much as wording. Earlier questions can prime later ones: asking about one benefit before the overall rating can lead people to weigh that benefit more heavily. Strella’s guide builder runs a bias check that flags leading questions before a study goes live. The full method is in Strella’s guide to crafting unbiased interview guides. When a bias check passes a guide, still have a second reader review the wording before the study goes live.

Choosing a method: monadic, sequential monadic or comparative

With the questions written, the last design decision is how many concepts each participant sees and in what order. Researchers call this the test design, and it changes which scores can be trusted: a concept rated on its own gets a cleaner score than one rated after three others, but comparing concepts side by side is closer to how people actually choose. There are three standard designs, plus a hybrid:

The question set shifts with the design. Monadic tests use the full rating block for one concept. Sequential tests repeat a shorter block per concept and treat the first-position scores as the clean read. Comparative tests lead with the choice and spend the time on why.

On Strella, concepts can sit in randomized question blocks so the order rotates across participants, and conditional branching routes follow-up questions based on earlier answers, such as which concept a participant picked. If the decision is go or no-go on one concept, run it monadic. If the decision is which of several to develop, run it comparative and ask why.

Interpreting results and sample size

There is no universal, category-by-category top-two-box pass mark, so the useful benchmark is the team’s own history in the category. A common practice is to compare scores with a database of the team’s or agency’s past tests and discount top-box answers against what later sold.  Without that history, compare concepts with each other and with a current product tested the same way. A workable pass rule for a survey read is that the concept beats that current product on top-two-box intent and does not trail it on believability or relevance.

Interview-based tests read differently. Counts describe the people in the study, and themes are leads to verify; the numbers cannot be projected to a market. For how many interviews it takes before new ones stop adding new themes, Hennink and Kaiser’s 2022 systematic review found 9 to 17 interviews were enough when participants were similar and the objectives narrow, with multi-country studies and studies aiming to understand each theme in depth (what the authors call “code meaning” saturation) needing more.

Concept tests need more than that, because they compare concepts as well as collect themes. If a decision depends on a segment, size the study so that segment alone reaches a usable count. If a theme will drive a launch decision, confirm it with a larger or quantitative read first.

How to run a concept test on Strella step by step

Teams run a concept test on Strella in four steps, and Strella starts synthesizing results as the first interviews finish:

1. Build the guide. Describe the objective and Strella drafts a discussion guide, or bring an existing one. Embed the concept images, Figma prototype, ad copy, video or live website directly in the guide. Add survey-style blocks for the ratings, randomize concept order and branch follow-ups by choice. Write the guide to 15 minutes and let the moderator probe toward 25; across Strella’s interview history, sampled in September 2026, the median concept test runs 18 minutes.
2. Recruit. Invite a team’s own customers by direct link, or recruit through Strella’s vetted panel partners, which give access to more than 100 million participants worldwide, and expert networks for hard-to-reach B2B roles. Set quotas from custom screeners.
3. Run. The AI moderator holds voice-to-voice interviews in 40+ languages, and hundreds can run in parallel around the clock. Participants talk at their own pace, with no scheduling and nobody across the table. To hear how the moderator follows up on your answers, try an interview.
4. Synthesize. Open-ended answers are auto-coded and charted by question and objective. Every insight links to the interview it came from, with verbatim quotes and video highlight reels ready to share.

Teams without the bandwidth to run a study can hand it off. With Strella’s fully managed service, Strella’s research team scopes, recruits, runs the AI-moderated interviews and synthesizes the findings, including real-time advertising and messaging tests ahead of a launch or ad buy.

FAQ

What are good questions to ask about a product concept?

Ask them in a set order. Start with a playback question (“In your own words, what is this product?”), then the ratings for appeal, need fit, uniqueness and likelihood to buy, then believability wherever the concept makes a numeric or scientific claim, and finish with what people would change.

What score counts as a pass in a concept test?

A pass is a score above the team’s own benchmark for the category, since there is no universal published threshold.

How do teams test price and willingness to pay in a concept test?

Teams ask the four Van Westendorp price questions (too cheap, a bargain, expensive but still considered, too expensive), and follow each answer by asking what would make the price feel fair. A conjoint study covers the final shelf price.

Can concept tests run as interviews instead of surveys?

Yes, every question in a concept test works as an interview as well as a survey. In an interview, the AI moderator keeps the follow-up going with a real participant until the reason is specific. Survey-style blocks such as multiple choice, ranking and rating scales sit inside the interview guide, so the score and the follow-up happen in one conversation.

How long should a concept test interview be?

A concept test interview works best at 15 to 25 minutes, with the guide written to the low end so the moderator has room to probe. Across Strella’s interview history, the median concept test runs 18 minutes; a single-claim forced-choice test can run shorter.

Bring your research question to us and we will talk it through. See Strella in action.