Preference testing is a type of consumer test in which consumers choose a product and indicate their most liked product from the two products. These tests seem easy, but there are several complications in the method. They are performed at the end of the product reformulation cycle and are followed by market research. Sensory tests are conducted with non-branded and coded products, while market research is related to branded products.
It depends on whether consumers like or accept the product over other products. The sensory specialist consumer is not interested in selling the product, branding effect, or cost. The success of a product does not depend solely on the hedonic test. Product success is also affected by price, packaging, and niche. The product fails if it scores low in consumer acceptance despite great marketing.
Have you considered how companies decide which flavor will captivate consumers? It is the relation between product preparation and consumer preferences. Dr. Herbert Stone has conducted a study and provided the role of consumer preference tests in product development to meet consumer demands. Let’s explore how techniques and sensory evaluation shape the product we like.
1. Preference Test Overview
Preferences tests involve the choice of comparing two or several products. Two product comparisons are called paired preference tests. This test is the oldest, simple sensory test and describes what consumers do while buying the product.
Variations in preference testing involve choosing a preferred product from alternative products. Ranking of the product is done in two versions. One version orders the products from best likes to least likes, and the other includes the products from small groups. Then, the consumers are asked which product is best or worst. It is known as the best-worst scaling method. Paired test and best worst are special cases for ranking.
The preference test has other variations, including using no preference option and replication by the same person. The preference provides more data but complicates the analysis. These tests were conducted for illiterate and semi-literate consumers. These consumers can’t write or read and rely on the paired preferences test with paired symbols and paired instructions. One author conducted the test method, and it worked for illiterate consumers.
At the end of sensory tests, temptations to add preferences can cause problems that should be avoided. Asking for preference after a difference test is not recommended for many reasons. Participants in two tests are not selected on the same basis. Participants in the preference test are product users, while participants in the discrimination test are selected to detect differences between samples. In paired preference tests, techniques are designed to answer only one crucial question. Consumers respond to the product and can analyze it based on one or two characteristics. Ranking and choice tests identify the direction of product preferences but do not describe the relative difference between the products.
2. Simple Paired Preference Testing
2.1 Recommended procedures
Panelists receive the two coded samples and identify the preferred sample. The panelist should ensure that consumer participant follows the guidelines on the score sheet. The paired test depends on the two randomized products and equal numbers of panelists. Testing conditions should be clear, including the serving size, temperature, and physical setup. All these instructions should be written so staff clearly understands what to do. Consumers should go through the screening to determine their suitability for the test.
2.2 Statistical Basis
The probability of selecting a specific product is 50% in paired preference methods. Consumers choose the preferred products an equal number of times in the paired preference method. The preference probability is 0.5, which refers to overall population preference, not sample data sample. It is a two-tailed test until we know which product is preferred. Data analysis used binomial, Normal distribution, Z table, and chi-square test. If study results have valid preferences and are due to chance, then binomial distribution is used to calculate the probability.
2.3 Worked Examples
The examples for the work in the paired preferences test involve using 45 consumer panelists and 24 panelists preferred sample A. We find that the table value for 45 consumers is 30 with an alpha criteria of 5%. The value is considerably larger than 24, and panelists didn’t prefer one sample.
2.4 Useful Statistical Approximations
Many sensory panelists use simple look-up tables to find the outcomes of the tests. Tables can only be used for two-tailed tests, not for paired preference tests. If you don’t have tables available, you can use the Z score formula to calculate the probability. The binomial distribution gives the value for large sample sizes near normal distribution.
We will expect the sample distribution mean to be accurate if the sample size is large. The Chi-square will identify the difference between observed frequencies and expected frequencies for the null hypothesis. It can help detect the preferred product in preference tests.
2.5 The Case of Equivalence Testing
It is a statistical method that helps to determine the differences between two products based on consumer preferences. We use this equivalent testing when comparing a new product with the ocean. The central hypothesis states that there is no difference between samples, but the alternative hypothesis describes that there is a difference between two samples. Equivalence testing reverse this approach. The steps in the equivalence testing are :
- Identify the acceptable ranges between the sample products that we consider identical.
- Collect the data from consumer participants and ensure the sample size is enough for analysis.
- Use statistical methods to analyze the data in equivalence margin. We can use two one-sided test procedures.
- If we determine that the test results are equivalent, we reject the null hypothesis and conclude that the products are identical.
3. Non-Forced Preferences
A paired preference test has the option of non-forced preference, but it complicates sensory analysis. There are rising questions about whether it is worth the effort. Legal regulations require it. The size of the preference sample group is helpful in informing. It helps in the case of equal preference split due to indifferent responses whether stable preferences group. We conclude that the 50/50 split in preference tests can fail the product, as it may involve two participants who prefer different products. A nonforced preferences test does not provide a solution for this problem. Avoiding responses with physically identical samples can solve this problem. A non-preference test provides valuable information; response options don’t offer a solution.
3.1 Approaches
There are four options to deal with non-forced preference responses in paired preference testing:
- Eliminate the response
- Apportion them
- Use of confidence interval analysis
- Using a signal detection model analysis
Dealing with responses depends on assumptions about the basis for non-preference choices. In the first approach, the analysis proceeds using two-tailed binomial statistics for proportion difference. This results in a decrease in sample size and the power of the test to identify preferences. If the actual number of responses is low, less than 10%, the approach seems reasonable. If the proportion is high, above 20%, it will still be a significantly reasonable result.
In the second approach, we apportion the responses by splitting them 50/50 into existing groups.
In the third approach, we construct a confidence interval around each proportion. Confidence intervals that don’t overlap mean there is a reasonable preference to win for the product. This approach is significantly better for a sample size larger than 100 and a low number of preferences.
In the fourth approach, signal detection depends on the Thurstonian model, which provides the degree of differences in liked products that exceed the person criterion called tau; below this point, they will choose nonforced preference options. It depends on extending the paired comparison test for differences with equivalent options.
4. Replicated Preference Tests
We do not primarily perform replication with preference tests. However, there are good reasons to use replication in the design of sensory testing. Recruiting costs, efforts, and screening consumer time are costly.
So, it’s better to take additional information from present consumers. According to research by Chapman and Lawless, many people change their preferences from trial to trial. Children had less than 50% consistent preferences, and the adults had higher consistency. They researched and found that 45% switched milk preferences with no preference option. Replication in the process can provide more precise results. A 50/50 split in preferences means equal preference of two samples. Stable samples with preferences improve product development and marketing.
Steps to analyze the replicated preference data:
(i.) Simple Approach:
Calculate the response value in the replication test to identify random behaviors. For example, in a tailed test, 25, 50 consumers chose sample A, and 25 chose sample B. A chi-square identifies the result deviation from chance expectations.
(ii.) 2×2 Contingency Tables:
Create a table with preferences on trial in column 1 and trial in a row. Place each consumer in one of four cells and perform a chi-square test to analyze the table. Binomial tests compare the proportion of consumers’ consistent preferences.
(iii.) Beta Binomial Approach:
Discussion by Birmingham and Cochrane, this method calculates the gamma statistic and provides a significant statistics level that determines the random and consistent behavior of the panel. Gamma statistics describe the presence of stable preference samples.
(iv.) Over-dispersion:
Consider the variations and individual consistency and recognize that the data are related from multiple trials. This allows the integration of no-preference options with replication and offers consumer preferences.
5. Replicated Non-forced Preference Test
Involving replication and no preference responses is the most complicated situation. Bi discusses this as a simple approach for stable preference segments. We use a Dirichlet multinomial model for alternative analysis and a beta-binomial approach for multinomial situations. The DM model considers the heterogeneity of consumer groups. In Ferris k-visit, we conduct multiple tests with the same consumers, and they select the sample without any preference options. It involves the key assumptions:
- Consumers with consistent preferences choose the same product in both trials.
- Double responses for A and B include non-preferencing participants/consumers.
- Some consumers can choose any sample, either A or B, just to please the experimenter. The switching response rate indicates the proportion of false preferences.
The DM model leads to consumer heterogeneity in different samples with consistency in replication. It is a simple way to consider the variability and analyze the preferences.
Analysis steps
- Data collection
- Data tabulation
- Estimation of true perception
- Statistical and testing.
6. Other Methods
6.1 Ranking
The ranking is the step to ranking the products using descending or ascending preferences. We force participants to choose one option, preventing them from ranking the same products equally. When there are two samples for ranking, we call it a paired preference test. We rank products easily and quickly, relying on their internal frame of reference, which makes comparing two products different. Ranking can be easy for tactile and visual analysis but difficult for taste and flavor evaluation due to multiple comparisons.
6.2 Analysis of Ranking Data
We treat ordinal ranked data as nonparametric. The methods to analyze the data are
(i.) Basket tables:
It analyzes ranked data by adding the rank values of all participants in each product and consulting the tables for accuracy.
(ii.) Newell and MacFarlane tables:
We use these tables, similar to Basker tables, to analyze the rank sums and determine significances.
(iii.) Friedman test:
It is a nonparametric test two-way ANOVA without integration and handles a small number of tied tanks.
6.3 Best Worst Scaling
Best-worst scaling is the method of maximum difference. It involves presenting multiple products to consumers, and they will choose the product they like best and the other one they like least. We apply this method in food testing.
- Simple difference scores: Count the proportion of best and worst preferred products, then analyze the difference. The ANOVA and parametric tests analyze these scores.
- Multinominal logistics analysis provides scores and variance data to differentiate the products. The best and worst scaling is easy to implement and can give detailed ratio scale data.
6.4 Rates Degree of Preferences and Other Options
Other methods for preference testing can be rating scales and non-preference options. In the rating scale method, consumers rate the degree of their preferences on a scale. We analyze the data using ANOVA and t-tests to determine strong preferences. Non-preferences and like-or-dislike options provide additional information about consumer preferences.
7. Conclusion
In sensory analysis, preference testing plays a great role in determining the consumer’s demands and preferences. This is how companies decide which taste the consumers will like. Preference testing involves letting the participants describe their preferences between two products. Preferences vary among illiterate and semi-illiterate consumers, but the area location can change them.
Clear testing conditions and guidelines can result in reliable results. Statistical analysis, such as the chi-square test, est, and binomial dist, helps observe preferences. Using statistical approximates and non-preference provides additional information on analysis and complexity.
Each method plays a significant role in sensory evaluation, but choosing what is relevant and appropriate depends on the study’s goals and the nature of the tested product. Statistical analysis methods and advanced combined solutions ensure valuable insights into consumer preferences. These techniques support marketing strategies and product development. Check out more information to sensory science at Grubiie.