The robustness of in-store concept testing depends on correctly attributing the change in performance to the specific concept being tested. A methodology often adapted is comparing the performance across two groups of “test” and “control” stores. The new concept is implemented in a set of test stores (Group A). Another set of stores (Group B) is selected as control to match the test stores so that the differences in performance of the two groups can be used to evaluate the impact of new concept.

Since the test/control methodology requires isolating the impact of the new concept, the store matching process is a critical element of its success. Often stores are matched on the basis of historical sales numbers and demographics data.

For example, if testing a new category planogram, the stores are selected to roughly match the category sales and store geo-demographics. The assumption is that if the sales in the two groups (A & B) and the shopper bases are matched, the change in sales performance in Group A (test stores) relative to Group B (control stores) can be attributed to the new planogram. Similar process is used for testing other new concepts – new packaging, new products, signage, displays, etc.

Adjacencies Influence Test Results

The analysis of behavioral data from billions of in-store trips across 100s of test scenarios by VideoMining has helped establish the importance of also including physical attributes in the store selection process.

Store and aisle layouts create different physical adjacencies which influence the shopper traffic patterns and can significantly impact a given test result. In fact, if adjacencies are not accounted for, it could even invalidate the test results.

Going back to the example of testing a new planogram. The test/control stores could be perfectly matched for sales and demographics, but if the category in each store group has different aisle adjacencies, either across from or same side as that category, then a direct comparison of sales between the two groups is will no longer reflect the true impact of the new planogram.

A different aisle partner, for example, one directly across from the test category, could drive incremental traffic (exposure) impacting the sales differently in the two groups, washing out the impact of the new planogram.

As an example, if the aisle partner category in the control stores (e.g. Potato Chips in the above aisle) happened to run a popular promotion that creates a spike in aisle traffic, it is very likely to influence incremental sales in adjacent categories (e.g. Water, Meat Snacks) . With a different aisle partners in test stores (e.g. Cookies and Crackers), the test/control performance comparison would be compromised.

Similarly, if testing the impact of a new signage or a display concept, different adjacencies or store layout would change the traffic volume or flow (direction), creating different results in a stores that were otherwise perfectly “matched”.

Including Physical Store Attributes in Store Selection Process

• Store level attributes: This includes the store size, number of entrances and overall layout including the relative order of the key departments in the store. These attributes impact the overall in-store traffic/shopping patterns – evident from reviewing the store heatmaps or purchase sequence data.
• Aisle level: This includes the relative location of the aisle in the store and the order of categories in the aisle. These influence the “aisle dynamics” – the direction of entry/exit of shoppers, where they take a U-turns, and aisle heatmaps.
• Category level: The relative location of the category in the aisle, set size and actual planogram. These influence the category shopping dynamics and at-shelf behaviors.
• Brand/Product level: The category product assortment and competitive set for a given brand. These influence behavioral data related to a product/brand important for packaging test, brand strength evaluation or new product launch.

The actual attributes employed in the store selection would depend on the concept being testing, but it is important have visibility into the relevant physical attributes to design an effective in-store test.  

To address the need for dynamic store selection for a given test, the recently announced Behavior Labs Program (BLP) by VideoMining maintains a retailer-specific store pool with searchable store attributes, from store/aisle level to the category/brand level. This allows the inclusion of relevant factors in the store matching process for a given test. The rigorous store selection process along with the use of in-store behavioral metrics allows for accurate testing of new concepts.

Taking into account different in-store factors that influence the shopping behaviors and conversion rates, the in-store tests can not only help in reliable evaluation but also help in creating diagnostics and guidelines for tweaking the new concept for greater impact and ROI.