Running qualitative research on marketing materials

Question

In an e-commerce company, it's common for the success of creative material used in marketing campaigns to be measured in terms of numbers, e.g. click-through rates and conversion rates. However, just numbers may not give a full picture of what are good creative material.

Lately, we are trying to focus more on qualitative research, where we bring in actual users and show them creative material for feedback.

A hypothesis would be for example: clean and modern looking materials bring back the customers and make them spend more in the long run, opposed to cluttered material with too many elements and information. Besides the obvious AB-test that would be to run, which would be resource-heavy and time-consuming, could we use a qualitative research to test this and get some insights?

Another idea of a test would show customers the website or app with competitors' images swapped in and get their feedback.

I know it's more common for testing out UI and new features, but has anyone done this sort of research for campaign materials before? If so what would be things to test to look for and how would you carry them out?

Answer 1

There are market research firms that specialize in this. They started in the days of traditional advertising post WWII and still exist today.

Working with a firm of this type you would hire them to test your hypothesis. They have pools of focus groups that they generally pay with some sort of rewards for their time which they cover plus some by what they'll charge you.

One of the most basic forms of this from the golden era of publishing is showing a bunch of magazines and ads to a focus group under a time constraint then see what they remember.

Here's an article from AdWeek on the subject that's pretty informative: https://www.adweek.com/brand-marketing/why-traditional-focus-group-dying-82346/

The same type of research can be done on nearly anything including websites and UX.

Answer 2

My ideal research tool would be some kind of electroencephalography AI driven apparatus to measure the brain response to different inputs and feelings... But the technology is still on diapers and for visuals only https://www.youtube.com/watch?v=6FsH7RK1S2E

But we can probably work with less. There are some ways to measure the movement of the eye and see where the user is focusing and for how long. The problem is that with only this information, some time on a zone could either mean "Oh, I like this" or mean "what the hell is this", that is why we need different additional inputs. https://www.youtube.com/watch?v=XsCkaUIUzRQ

Probably a more sophisticated eye analysis would also measure heart beat rate, blood circulation on the face, etc, to make sense of the emotional response. https://www.youtube.com/watch?v=e9ASH8IBJ2U

But for most mortals, a card and a question asking which one do you like more are the tools we can afford... probably an automatized webpage is at our range too.

Answer 3

You must find a performance-based standard creative problem to solve that allows a comparison of the result to be evaluated by a standard user.

It doesn't exist, yet.

We've been working on creative evaluation since hand paintings were painted on the walls of caves found in Altamira, Spain and Lascaux, France 20,000 years ago.

There are some who claim to be able to test for creativity and creative solutions but most of it is bogus.

Sales is the most reliable metric for advertising; but, as the saying goes, "Half of every advertising dollar is wasted; but which half is that?"