Bastion Latitude’s Research Director, Mehdi Khallouk, writes about the vilification of focus groups and how quality comms testing comes from decoding meaning and decoupling creative strategy from execution.
I was really excited to be able to attend Mumbrella360, the annual media, and marketing conference, last year. The mood changed somewhat after sitting through a presentation by Adam Ferrier, best known for his regular appearances on ABC TV’s ‘The Gruen Transfer’. In ‘The Consumer is not the Answer’, Adam questioned marketers’ current obsession with frictionless customer experiences at the expense of brand differentiation and stickiness.
The thing is I nodded with Adam for 99% of his presentation – in fact, I agree with 99.9% of what Adam usually publishes. Where it hurt was when he briefly used a video of a fake “focus group” to indiscriminately discredit the entire comms testing industry.
The video in question seems to have been created in 2008 by Roger Baldacci, a US Creative Director frustrated by how “many clients these days sometimes overlook the decades of expertise of people working in advertising in favor of the comments from the random people in focus groups who show up for 50 bucks and all the cookies they can eat”. To make his point, he focus tested an updated animatic of Apple’s celebrated ‘1984’ Super Bowl ad to demonstrate that had it been tested at the time, it would never have seen the light of day.
Do yourself a favour and watch it! It’s edifying! Group participants call the concept “depressing, dark, slow and drab” in tone and lacking clarity in “what it’s trying to promote”. Asked what they would change, participants say they would use “vibrant colours and real people instead of drones” as well as “Apple logos on all their shirts or something so you have a clue”, all suggestions that would obviously have killed the core creative idea. At the end of the video, we are provided a research report that more or less… regurgitates participants’ contribution verbatim.
Please please please! If you ever receive a research report that is a direct reflection of participants’ verbatim, do yourself a favour again and fire your research agency immediately! In fact, question their fee and only pay for the direct fieldwork costs.
Quality comms testing comes not from what people “say” or “like”, but from the experience of the researcher to decode meaning and decouple creative strategy from execution. People find the concept depressing? Depression is a strong emotion: maybe instead of taking it at face value, an experienced researcher will take this as an indication of strength in the creative idea; at least people are feeling something, keep it drab! People are unclear what it’s trying to promote, yet they are still watching it? Well, maybe it shows that unbranded content, when engrossing, is adding more value to people’s lives than shameless product promotion. Keep it unbranded.
Or you know what? Maybe the concept tested here really is a big failure. An experienced researcher knows context is everything and 2008 – when these experimental focus groups were conducted – is markedly different from 1984. The core creative idea of the original 1984 ad is based on how Apple is going to ensure 1984 is nothing like ‘1984’ the book so referencing it back then is clever – but does doing so in 2008 make sense? Could changing the tagline to “2008 is not like 1984” be the creatives doing to the ad what they are accusing research of doing – f****ing with it?
Is the Apple brand even still understood the same way? Apple’s recent tonal language is indeed more vibrant and dynamic: is the cognitive dissonance with this new concept too strong?
Sadly, I have seen research reports where customer verbatim indeed was the report. I have seen terrible researchers – who cannot ladder up from verbatim to meaningful implications – as well as researchers lacking the understanding of comms and the ‘art of advertising’, but these are not the majority of the industry. At least, not the majority of the researchers with a true interest or background in comms.
I actually wrote an article about what I believe makes for great comms testing a couple of months ago: ‘The Researcher’s Eye Roll’. And for any questions or comments, please reach out at email@example.com.