Some time ago a client asked me to work on the China leg of a global qualitative project. The project was to be managed by a well-regarded overseas research agency based in the same country as the client.
As is typical in these projects, the overseas agency would be responsible for developing the methodology, writing the discussion guide, and briefing us via Zoom.
The objective of the project was to ‘test storyboards' (their words, not mine). We were briefed by the lead researcher, who sent the discussion guide and draft ideas through beforehand.
The guide was reasonable. There was perhaps a bit too much ‘purchase intent’-type stuff in there for my liking, and a few too many questions asking participants to predict their future behaviour. But in general the structure, flow and coverage were pretty sound.
The three 15" creative ideas were in a PowerPoint document. On each slide was a 10-15 frame storyboard, with narrative and VO under each frame.
So far, so good.
Then the briefing meeting started, and the first cracks began to appear.
The researcher launched straight into the creative work without any explanation of the background to the project. The advertising objectives went unmentioned. And from his superficial take on the ideas, you'd be hard-pressed to know that he had already researched them in two other markets.
But it was when I asked what stimulus we’d be using that things really started to unravel.
“Well, it will look like what you see in the PowerPoint document”, he said.
“Yes”, I replied, “but these are storyboards. All the frames are on one page, with the narrative and VOs written underneath. What do you actually want us to show in the groups?”
“Well, the storyboards. The things you're looking at now. You let each respondent study the board, long enough for them to read through it, and then start asking the questions.”
“Er, right. So, let me just clarify this. For each idea, you want us to show all the key frames at once, on a single board?
“Yes."
"And you want the respondents to read the narratives and VOs themselves?"
"Yes"
"Er, ok. Wouldn't you prefer it if the moderator read out the narrative and VO?"
"No, that's not necessary. Haven't you done this type of research before?”
I felt like asking him if he'd done this type of research before, but thought better of it.
"In this type of research", I replied, "we usually show the frames individually, one per slide, not the whole board at once. We present them sequentially, one after the other, to reflect the way the idea would unfold in reality. And we'd read the narrative and VO out loud, for the same reason.”
“Oh no, don't do that," he countered. "It would be too confusing. And it would take too long to present, much longer than 15 seconds.”
“OK…and, er, is that how you've been doing it in other markets?”
“Yes.”
“Right, I see…and it worked ok, did it?”
“Yes. We projected the storyboards onto a TV screen. A few of the respondents couldn’t see the narrative and voice-overs clearly. So we asked them to stand up and walk over to the screen. That reminds me, don't forget to ask them to bring their glasses.”
This was getting surreal.
“Well, if we’re going to do it this way," I said, "might it be a good idea to print each idea out instead of showing it on a screen? That way, the participants will definitely be able to see it clearly. They won't have to get up and walk over to the TV."
“Yes, ok. But try to show it on the TV screen first. It’s a TV ad, so it makes sense to show it on a TV.”
"Well, I can't fault that logic," I didn't say.
The agency concerned claims to use approaches that are ‘unique and innovative’. Well, that's certainly one way of describing them.