In praise of the humble data table (or how to get your late night number crunching on the front page

I've been working with the travel industry to help them influence government policy during the pandemic

Communicating insight is as big a challenge as collecting it. It requires a razor sharp focus on the insight or message you need to get across and an innate understanding of the audience you’re communicating with. Get either of those things wrong and you’ll see eyes glazing over and all your hard work come to nothing.

Too much insight is presented back in charts and graphics which leave you staring at them asking “What is this trying to tell me?” Even people who generally do this kind of stuff quite well get it very badly wrong sometimes. I mean, what ...?

When planning to present data ask yourself two things:

1. What's the message or insight that needs communicating?

Is there data that not pulling its weight or emphasising the message that should be excluded to give the insight room to breathe? I sense in the example above quite a lot. And I'm still unclear what I'm supposed to take out of this spaghetti mess

2. Who do I want to communicate this to?

The second point is crucially important. Let’s face it, numbers are a bit marmite.

Two decimal places are generally a no no in my book but if you’re reporting a clinical trial, they may have a place. But if you’re talking to people who make a living with words, it’s often better to keep it simple. It’s not dumbing down, it’s just making their life easy whilst ensuring you get your message across.

For example, I used to present some quite complex numbers assessing interest in 80-100 book concepts across 4-5 countries and multiple demographics back to book editors inform their publication roadmap. I recognised that the only thing they wanted to know was what global blockbuster to start work on next. I did it by removing all the significance testing and all the numbers and just colour coded the results. If it was green in all 5 markets, it was likely to be a global winner. A mix of colours, less so. It also reduced the report from 80 page to one, with a large appendix for those who cared about the numbers.

And so in celebration of the humble table, I give you a humble table. This is the original humble table that was pulled together close to midnight on Saturday 19th June when I discovered Test and Trace were actually publishing positivity rates of travellers coming from Red, Amber and Green countries.

It was a massive messy spreadsheet which I was keen to share with people who could get the message – that travellers from Amber countries were presenting minimal risk to the public health of the nation - into the mainstream media (so journalists, better at words than numbers). I used my data reduction skills to put 1400 rows of data covering countries, traffic light colour, tests, sequencing etc into a single table. Nothing else - certainly not a bar chart - would have been able to convey 24 bits of data so succinctly or impactfully in my view

It was subsequently picked up wholesale, including my rambling footnotes, by the Telegraph, Mail and many more, as well as by various MPs fighting the corner for the beleaguered travel and tourism industry. Hell, it even made the front page of the Times.

It may not have bells and whistles, but it told a story and that story got told. And maybe, just maybe, it made a bit of a difference. I do hope so. The travel and tourism have deserved better for a long time. So here’s to the humble data table.

And for anyone looking for a good read on this topic I can highly recommend Andrew Ehrenberg’s A Primer in Data Reduction. https://www.amazon.com/Primer-Data-Reduction-Introductory-Statistics/dp/0471101354

For instant insight, dump the NPS surveys, just walk the walk

As a research professional I’ve always been a bit cynical about NPS and CSAT surveys.

Rarely have I found a direct correlation between those scores and business performance. It can happen but it’s the exception not the rule. Companies invest considerable sums of money running them and then trying to squeeze some insight out of them, which is not always easy. Not least because the people who fill them in are usually either very happy customers or people who have something to get off their chests.

I want to use my recent airline experience to explain why there is a much better and faster way to find out what’s wrong and fix things.

We travelled with BA from T5 recently. It wasn’t great to be honest.

First, online check in is no longer allowed due to the vast amount of paperwork that needs checking.

So you arrive nice and early and stand in a cattle pen for an hour waiting to check in, until it gets too close to your flight time when they haul you out the queue and check you in anyway.

The member of staff who checked us in asked to see our PCR tests. We told him we didn’t need them for Greece as we had vax certification

On the four hour flight there was no food service whatsoever beyond a small bottle of water and a miniature breakfast bar, given to everyone for free. Because of all the queuing we didn’t have time to buy food before boarding so that was it.

The cabin crew huddled at the back out of sight once we’d been given our small bundle of sustenance

The flight left on time, arrived early, captain talked to us a few times. My husband gave him a good score on the landing.

Two days later BA send us a survey asking about our recent flight to Mykonos. Would we recommend BA to a friend or relative? Well we were very grateful to BA for taking us to Mykonos after EasyJet cancelled 3 flights but not sure that answers that question.

So here’s my point. Rather than spending hours crafting and analysing survey results from people who could be bothered to tell you about their trip, BA could have developed a plan of action in one day. By living the experience with me. During my six hour end to end experience I can tell you:

1. It was my worst travelling experience for some years

2. It was clear a lot of people were turning up without the right paperwork. It’s complicated. So some pre-departure advice or checklist for travellers from BA may helped have reduced this

3. Alternatively disconnect the paperwork checking from the bag drop. That would allow hand baggage only people to get through the system more quickly

4. Consider checking the paperwork at the gate – as was the case we experienced last year. This means the people checking are 100% knowledgeable about what the entry requirements are for that destination and don’t ask you for things you don’t need. If we hadn’t been clear ourselves we may have panicked

5. Let people know in advance they might need to bring some food with them or promote pre-ordering more prominently.

6. And if cabin crew are there for our safety, why hide them at the back of the plane? A lot of the survey questions were about how “safe” I felt. I was on the plane. Doesn't that say something about my confidence in BA?

All that insight in six hours with some actionable suggestions versus a big survey which will feed into a dashboard and verbatims cogitated about over many weeks

“Walking in their shoes” and observational research are amongst the most powerful forms of research if you’re looking to understand where an experience is going wrong or well. I recently spent a very insightful two days watching people clean their toilets and was able to feed in more insight than any survey asking people how satisfied they are with their bottle of Toilet Duck ever could

There’s a lot to get fixed here, most of it crap that's been imposed on them and they're doing their best, but I do hope BA don’t wait weeks to get round to fixing it.

Living well with lower research budgets

It’s inevitable pretty much every business will be looking to make savings post COVID and that means less money and less resource for research and insight. So what are your options?

Put up a fight for your budget

Now, I appreciate this one is probably a long shot but, if you’ve done your job well, you should be in a position to remind your stakeholders that the insight you delivered on Project X contributed to that project’s bottom line. Risk aversion is likely to be higher than ever and your track record could help your case. Recession or not, if you want the impact of what you do to be noticed it’s your job to make sure your research has been implemented. At eBay the team regularly reviewed the progress of business initiatives with stakeholders and found out how they had applied the research we delivered. It reminds stakeholders of your contribution to the success of a project, even if your bit was done 2 or 3 years ago

Start with a blank sheet of paper

Rather than cut back on your current programme, start again. I’m a big fan of zero sum budgeting and believe your research programme (and therefore your budget) should reflect the challenges of informing the decisions your business needs to make now. I’ve happily ridden the rollercoaster of totally flexible budgeting knowing that I will be judged not on how much or how little I’ve spent but the impact the business has seen as a result. And because your business’s priorities are likely to be growth, growth and growth …

… take a long hard look at all that tracking or continuous research you do

As time goes by a lot of research budget and people time gets taken up by continuous research, things like CSAT, NPS and brand tracking. This puts an inevitable squeeze on ad hoc budgets when times are hard. In my first research leadership role I took the slightly radical step of pausing all routine continuous research and its reporting and waited for someone to notice. They didn’t. Suddenly we had a lot of time and money to play with. This is not an approach I’d advocate for everyone but it can be useful to review the frequency of some of what you're doing. Continuous research needs to reflect the rate of change going on in your market, customer experience, brand, product. There may be studies eg your brand tracker that can be shifted from, say, a monthly to a quarterly or even reporting annual cycle.

Think really hard before dumping the agency and bringing research in house

I’ve worked in organisations where agencies have done everything and ones where agencies were kept at arms’ length. Neither was optimal. What I would say is that, whilst it feels like you might save some money in the short run bringing more in house, it may come at a price

Every minute you spend scripting surveys or moderating online communities is time you’re not spending influencing stakeholders and raising the profile of insight in your business. If you’ve worked hard to get your voice heard, don’t blow it by becoming an insight sausage factory and doing the things you were doing at the start of your career

Many of the functions that are brought in house are actually the lower cost bits of a project: data analysis, survey scripting, coding, panel management etc. Always remember your time has a value too and it can be used to support the business in a variety of ways with differing levels of impact

Agencies bring some of the smartest freshest thinking around, as well as some of the nicest and most interesting people. To not have them in your life will make you a poorer and less fulfilled researcher: believe me I’ve been there and I couldn’t wait to break free

That said, there are things which can be successfully done in house: user testing for example has some great self service tools which provide perfectly good levels of insight. UX research really isn’t rocket science. Similarly, if you’ve got a relatively stable brand tracker, move it to field and tab only and you’ll be stunned at the savings (I achieved over 80% savings on full service agency costs).

Consider demanding more syndicated research

It pains me to think of how much of our collective budgets will get invested in post COVID-19 trackers. These will be largely the same for every commissioning client. Whilst a boon to the research agencies, they will limit the budget you have available for forward looking growth oriented research. Maybe now is a good time to think about industry focused syndicated trackers?