I've been sitting with a question lately that I suspect every researcher knows well: where do you actually start? Even when a study is grounded in solid theory, the analytical choices are anything but automatic. Statistics is a language, and like any language, there are many ways to say the same thing — and the one you choose matters.

This update is about data preparation: the essential work that happens before any meaningful analysis can begin. I had planned to move quickly through this phase. Guess what, it is not quick!

Getting the data ready

Before we can explore what coaching does to resilience and wellbeing, we need to be confident the data itself is trustworthy. This means more than simply cleaning responses — it means verifying that what participants recorded actually measures what we intend it to measure.

To set the scene: most research surveys are built from many individual questions. Those questions are grouped together because they've been tested and validated to reliably capture a broader concept — such as resilience or psychological wellbeing. These broader concepts are what researchers call latent variables: things we can't observe directly, but can infer from patterns in responses.

The thing I thought I understood (and then didn't)

I spent time this phase working through Confirmatory Factor Analysis — or CFA. I'll be honest: two months ago, I had never heard of it. I read about it, felt fairly confident I'd grasped it, and then discovered I hadn't. The moment I finally understood it properly was one of those deeply satisfying realisations that reminds you why this kind of work is worth doing.

Here's where I initially went wrong: I assumed CFA was testing whether a survey measured what it claimed to measure. My logical next question was — if the survey has already been validated by its developers, why would I need to check that again?

Because that was a different dataset.

CFA isn't re-testing the survey. It's asking whether this particular dataset — collected from this particular group of people, at this particular moment in time — behaves in the way we'd expect. Even a well-validated survey can produce data that doesn't quite fit the expected pattern, and if that happens, we need to know about it before drawing any conclusions.

The insight I didn't anticipate

The second revelation came when I started thinking about what it would mean if the data didn't fit — not at the start of the study (e.g., the baseline measure, since by this time I had established it did fit), but at a later time point?

By running CFA across all measurement points in the study, we can examine something called measurement invariance: essentially, whether participants are interpreting and responding to the questions in a consistent way across time. In other words, we need to check that we are still measuring apples with apples.

If the measurement structure that fits well at baseline shows signs of shifting later on, this requires investigation. One possible explanation? The participants' understanding of the concepts has shifted.

And in a coaching study, that raises a genuinely fascinating question.

Not just improving it — but reframing how they understand it? If someone's inner definition of resilience shifts through the coaching process, their responses to the same questions might naturally move in ways that go beyond a simple before-and-after score comparison.

This isn't something the current study was designed to test. But it opens up a genuinely exciting direction — both for research and for coaching practice.

From a research perspective, a future study could explicitly ask participants at the end of a coaching programme: "How, if at all, has your understanding of [resilience / wellbeing] changed since we began?" — comparing their interpretations of the same questions at the start and at the end.

But there's an immediate practical application too. This kind of reflective question could be built into any coaching programme review — not as a measure, but as a conversation. Something as simple as: "We asked you this question at the start. How would you answer it now, and how has what it means to you changed?" That shift in perspective might itself be one of the most meaningful outcomes of the coaching work — and one that a numeric scale alone will never fully capture.

Next update: testing the research model. Watch this space.

REFERENCE READING:

Putnick, D.L. & Bornstein, M.H. (2016). Measurement invariance conventions and reporting. Developmental Review, 41, 71–90. 10.1016/j.dr.2016.06.004

Sheela Hobden

www.bluegreencoaching.com

June 2026

I share musings and monthly updates from my research into coaching, resilience, and wellbeing. If you’re a coach, a psychologist, or just curious about what doctoral research actually looks like from the inside — I’d love for you to follow along.

Looking for more research based inspiration?

Check out this podcast series I recorded for the Association for Coaching:

Apple & Pear Image credit: Photo by Melanie Dijkstra on Unsplash​