Chapter 3 QAing your own work

3.1 Self QA

Before you ask anyone else to QA your work, you should QA your own work.

There are many examples of self QA that you can use:

  • Balance checks. Do percentages add up to 100%? Do totals add up correctly?

  • Sense checks. Do outputs look right or logical?

    • Example: Suppose your analysis tells you that you have a mean average primary school size of 600 pupils across 20,000 schools. This means that you have 12 million primary pupils. Something has clearly gone wrong!
    • Use your common sense, public data and analysis that other analysts have done to sense check your work.
    • If things don’t look right, always investigate further.

  • Spot checks. Do some spot checks of formulae, values and calculations. If your spot checks show some errors, it is worth investigating further as these errors may have been replicated elsewhere in the analysis.

Remember, before you ask anyone else to QA your work, you should try to prove yourself wrong.

3.2 Odd, funny or unexpected figures

If your analysis gives you answers that are unexpected, investigate them!

For example, if the Teacher Supply Model (TSM) estimates that every subject needs more Initial Teaching Training (ITT) places this year apart from English, investigate why this is. Is a trend reversing or running counter to expectation? Are values for this year very different to previous years?

It is inevitable that someone senior will ask you about any unexpected results. It is important to understand why your model is giving you such answers, and to put these answers into some context using analysis that has been done previously. If you do not have the answers to these questions prepared, it will give people less faith in you and your work.

Doing these checks will lead you to either find an error, or produce a logical answer to explain this oddity to stakeholders. Either way, it’s a win!

If you have calculated the outputs elsewhere (e.g. a separate signed off model), do the outputs match? If you are modifying a simple version of a model for a more specific scenario, the first stage of this update should be ensuring that outputs match those that have been signed off.

Being able to demonstrate that you haven’t just taken analysis at face value makes you look good and makes your analysis better! Always ask yourself, when looking at results, what questions are your stakeholders likely to ask? Preparing anaswers for these types of questions is a very important skill for an analyst. You can guarantee that senior stakeholders are most likely to ask about numbers that stick out or look odd…

3.3 Finding errors

The best way to find errors, outliers or things that have gone wrong is to use charts. Charts enable quick sense checks and any ‘odd’ results will stick out like a sore thumb. One of the first things a good QAer will often do is produce some charts.

How about finding a small problem within a big spreadsheet or extensive code? It is useful to always work backwards to find errors. If something looks wrong, work backwards through your spreadsheet or syntax. Look at recent changes and updates. This will enable you to find the place where an oddity first started to appear, and help you to identify the specific problem or cause.

When looking for errors, keep calm and focus on each step of a complex process one piece at a time. If you get flustered or stressed, you will often miss errors, so sometimes it pays to have a drink, take a break or sleep on it. A fresh mind will often be more capable of solving a complex issue.