Should We Trust Graphs?
Whenever there is a debate going on (which is most of the time) graphs are thrown around in defence of people’s arguments. Now we all love a graph, so we often believe what is on them, or we find one that we agree with, and share that. So, can we trust graphs? Are they accurate? If they are accurate, why do they appear to show different answers? If they are not accurate, can I trust them in my business? All good questions, I will do my best to answer them in this post.
Here are two points of view.
Let’s look at a situation that I came across recently, these aren’t the exact figures, but the principal is. The first graph below is a simple graph showing the turnover of a business. Now in this case, the turnover is pretty much profit, as they are selling digital products. For the sake of this example, the more turnover they do, the more profit they make. This shows the turnover, and it also shows (with a red dot) the months where they had a sale. Actually 50% off.
What are your immediate thoughts? It looks to me like the sales are good when they have a sale. It also looks like the sale doesn’t compromise the turnover in the months following the sale. If I saw this graph as a business owner, based on this alone, I would think that the sale was a good idea. I’d seriously think of doing this again. If there was someone writing a post about the importance of doing sales, they would probably use this graph in their presentation.
However, that is not the only story. What I haven’t told you is that the sale was for a specific range of products only. The turnover shown here is for ALL product types combined.
Here is the graph showing only the product type which was on sale.
Now this shows a completely different story. As you can see, the turnover barely increased, yet twice as many people bought products (at 50% of the price). This meant that no one bought the next month (probably because they bought something the month before at half price) and even the following month was lower. In theory (based on this graph), had they not had the sale, they may well have sold just as many products at full price, as they did at half price. So they actually lost out here, and based on this, I would suggest they don’t do this again.
What’s the full story?
So, why are the graphs different, and which one is right? Well, they’re both right, they’re just only telling part of the story. Here is the complete story in one graph.
There is the full(est) picture. Now we can ask if maybe the sale of the red products, had an effect on the sales of the blue products. If so, it may still be worth it. It doesn’t look worth while from the point of view of red product sales.
Here’s the point.
Anyhow, before we get bogged down with sales, this is not about that. It is about how graphs can tell different stories. Assuming that graphs (and data) are correct, different graphs showing different aspects of the data, can tell very different stories. That doesn’t really help you when you see graphs presented by others, but it does when you’re deciding what graphs to use with your own business. I create bespoke spreadsheets for clients, and I ask them this question before I start. “What do you want your report to tell you?”. This is often before I even have any actual data. I want to understand what the client wants to see, so that I can make the graphs before using actual data. This way there is no manipulation going on. The data is then added, and the relevant graph is created, showing the client the information they wanted to see. They understand what the graph is showing, and they know that their data is correct. The danger with using a graph in a discussion, is that you could use a specific graph that shows your point of view. Even if the data is correct, the graph could still be misleading.
What you can do to protect yourself from mis-information.
Here are some things that you could keep in mind when next viewing a graph.
- Do you know what data is being used, and is it accurate?
- Do you understand how the data is being used in the graph, and is that a fair representation?
- Are there any other aspects that are relevant, that are NOT included in the graph?
- Are there other ways of displaying this graph that could tell another story?
- Is there enough data to tell a true story, like an extended period of time?
I hope that has been helpful. You can trust graphs provided the data is correct, but you need to understand what they are telling you (and what they are not telling you). If you’re reading this article with existing graphs in mind, then ask yourself the questions above. If you’re thinking of getting some graphs made to analyse your data, get in touch and let me know what you need to know. I’ll be happy to help.