Diagnosing Traffic Drops During a Crisis: Was It You, Google, or the Whole World?

Posted by Dr-Pete

We want to fix things and believe we’re in control. When your house is filling with water, you grab a bucket. If there’s a hole in your roof, the bucket might help. If your sink is overflowing, the bucket is distracting you from the real problem. If the river is overflowing, that distraction could be deadly.

When traffic is falling, it’s easy to panic and focus on what you can control. Traffic isn’t just a nice-to-have — it puts food on the table and the roof over your head that keeps the water out. In the rush to solve the problem, though, we often don’t take the time to validate the problem we’re solving. Fixing the wrong problem is at best a waste of time and money, but at worst could deepen the crisis.

In any crisis, and especially a global one, the first question you need to ask is: is it just me, or is it the whole world? The answer won’t magically solve your problems, but it can keep you from making costly mistakes and start you on the path to a solution. Let’s start with a fundamental question:

(1) Did your traffic really drop?

My “fundamental” question might sound like a stupid question, especially given the wide impact of the COVID-19 pandemic, but it’s important to remember that traffic fluctuates all the time — there are weekends and seasonality and plain, old regression to the mean. What goes up must come down, and as much as we’d like it to be true, business is not perpetually up and to the right.

Using Google Analytics, let’s consider some ways we can validate a traffic drop. Here’s four weeks of GA data (March 1-28) for a site which was seriously impacted by COVID-19:

Given the known timeline of COVID-19 (the WHO declared it a pandemic on March 11), this is about as clean a picture of a traffic drop in the presence of a known cause as you’re going to get. Most situations are far messier. Even here, we’ve got the impact of weekends and day-to-day fluctuations. One quick way to get a cleaner view is to summarize the data by week (make sure your date-range covers full weeks, or this data will be skewed).

The trend is much clearer now. In a two week period, this site lost more than half of its traffic. I’m restricting the timeline for clarity, but as we gather more data, we can validate the trend pretty easily. The graph above covers all traffic sources. From an SEO perspective, let’s add in a traffic segment for Google traffic:

This graph is just eight data points, but it tells us a lot. First, we can clearly see the trend. Second, we can see that the trend is almost identical for both Google traffic and overall traffic. Third, we can see that this site is very dependent on Google for traffic. Don’t underestimate what you can learn from small data, if it’s the right data.

This isn’t meant to be a GA primer, but let’s look at one last question: Is this traffic drop seasonal? Usually, your own industry experience and intuition would come into play, but one quick way to spot this is to compare year-over-year traffic. One note: match your full weeks so that you’re covering the same amount of weekdays vs weekends. In this case, I’ve shifted the 2019 range to the four full weeks of March 3-30 …

This isn’t the easiest graph to read, and I probably wouldn’t put it in a report to a client, but you can see from the green and purple lines that both overall traffic and Google traffic for this site were relatively flat last year during March. This really does seem to be an unusual situation. Even if we knew nothing about the context and COVID-19, we could tell from just a few minutes of analysis that something serious is going on here.

(1b) Did your rankings drop?

As a search marketer, and given that we’ve clearly measured a Google traffic drop, the next question is whether this drop was due to a loss of rankings (we’ll get to other explanations in a moment). In Moz Pro, one quick way to assess overall weekly search visibility is to use either the main view under “Rankings” or go to the “Competition” tab. I like the competitive view, because you can quickly see if any changes impacted your broader industry …

I’ve simplified this view a little bit (and removed the site’s and competitors’ names for privacy reasons), but the basic story is clear — neither the site in question nor its competitors seemed to have any drop in visibility during March.

For a richer view, go back to the “Rankings” tab and select “Rankings” (instead of “Search Visibility”) from the drop-down. You’ll see a graph that looks something like this …

This visualization takes some getting used to, but it contains a wealth of information. The bars represent total ranking keywords/phrases, and the color blocks show you the ranking range (see the legend). Here we can see that overall rankings have been relatively stable, with even some small gains in the #1-3 bucket.

If your account is connected to Google Analytics, you can also overlay traffic during the same period, which is shown by the dark gray line. Dual-scale graphs can get tricky, but this visualization really makes it clear that there’s a mismatch between the traffic drop for this site and their search rankings.

(2) Did Google do something?!

Usually, when we ask [demand / shout / sob] this question, we mean “Did Google do something to the algorithm to make my life miserable?” We can argue about whether Google is trying to make your life miserable at another time (preferably, when the bars re-open), but the core question is valid. Did Google change the algorithmic rules in a way that’s negatively impacting your site?

For large-scale algorithm updates, you can check our own Google Algorithm History page. For smaller/daily updates, you can check our MozCast research project. While having a gut-check against major changes can be very useful, the messy truth is that Google rankings are a real-time phenomenon that’s changing minute-by-minute. In 2018 alone, Google reported 3,234 “improvements” to search.

Keep in mind that all Google algorithm tracking tools are based, to some degree, on fluctuations in rankings. In our example scenario, we’re not seeing ranking shifts. Let’s pretend, though, that we have seen a traffic drop with a corresponding ranking drop, and we’re trying to determine if it’s just us or if something changed with Google.

Here’s a graph of MozCast data from my analysis of the January 2020 Core Update …

In this case, we’ve got a pretty clear three-day period of ranking fluctuations. If our traffic dropped during this period, it’s not absolute proof that an algorithm update is to blame, but it’s a solid, educated guess and a useful starting point.

Let’s look at the two weeks around when COVID-19 was declared a global pandemic …

I’ve kept the same scale and 30-day average reference (from a relatively quiet period early this year). Note that algorithmic activity (i.e. ranking flux) is way up compared to the period before and after the January Core Update. One day (March 18) doesn’t even fit on the scale of the original graph and came in at 104°F on MozCast.

What does all of this mean? It’s possible that Google is changing the algorithm rapidly to address the broader changes in the world, but I strongly suspect that the world itself is impacting this flux. Sites are changing rapidly, adding and removing products and content, news sources have dramatically shifted their coverage, and some businesses are closing completely. On top of that, we’re seeing an unprecedented shift in searcher and consumer behavior.

Algorithm flux can be a useful answer to the question “Is it just me, or is it Google?” during normal times, but all that it’s telling us right now is that the world has turned upside-down. While that’s an accurate assessment, it’s not particularly helpful. If you’d like to hear more about the impact of COVID-19 on Google rankings, check out “SEOs talk COVID-19 search disruption” from Barry Schwartz with myself, Marie Haynes, Olga Andrienko, and Mordy Oberstein.

If traffic has dropped, but rankings haven’t, it’s also possible that the behavior of searchers has changed. We can get some insights into this by using Google Search Console. Here’s the graph of total clicks for our example site from March 1-28 (corresponding with the GA data) …

As expected, total clicks on Google results show roughly the same trend as Google organic traffic in GA. Total clicks are a function of two variables, though: (1) search impressions, and (2) click-through rate (CTR). Let’s look at those individually. Here’s the graph of total impressions for the same time period …

Now we’re getting somewhere — there’s an overall drop in impressions. This isn’t just about the example site, but searcher behavior before they even see or click on that site. People are searching less for the phrases that drive traffic to our example site. Finally, let’s look at CTR …

CTR has also dropped, even a bit steeper than impressions. This is a bit harder to interpret. Knowing what we know, it’s likely that people are clicking less because of overall lack of interest. This is consistent with the COVID-19 scenario. People are less likely to be looking for the service this site offers. On the other hand, it could be that something about the site or the competitive landscape has changed that’s driving down CTR.

If you see a CTR drop without a corresponding impression drop, review recent changes to the site, especially changes that could impact what’s displayed in search results (including your TITLE tags and META descriptions). In this case, though, it’s reasonable to assume that we’re looking at an overall drop in demand.

(3) Has the world gone mad?

Spoiler alert: yes, yes it has.

The Google Search Console data above has already suggested that we’re seeing a shift in the wider world and searcher behavior, but if you want to get outside of your own data, you can explore the world a bit with Google Trends. For example, here’s a Google Trends search for “movie tickets” for March 1-28 …

Not surprisingly, searcher interest in movie tickets declined sharply after the COVID-19 outbreak. People who aren’t going to movies aren’t going to be searching for showtimes and ticket prices. Google Trends data can be spotty in the long-tail, and we can’t necessarily attribute a trend to an event, but non-brand trends are a good supporting data point for whether your traffic drop is isolated to your site or is impacting your broader industry.

One final tip — everything discussed in this post can also be used to explore a traffic increase. Even during COVID-19, traffic has gone up for many topics and sites. For example, here’s the Google Trends data for “how to cut hair” from the same March 1-28 time period …

Whether or not cutting your own hair is a good idea, people are definitely showing more interest in the topic (I admit I’ve watched a couple of YouTube videos myself). We don’t typically dive deep into traffic increases — it’s too easy to just sit back and take the credit. I think this is a big mistake. Understanding whether a traffic increase was driven by changes you made or broader market shifts can help you understand what you’ve done right so that you can replicate that success.

The big picture is everything

Over the last few years, I’ve heard more people say things like “I don’t care about traffic, I care about conversions!” or “I don’t care about Google rankings, as long as I’m getting traffic!” Our gradual move toward bottom-of-funnel metrics makes sense — we’re all trying to make a living. Taken to extreme, though, we lose valuable information. Focusing on conversions is certainly better than focusing on “hits” a la 1998, but no single metric tells the whole story.

Let’s say that the only thing you track is leads. Leads are where the money is. Sales are up, leads are up, times are good. Great. Inevitably, disaster strikes (even if it’s a minor disaster), and your leads drop. What do you do? You’ve cut off your ability to read anything but the last chapter of the story. You know how it ends, but you don’t know how you got there. Without understanding the path from leads back to visits back to rankings back to impressions, you’re not going to see the whole story, and you’re not going to know where things went wrong.

Even when times are good, this approach is short-sighted. Sales-focused culture creates a tendency to celebrate the wins and not ask too many questions. If traffic is going up, why is it going up? What content or keywords are driving that traffic? What industry trends are driving that traffic? If you can answer those questions, you can replicate success. If you can’t, then you’re going to have to start from scratch as soon as the celebration ends (and the celebration always ends).

It may be cold comfort to know that your entire industry or the whole world is suffering with you, but I hope that this process at least prevents you from fixing the wrong things and making costly mistakes. Ideally, this process can help you uncover areas that may be trending upward or at least help you focus your time and money on what’s working.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!