Few things arouse such fear and trepidation as Google’s disavow links tool. But if your site has been affected by one of the Penguin updates, or if you’ve been given a manual action for unnatural links – or if you think you’re at risk for either of those – then you really don’t have much choice. (Unless you think Google is just lying about the whole thing.)
Recently Google’s Gary Ilyes has stated that Google expects to update Penguin before the end of the year. If your site has some backlinks you’re not crazy about, now is the time to disavow them. And if you’ve already submitted a disavow file, it might be time to revisit your disavow file and make sure everything’s correct.
Over the past year, I’ve completed more than a hundred backlink risk assessments and looked at a few dozen disavow files. Here are the most common mistakes I see.
First, let’s make sure we’re all on the same page. If you’re working on a disavow file, you should have these pages bookmarked for easy reference:
- Google Search Consol Help: Disavow Backlinks
- Moz: Your Start-to-Finish Guide to Using Google’s Disavow Tool
- Search Engine Watch: 7 Things You May Not Know About Google’s Disavow Tool
- Search Engine Watch: Everything You Need to Know About the Disavow File
Disavowing Individual URLs When The Whole Site Needs To Be Disavowed…
I’ve seen plenty of disavow files where the webmaster just tossed in a bunch of URLs, like this:
There are a couple problems here: For one, just disavowing the URL http://www.example.com/spam-post doesn’t guarantee that page is disavowed, because it might load under several URLs – with or without “www”, with or without a trailing slash, etc. More importantly, there could be other links from that same domain that are causing the penalty.
In May 2013, Matt Cutts, then the head of Google’s spam team, recommended using the disavow like a machete instead of a scalpel, and disavowing links at the domain level:
Of course, it’s possible there might be a domain that has good, valuable links pointing to your site, and you just need to disavow a few pages. But I have yet to see a situation where disavowing individual URLs was preferable to disavowing the entire domain. In my experience, if you need to disavow one link from a site, you need to disavow all of them.
Disavowing Certain Subdomains When The Whole Domain Needs To Be Disavowed
Similarly, sometimes webmasters go crazy trying to list all subdomains they found:
Disavowing the root domain also disavows all subdomains, so the above lines can be replaced with this:
Of course, there are situations where you want to disavow a specific subdomain but not the entire domain, especially when you’re dealing with free blogging platforms like blogspot.com and wordpress.com. That leads to the next common mistake I see…
Disavowing The Whole Domain When Only Certain Subdomains Need To Be Disavowed
It’s not uncommon that webmasters put free blogging platforms into the disavow:
This isn’t using disavow like a machete – this is more like slash-and-burn. By disavowing these blogging platforms at the domain level, you’re asking Google to cancel out the link value from all blogs hosted on these domains. Instead, specify each subdomain you’re disavowing:
Sure, it’s possible that there are just too many spam blogs pointing to your site, but if you’re going to nuke these sites from orbit, then you’re probably at the point where you need to just start all over with a new domain anyway.
Disavowing Different Country-Code Blogspot Domains
As detailed in this article by The SEM Post, there’s no need to specifically disavow all the different Blogspot domains, because they’re all canonicalized to the “.com” domain.
You can easily convert all international Blogspot blogs to the .com using Excel. If you have a list of just the subdomains, do a Find and Replace for:
and replace with:
If you have a list of URLs instead of the bare subdomains, then instead find:
and replace with:
Using “Site:” Instead Of “Domain:”
Actually, this one is a mistake I’ve made. I mixed up the “site:” search operator with the “domain:” disavow directive. Whoops! Luckily Google will catch this error when you try to upload the file.
Adding A Space After “Domain:”
Does this break anything, or does the parser ignore it? I have no idea. (Please comment if you do!) This is a “better safe than sorry” recommendation. If you’re using a spreadsheet to generate your disavow file, then you can use this formula on your list of domains:
(Where A2 is the cell containing the domain you want to disavow.)
Including Unnecessarily Verbose Comments In The File
You can include comments by starting a line with “#”, which tells the parser to ignore that line. This doesn’t mean you should fill up the file with your notes, however:
# contacted webmaster 3 times (firstname.lastname@example.org)# but got no responsedomain:spamexample.com# contacted webmaster 2 times (email@example.com)#and he sent back a mean reply :(domain:spamlinks.com# the "ww" in the next line is NOT a typo:domain:ww.spamlinks.com
This isn’t a mistake per se, it’s just a waste of your time. Disavow files are processed automatically, so no human at Google will read the file and appreciate your thoughtful notes. In fact, the more notes you include, the greater the chances you’ll include a typo that prevents the file from being parsed correctly.
Google is partially to blame for this one, because their disavow example uses this kind of overly fastidious commenting:
You should only include notes in the disavow file that will help you (or a future analyst) understand and edit the file later. Contact info and dates of contact are important to track, but they don’t need to be reported in the disavow file.
Including No Notes At All
That being said, it’s not much help when I find a disavow file without any notation, especially if I’ll need to modify it, or if the site has already been through a few rounds of Reconsideration Requests. Why were sites X and Y disavowed? Has site Z been disavowed since the start, or was it added recently?
I like to include a line like this at the start:
# disavow file for example.com revised 24 August 2015
Then I’ll break up the disavowed domains by type and indicate when they were added:
# spam blogs added 24 August 2015domain:spamexample.blogspot.comdomain:superspammy.wordpress.comdomain:unnatural-links.tumblr.com# spam directories added 5 June 2015domain:unnaturallinksdir.comdomain:russian-spam-directory.rudomain:spamlinks.com
This will help later if you don’t remember why you disavowed certain domains. And just in case you lose rankings because you disavowed good sites, you can go back and narrow down the culprit based on the dates they were disavowed.
Forgetting To Re-Upload Your Disavow File When Switching From HTTP To HTTPS
Make sure you upload your disavow file to the correct Google Search Console profile – remember that there may be separate profiles for the “www” and non-“www” versions of your domain, as well as secure (HTTPS) and non-secure. So if your site makes the switch from HTTP to HTTPS, make sure you re-upload the disavow file to the new profile.
Worrying About Links From M.biz And Other Thin Aggregators
If you’ve downloaded your backlinks from Google Search Console, you’ve probably seen a ton of links to your site from m.biz, which is ostensibly a search engine that happens to have thousands of subdomains.
These links may look risky and unnatural when you’re looking at them in a spreadsheet, because the anchor text usually contains keywords and there’s so many of them. But the keyword-rich anchor text occurs because they usually use a page’s title tag as the anchor text – and there’s so many links from these sites because they link out to everybody. Seriously, every site gets links from M.biz.
Several years ago, John Mueller stated that Google ignores links from a similar but now-defunct site, Updowner.com. That might be an indication that Google just ignores links from all of these scraper sites.
There are lots of site like M.biz around, that exist just by scraping data from elsewhere – they come and go all the time, so it’s not worth the time to try and keep track of them. A few years ago in backlink assessments, I used to see large numbers of links from a series of thin aggregators: 22searchengines.com, 57searchengines.com, 99searchengines.com – every possible combination between 1 and 100. Now almost all of those domains redirect or are expired. Marie Haynes recently noticed that two scraper sites whose links often show up in backlink audits, AskIves.com and MrWhatis.com, have now rebranded or shut down altogether.
Do you need to disavow M.biz and other thin aggregators? Probably not. But I’ll admit that I could be wrong in assuming that Google ignores these links – and it’s theoretically possible that these links could push a site past the threshold for losing rankings due to a Penguin update. So if it’ll help you sleep better, then toss “domain:m.biz” into your disavow file and stop worrying.
You can distinguish thin aggregator sites by these characteristics:
- Identifies itself as a search engine or question-and-answer site
- Page looks like it was stitched together from scraped data
- Links often use the Title tag of the linked page as anchor text
- Links to 10 or more similar pages, likely including your competitors
Disavowing Natural, Valuable Links
It’s a real possibility you might unnecessarily disavow links that Google considers natural votes of trust for your site. But if your site is suffering from the effects of a manual action, maybe you can’t afford to spend too much time going through the links too finely. Sometimes desperate site owners just toss up their hands and say, “Disavow ’em all, let Google sort ’em out.”
Directories can be especially tricky to evaluate. In general, the high-quality directories are specific to a business niche or geographical location, and prioritize user experience. Low-quality directories are typically generic-looking with no specific topical focus. They usually have a “Submit Site” link in the main navigation menu, and sometimes include a “Remove This Link for $20” link on the linking page.
Links in forums are often nofollowed by default, but if they’re not, they’re generally OK, if they’re just people discussing a topic and linking to your site to help other users. You don’t need to disavow forum links unless they’ve been spammed – if someone has gone through and posted the same spammy links in multiple threads – or if there are risky links in forum signatures or on profile pages.
Here are some more articles to help you figure out what not to disavow:
Good luck with all your future disavow endeavors!