Have you received the dreaded “Googlebot cannot access CSS and JS files on www.yourwebsite.com” message from Webmaster Tools? Read on to find out what it means and how to fix it…
What it means…
When Google crawls the Internet it reads your website’s files in order to understand what your site looks like and how it functions.
Most websites are built with a combination of different file types, and your site likely has many of these JS and CSS files. If your site is built in WordPress, your theme and plugins contain lots of these files too, and many will block Google from crawling them by default.
Why is Google mentioning this now?
As Google’s algorithms evolve they become better and better at ‘seeing’ at websites more closely to how humans see them. Reading these JS and CSS files, whilst not always important for understanding a website’s content, is important to understand how a website appears. We suspect that Google wants to make this understanding a more significant part of its ranking algorithm, but in order to do so it needs to make sure that it can ‘see’ your site.
To clarify, we believe that the effect on ranking of blocking these files is minimal at the moment . But we do expect it to have more effect soon, hence the message to millions of website owners.
How can you fix the JS and CSS files issue?
The simplest fix is to make a change to your robots.txt file and add the following code:
User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ allow: /wp-content/plugins/ allow: /wp-content/themes/
This tells Google and other crawling bots that any file in your plugins and themes folders can be read, including these JS and CSS files.
You’ll then need to either wait for Google to recrawl your site and notice the changes, or (preferably) resubmit it through Webmaster Tools to request these changes to be applied sooner.
Will this affect my site’s security?
All this fix will do is give crawling bots Read access to your plugin and theme files. These changes don’t allow anyone to edit these files or make any changes to your website, so it makes no difference to your site’s security.
Do I need to get this sorted?
Yes you do. This is not something to ignore. It’s fairly rare for Google to explicitly tell us that something will negatively effect ranking, so it’s time to take notice.
Can Exposure Ninja fix this for me?
Yes. If you’re an existing SEO client we’ll be rolling out a fix for your site as part of August’s work. If not we can still fix this for you. Just click the Fix this for me! button and we’ll make this fix for you for £49+VAT.