What can you do to help Google find all of your website’s hosted content?

Sitemap Diagram Example

One of the best ways to do this is to create a sitemap, a sitemap is a list of all the pages on your website(sort of like a directory) and is designed to help search engines discover and better understand your content; as well as improve future crawl campaign. They help Google and other Search Engines(Bing, Yahoo, Ask) better understand where your content lives and what it’s about, so they can start serving the content to users for relevant searches.

How to create a sitemap

You can create a sitemap by hand(not recommended) or use a 3rd party generator to create your sitemap file. Once you have that file, upload it to Webmaster Tools. Through Google’s crawling process they’ll automatically detect changes in it. Search engines are constantly working to build out what information you can give them in a sitemap. To stay up to date with what exactly you can put in this file check out the resources in the help center.

Google Webmaster Sitemap Help Center

Google Webmaster Sitemap Help Center

Creating a sitemap using a 3rd party website

In my opinion, this is not the ideal way to create a sitemap, because you need to manually re-generate and upload it every time, but it is much better than having no sitemap at all. This type of sitemap is ideal for HTML websites. You can Google “sitemap generator” and select one of the tools which you think is appropriate. The sitemap generator I have used in the past is xml sitemaps.com. Once you have created the sitemap, you will need to login to your FTP client, and upload the .XML file to the root folder of your domain. For example, my sitemap is located at http://justincaron.com/sitemap.xml. **One thing to keep in mind here is that the /sitemap URL directory will be taken by the sitemap file, so DO NOT CREATE A /sitemap directory (ex: http://justincaron.com/sitemap). If you do want to create a manual sitemap for visitors (not Google) then do something similar to me, by creatingthea URL http://justincaron.com/site-map. This will differentiate the different pages and functions they serve.

Creating a sitemap using a 3rd party WordPress plugin

As you have probably noticed by now, I’m an avid WordPress user, and sitemaps are just another reason WordPress is a fantastic CMS platform. I use and recommend using the Google XML Sitemaps plugin, it is very intuitive and user-friendly – and has nearly 8 Million downloads. An advantage to using a plugin vs a generator is that they automatically update and re-submit your sitemap to the major search engines (Google, Bing, Yahoo, Ask) when new content is posted/updated. This will ensure that each crawl will have your most relevant and recent content indexed.  Image and Video sitemaps are also very useful if you have a lot of images or videos on your website. I do have a lot of images on my website, but not very many videos, so I also use a image sitemap; specifically the Google XML Sitemap for Images plugin. I have seen some major success for my search terms since using an image sitemap, I now get roughly 1/3rd of my traffic just from image results; so I strongly suggest you take a look at it.

Other advantages of using sitemaps

There are some additional advantages to creating a sitemap which aren’t discussed in the video:

  1. Ability to rank the importance(0.1 – 1.0) of all pages, categories, etc. This becomes very important when you want to ensure certain pages are given more weight (link juice) and want other pages and categories to be given less weight.
  2. Timestamps of most recent changes – Google gives a lot of weight to content that is updated frequently (fresh content) because when your content is updated regularly, it shows that you are invested in making sure your content is up to date, is relevant, and will best serve your end users (website visitors); which reflects good on Google for serving those results.
  3. Compliments other SEO tools which can be used (Robots.txt)
  4. Keywords in your page URLs will be better indexed and will improve your ability to tell Google what search terms (keywords) are most relevant to your site. This can be found in the Webmasters Dashboard under > Optimization > Content Keywords

 Testing your sitemap

Once you have created your sitemap, you want to test and make sure that it is functioning properly. After your sitemap has been generated and uploaded, or you have installed and run the plugin; visit the page where the sitemap should be located, it should look similar to my sitemap located here. This is all you need to do until your website has been re-indexed. You can check to see when it was last indexed by viewing the “Cache Version” of your site – as the images below show.

Checking Cache Website Version 1

1. Click the Right Arrow key to show the “website preview”

Checking Cache Website Version 2

2. Click on the “Cached” link

Checking Cache Website Version 3

3. Check the date listed (as underlined)

If the date listed under the “Cached Version” is after you submitted/uploaded your sitemap, then you know the most recent version is being displayed – be patient if it isn’t updated, it can take up to a few weeks for your site to be re-indexed; although it is typically faster. The last step in testing is to do a search in Google with “site:domain.com” (ex: site:justincaron.com). By adding “site:” you are telling Google to only display pages that they have indexed with that specific domain. Here you can see I have 863 URLs indexed for my “justincaron.com” domain. If you do not all the pages you included in your sitemap displayed here, then you will need to go back and review your sitemap and re-generate it.

 

Using Sitemaps – Help Google Find Content Hosted On Your Website


This video can be found in the Google Webmaster’s YouTube Channel.