Gadooney.com

It’s easy to forget about sitemaps, especially these days as XML sitemaps become standard, since they often do their work behind the scenes, often not even visible to any normal user on your site. Depending on what tool you use to create them, it’s easy for even you to forget about them, as they’re often created and updated automatically.

Sitemaps are essentially a table of contents of your site, listing every single page on your site. Back in ancient times, sitemaps were displayed just like any other .html or .php page, and were often shown hierarchically, looking a bit like a family tree, with each page accounted for and hyperlinked to the corresponding page where it was at.

Sitemaps served double duty, acting as a potential navigation tool for users on your site (I’ll sometimes still use sitemaps for that purpose today, if I’m not finding a specific page that I believe might exist, especially when hunting for affiliate programs on merchant sites, as they don’t often make those links prominent to the general public) as well as an aid to search engine spiders that are indexiing your site.

Over time, sitemaps trended towards the latter, as webmasters realized the value of sitemaps in an SEO sense, since they provide an extra layer of protection to ensure that spiders find all of your pages and that no orphan pages get left out. Search engines themselves saw the value in sitemaps as well, since it made their jobs easier, so Google has always been a strong proponent of using sitemaps on your site.

As sitemaps shifted towards more of a back-end tool, their format began to change, too, with Google pushing webmasters to use an XML format that Google developed in 2005. Yahoo and MSN agreed to support the XML-Sitemaps format in late 2006, so all three of the major search engines are on board, and send their spiders to periodically search the same pool of sitemaps submitted to them.

What does all this mean to you, as an affiliate? You need to get on the sitemap trolley if you aren’t, and create a sitemap in the standard XML format and submit it to Google.

Depending on your setup, this can be pretty simple. Since I’m always singing the praises of WordPress, let’s start there, as, no surprise, there are existing tools that make it a piece of cake to create and submit XML sitemaps iff you’re using WordPress.

First you need to grab a handy Sitemap Generator for WordPress free plugin. Installation is a snap, just upload the sitemap.php file to your Plugins folder and follow the instructions. Activate the plugin, have it generate the first sitemap for you, and you’re done, as it automatically pings Google and lets it know that your sitemap is ready to be spidered. If you don’t trust the wonder of automations, you can also submit the URL for the XML sitemap to Google at this page.

If you submit it manually, you now have to jump through a few hoops, as Google is now making you wade through some extra steps. The first time you hit that page, it’ll ask you to log-in to take advantage of it’s publisher tools, yada yada yada. I’d recommend doing so, even if you don’t have an existing Google account, as there are some useful tools there, which I’ll touch on later. So do all that, log in, and then click on the link above again. That’ll take you (finally) to the specific page to manually submit your XML sitemap.

One very cool thing about the plugin above is that you can set it on auto-pilot, so that it automaatically rebuilds your sitemap to include new pages you’ve created and pings Google, letting it know that your sitemap is ready to be crawled. (And no, I’m not being Google-centric, as Google shares its database of XML sitemaps with Yahoo and MSN, so submitting your sitemap to Google is just like submitting it to all three places.) Another cool feature of the plugin is that you can prioritize certain types of content in your sitemap, rank the types of things that are most important to you to get spidered, etc.

If you’re not using WordPress, don’t despair, as it’s pretty simply to create an XML sitemap by hand, upload it to your server, and submit it to Google. XML-Sitemaps.com offers a free generator (which works for sites with up to 500 pages), where you simply input your URL and follow the instructions. After you’ve created your XML sitemap and uploaded it to your server, use the link above to manually submit it to Google Sitemaps for crawling.

And that’s pretty much it. One thing to remember is that you shouldn’t rely on sitemaps to save you from sloppy site design, and should always use good link structure with multiple ways for users and spiders to find the content on your site. Sitemaps should simply be seen as an insurance policy to make sure that all of your good, juicy content gets indexed, not a solution in and of themselves.