Speaker 1: 0:54

Hello and welcome to. Seo is not that hard. I'm your host, ed Dawson, the founder of the SEO intelligence platform, keywordfewplayerscom, where we help you discover the questions people ask online and then how to optimize your content for traffic and authority. I've been in SEO and online marketing for over 20 years and I'm here to share the wealth of knowledge, hints and tips I've amassed over that time. Hello, welcome back to SEO is not that hard. It's me here, ed Dawson, as usual, and today this is really, I suppose, a bit of a follow-on from the episode the other day about robotstxt. Today I'm going to talk about sitemapxml, what it is and why we use it, essentially to make it easy for bots to find your content. So we'll go over what a sitemapxml file is, why it's important and, most importantly, how and when to use it effectively. So, whether you've heard of them before or whether it's completely new to you, I'm hoping you'll find something useful in this episode. So let's get into it.

Speaker 1: 1:51

So, first of all, what is a sitemapxml file? Well, basically, a sitemapxml file well, basically, a sitemapxml file is an xml document that lists all the important pages of your website. It's a bit like a roadmap that guides search engine crawlers like googlebot through the structure of your site and its purpose. The primary purpose is to ensure that search engines can find and index all your important content. Even if your site's internal linking isn't perfect, the format of it is. It's formatted in XML, which is a language it's like a markup language that both machines and humans can read. They're primarily designed for machines to read. Now, why do we use a site XML file? So let's think about why it's such an important tool to have on your website. One because it enhances crawlability. A sitemapxml file helps search engines discover all the pages on your site, especially those that might not easily be reached via your navigation or internal links, and this is particularly important for large websites or those with complex structures, where some pages might otherwise get overlooked, missed.

Speaker 1: 3:03

There are metadata that you can put in there. Um, now, not all search engines do this. Okay, this metadata some of it is paid attention to. So for in the sitemap. With each page that you list in the sitemap, you can include additional metadata, such as the last modification date and the change frequency of the page, and this information tells search engines how often your content changes and which pages are most important, helping them prioritize what to crawl Now. Just a note on this because Google, for example, ignores the priority and the change frequency values that you can put in there. Some other crawlers might pay attention to it, but if we're doing this mainly for Google, priority and change frequency they basically ignore.

Speaker 1: 3:40

But the last mod value, which is the last modification date that you can put against the page record in your XML file, is one that Google will listen to if you use it consistently and verifiably, so that if you're telling it I've made a change, I made a change, you're changing it all the time that changing the page, google will start to ignore that for your site. So it gets smart. But if you're genuine and you only change that last modification date when you genuinely actually change and the data in a file, then Google will listen to it. So it's a. It's a great way for Google to come to your sitemap. It'll download it before it crawls on a particular day or a particular session and it will look for any that have had a last modification that's different from when it last crawled it. And if you're honest with Google, google will pay attention to that If it finds that you're spoofing it and trying to fake it to keep recalling your content even though it's not changing, and it'll start to ignore it for your site. So don't um, don't bugger about with it. Basically just use it genuinely.

Speaker 1: 4:42

Thirdly, it helps with the indexing of new or updated content. So when you add new pages or update existing ones, a sitemap helps the search engines quickly identify those changes, ensuring your fresh content gets indexed faster. Like I just said, if you're honest with your last modification input on there, then it will take that, and also if you put new pages on there, it will discover them from the sitemap. So it's a really good tool for that. Fourthly, it's good for handling dynamic content. So websites with dynamic content, e-commerce sites with loads of product pages, they can really benefit from sitemaps because they can ensure that all the product pages are visible to search engines. And finally, fifthly, it's really good for facilitating SEO audits. So if someone's doing an audit of your site, then they're very likely to look at your sitemap XML to get a snapshot of your site structure, which can be really helpful when conducting SEO audits or troubleshooting indexing issues.

Speaker 1: 5:38

So when do you want to use a sitemap XML file? Well, so now we know what it is and why it's important, the reasons you and the times that you want to use it are one any website with multiple pages. So if your site has more than a few pages, especially if some pages aren't well linked internally, a sitemap XML file is essential. Secondly, large or complex website. So sites with thousands of pages, like e-commerce stores, large blogs, should definitely be used in sitemaps to ensure that every page gets crawled. Sites with dynamic or frequently updated content if your website is constantly being updated regularly and you're adding new pages often, a sitematic sml file will really help search engines keep up to date with those changes.

Speaker 1: 6:12

Websites with media content that you can. I'm just talking here about the normal sitemapxml. You can actually create video sitemaps and image sitemaps so you can point to Google to every single image you put on your site, every single video you put on your site Particularly important if you have a site that's very important, like you're doing a lot with images, a lot with videos. It works in a very similar way to sitemapxml, but it's specifically just for images and video. So now you know why it's a good thing to do and how to do and why you might.

Speaker 1: 6:43

In the cases you want to use it. How do we create one. So there's a few ways of doing it. So, in creating one, you know most modern content management systems cms is like wordpress, shopify, joomla, all those. They will automatically generate a sitemapxml file for you and if you're using one of these platforms, check your settings to see if it's enabled, because some of them will let you disable having a sitemapxml. You really don't want to disable it. To make sure it is enabled.

Speaker 1: 7:11

If you need to create one manually, say that you're doing it in a system or doing it in a way where it isn't created for you there. It isn't created for you. There are plenty of tools available that will help you do it and this can be useful. If you want more control, say if you might be using WordPress, but you want more control than over this. Wordpress does it as standard. You can use things like Screaming Frog, yoast SEO for WordPress, or there's online generators like xml, sitemapscom. They can help you generate a sitemap. So, thinking about the content and structure, you need to ensure your sitemap includes all your important pages and you should update it whenever you add or remove pages. You want to include metadata if possible, like the last mod tags. Again, automated systems on WordPress should be doing this for you. So this is something you need to worry about how to implement if you're not using a system like that.

Speaker 1: 7:59

And then, thirdly, you want to submit it to the search engine. So once your sitemapxml is ready, you need to submit it to search engines. Now, obviously, the one way of doing it as I mentioned in the episode about robotstxt is to put a link to it in your robotstxt file, because the robots will look there. But you can also go directly to Google Search Console and you can submit your sitemap. This is really really good if you're just launching a new site, even if you've only got a few pages to get that sitemap generated, and then go to Google Search Console, where you've registered your site and registered your ownership of your site, and you can then say to Google this is where the sitemapxml is for this website. Give it the URL and google will go and fetch it and it it boosts your indexing. Basically, it means that you're going to get that website index a lot faster than if you just leave it online and don't tell google where this sitemapxml is and don't register for google search console. So it's really important to put it there. You can do the same for bing webmaster tools as well. Also, if you have any kind of major big changes like at any point say, you put a big new section on your site and you want to speed up its indexing you can go and resubmit your sitemapxml in Google Search Console and Bing Webmaster Tools and again that will bring forward the indexing of those sections of the site for you.

Speaker 1: 9:18

And then maintenance Now update your site map regularly, especially after significant site changes. Again, if you're using something like WordPress or a tool to do it for you, that should all happen automatically for you. But if you're doing your own, make sure that you keep any new changes up to date on that site mapxml. So let's think about just a few little extra bits that were you were thinking sort of more advanced features. Okay, so you want to be careful with um sort of url parameters and duplicate content. So if your site has urls with parameters or duplicate content issues, you want to be really careful to use canonical tags alongside your site map to signal the primary version of the page. So you use those canonicals on the actual page. So on every page make sure you've got the chronicle setup. I've done other episodes about canonicals and so go back. If you don't know what canonical is, just search for those episodes and and you'll understand that. Now in your sitemap you want to make sure that you only put the canonical version of url you. It's not going to be the end of the world if non-canonical versions get on there, but it's just going to mean that google's going to waste time calling and crawling your you know, your non-canonical versions to find out what the canonical version is. It's easier if you just put have just the canonical version in your sitemap.

Speaker 1: 10:38

Don't confuse your XML sitemap with a HTML sitemap. Now, a HTML sitemap you might see on many websites. It's less common nowadays, I think, but it used to be very, very common to have in the footer a link to a sitemap page which then gave a HTML sitemap of a website. It can get very, very cumbersome if your site has a lot of pages. Now, just because you've got a HTML sitemap if you have one doesn't mean that you shouldn't have an XML sitemap. They're different things. They both have their uses. If I had to choose just one, it would be the XML sitemap, but it is good to do both and you can monitor and troubleshoot your sitemap in Google sitemap report, in this Google search console and you can monitor if your sitemap have been read correctly, look for any errors like URLs blocked by robotstxt, correct them and to ensure that your sitemap is effective as possible.

Speaker 1: 11:30

So if you've got something your sitemap that is blocked in robotstxt shouldn't really be there and it will cause. You know you're wasting crawl budget because Google tried to crawl it and then find it's blocked. So just try and make sure that your sitemap only contains pages that you want Google to be able to crawl. So why does it matter for SEO? Well, it really really helps because it gives you improved crawl efficiency. By guiding the search engines to your most important content, you ensure that every critical page is indexed. You get faster indexing of new content, especially for blogs and news sites. A regularly updated sitemap can help you let your content get indexed faster. Error prevention A well-maintained sitemap produces the risk of important pages being missed due to poor internal linking or any kind of dynamic navigation issues. You might have Strategic insights. Reviewing your sitemap can also give you insights into your own site structure and help identify any areas where you might need to improve your internal linking or your content hierarchy. I mean just sort of wrapping it all up.

Speaker 1: 12:30

Your sitemapxml file is a really important part of SEO. It's not just a list of URLs, it's actually a roadmap that guides your search engine through your site, ensures efficient crawling and helps avoid common pitfalls like duplicate content and unindexed pages. So, while it might seem straightforward to set up, do pay attention to details such as proper metadata, handling duplicates and ensuring that it gets regularly updated. A really good tool, um. You know a really good thing. Um, taught on your wordpress or any other cms should handle this well, but just worth understanding why and how and then checking it out yourself just to see that it's doing what you're expecting it to do. And then, yeah, that's it for sitemapxml. So, um, I hope you found this useful. I hope you learned something. Um, do go into if you know, if you want to see what one actually looks like, because it's almost impossible to explain how x of xml on a podcast. Just go on to go, you know. Just go and google it or talk to chat gpt about how how a site mapxml works and understand it from there. So, until next time, keep optimizing, stay curious and remember SEO is not that hard when you understand the basics. Thanks for listening. It means a lot to me.

Speaker 1: 13:42

This is where I get to remind you where you can connect with me and my SEO tools and services.

Speaker 1: 13:46

You can find links to all the links I mentioned here in the show notes. Just remember, with all these places where I use my name, the Ed is spelled with two Ds. You can find me on LinkedIn and Blue Sky just search for Ed Dawson on both. You can record a voice question to get answered on the podcast. The link is in the show notes. You can try our SEO intelligence platform, keywords People Use at keywordspeoplesusecom, where we can help you discover the questions and keywords people are asking online, poster those questions and keywords into related groups so you know what content you need to build topical authority and finally, connect your Google Search Console account for your sites so we can crawl and understand your actual content, find what keywords you rank for and then help you optimize and continually refine your content and targeted, personalized advice to keep your traffic growing. If you're interested in learning more about me personally or looking for dedicated consulting advice, then visit wwweddawsoncom. Bye for now and see you in the next episode of SEO is not that hard.