Welcome Back And Yearly Recap

Speaker 0:00

Hello, and welcome back to SEO Is Not That Hard. It's me, Ed Dawson here as always. And today, I'm just gonna recap what I think the biggest SEO stories have been over the last year since, since the podcast went hi- on hiatus. So

Search Console Metrics Suddenly Shift

Speaker 0:18

the first thing I wanna talk about is Search Console. Now, I did mention this because this did start just before we went on hiatus, and that was when Google switched off their norm equals 100 parameter. That was a little bit of text you could put at the end of the URL that used to tell Google to give me the first 100 results. So not just the first page of 10 results, but give us 100 results all on one page. And loads of rank trackers used to use it to pull 100 res- results per page, so they could see huge amounts of data on who was ranking where for a query. Now, Google, yeah, around about a year ago, started tightening up on who could scrape their results. I think this was because less to do with the SEO tools that had always been doing this, but probably more to try and stop the AI companies like OpenAI and Anthropic and others from scraping their results out. I think that was probably the trigger for it. Because essentially it let other AI companies essentially steal their index. They did this and it had the knock-on effect of making search console impressions look very different. So essentially what happened was overnight it looked like all of a sudden all websites were getting less impressions, search impressions, because there were so many impressions being driven by automated means But everyone's average position went up because now people weren't being found in those top 100 results. They were still there if people had actually gone through all 10 pages. But because people, these tools weren't doing that anymore, it meant your av- average position went up because all your lower position ones didn't exist in Search Console, 'cause Search Console is driven by actual searches So yeah, average positions went up, impressions went down, but clicks remained flat because automated tools weren't clicking results. So it made everything look weird, and it highlighted how much of the actual traffic to Google was automated, not real people. So what matters still matters, but you're always gonna have to look at that data and the day it changed through that lens of those metrics changing because of what Google did. So that, that's the first one big one that came out.

AI Overviews And AI Mode Rise

Speaker 2:42

Secondly, the results pages have changed quite a lot in the last year. So two things sit on top of Google now, and that's the AI overviews which is the written answer that you get before the links, and then AI mode, which is a separate tab off that. That's a conversation, not a list. Now, AI overviews has gone from around a third to over half of all results now. So for every qu-- for more than half the queries, you're gonna get an AI overview over the top, which essentially in most cases gives an answer, so people don't need to visit web pages so much anymore. So this is causing traffic to drop for lots of sites, especially for simple queries That second part, AI mode. Now Google are claiming over a billion users are seeing AI mode now and interacting with it. And again, this is having an impact on informational queries, a click impact. So there's various estimates around there. There's probably between from a fifth to a third fewer clicks. It's potentially even more depending on the niche you're in and what your site is about And that's one massive change, and I think it's affecting lots of sites. And if you are an informational type site, like an affiliate site, you're probably being hit more than others. And even if you are a transactional site, like an e-commerce site, you might be getting less traffic, but you might be getting the same number of sales, so your conversion rate may seem to be higher, but that's because more and more research is being done off of your site before people get to you And this expands out to point three, which is where search is leaving Google.

Research Leaves Google For LLMs

Speaker 4:30

So ChatGPT, Perplexity, Claude, Gemini, Copilot, they all answer questions now that people used to type into Google, and none of them send you an impression report. None of them send you a click. I'm seeing it myself in the way I'm using The internet now for finding information, solving problems, making purchases. I spend a lot of time now in Claude Code or in Codex. My first point of call when I've got a question or a problem I'm trying to solve is to talk it th- talk it through with Claude and give a whole load of information that I didn't use to give to Google when I was trying to find solutions to problems with Google. There's a lot less reading around the problem that I'm trying to solve or the topic that I'm interested in. And part of it is down to speech. So how people interact is starting to change as well. I'm now an avid user of Whisperflow. I spend a lot more time talking to my computer than I do type. Now, if you've not heard of Whisperflow, essentially it's a bit of software you can put on your iPhone, on your PC, on your Mac, that lets you talk to your computer, and it will transcribe your text there and then and will paste it into whatever piece of software you're using. So it'll paste it into the terminal if you're using Claude Code. It'll paste it into your email if you're using, if you're using email. It will paste it into the webpage that you're using if you're talking with Claude. It means that you can sit there now and have a much more in-depth sort of str- stream of consciousness when you're trying to solve a problem. So for example, I would-- I was looking today at there's a leak in my-- in our horse box. The window is leaking. I'm gonna have to try and repair it. So I'd taken a photo of it. I uploaded the photo to Claude, and then I just talked through the whole problem with Claude. I said, "This window's leaking. I want to know how I find the source of the leak, what I'll need to fix the leak," and just went through the whole th- problem with it, probably maybe 100 to 200 words, and then hit Enter. Okay. So this is the kind of level of detail that you can go into now when you're trying to solve a problem. So I'm not going to a website. I'm going to Claude saying, "This is my problem I've got," and we'll talk it through. And you can have that conversation, and you can give more and, Claude can ask questions, you can give responses. As-- When I say Claude here, put in your AI of choice, essentially. And Instead of having to search websites now, you can have this conversation. And if it leads to needing a product, I can then say where do I need to get this product?" So there's some sealant. I can say to it, "Where's the best place to find this sealant?" And it will go off and find the best places to buy it for me, the best prices, best delivery, all those kind of things. This is all stuff that previously would have gone through multiple Google queries, through reading multiple websites, and doing a lot more research myself, and it would have taken a lot more time. So for me, it saves a lot of time, and I get to the solution much more quickly. So same with the project that I spoke of in last episode about getting AI to control the heating in the house. Part of that was finding all the bits and working out what bits we needed to buy to make everything talk to each other, hardware pieces, that kind of thing. And again, that was where I would talk it through. I was using Claude Code then, had a project in Claude Code. We went through, we built the software I needed to build. It helped me find the hardware I needed to get. It helped me put the hardware together. I could show it screenshots, and it would find the places to buy things. All those kind of searches and stuff that have left Google Now it still shows how important it is that you have a presence on the web because obviously these LLMs, AIs are recommending places on the web to make those final purchases. But a lot of the research now is being done by the agents, not necessarily by a human So how's Google reacting over the past year? They've

Google Tightens Spam Rules

Speaker 8:49

put in much stronger policies about scaled AI, and this is their spam policies, and they're trying to stop scaled AI and manipulative tactics. If you spent a year pumping out AI articles, then you've probably discovered since then that this doesn't work and it's gonna come back and bite you They're also, and this is the interesting bit, have just put out their earnings reports, and they're actually losing money for the first time in a long time. They're spending more than they're earning, their free cash flow. They're still making huge amounts of money, but they're also investing a huge amount, and their actual advertising sales aren't growing like they were. They are essentially eating their own tail when it comes to the changes they've made because they're trying to compete with the LLMs. They're trying to compete with the ChatGPTs and the Claudes and others by having these AI modes. But it means that they're sending less traffic out. Their ads don't get clicked as much. And I think it's a tricky one for them because they're in a situation very much go back 30, 40 years. Kodak, for those of you old enough to remember when we used to have to buy physical film to put in analog cameras. Kodak made all the film for cameras. They were the most popular brand by a long way. They actually invented the digital camera, but they didn't do anything with it. They had patents for it and all sorts, but they didn't ever develop it and become the, the default brand for digital cameras because they didn't want to destroy their traditional film business. And It didn't work out for them. They went-- they had one of the biggest bankruptcies. They went from one of the most profitable, biggest companies in the world to nothing in a very short space of time because th-photography changed. It went digital. They didn't keep up. They disappeared. Same with Blockbuster. They were all about going to the video store, renting a video. They even had op-opportunity to buy Netflix when Netflix was still a, a young startup, but they passed on it and There are no Blockbusters anymore. They got eaten because the whole video rental market and went from physical DVDs and videos to streaming, and they missed it. Google are clearly worried about the same thing happening to them, that search is changing and therefore they are going to become redundant. But it's put them in a strange situation because they're trying to compete, but at the same time also keep things as they were in many ways, 'cause they rely on advertising and selling ad clicks. So how it's all gonna pan out, I don't know, but they're clearly trying lots of things, and it's affected lots of people, not just them Now, the thing I keep trying to ask myself when I look at this is, am I an outlier when it comes to the way I use search?

Are We Early Adopters

Speaker 11:51

Because most likely like yourself listening, we're at the cutting edge of search, have been for a long time, the cutting edge of technology, so we're always gonna be the first people to try the new things How fast is it gonna take over the world essentially? How fast is it gonna take over search? Are most people using AI search? Are most people using Claude, ChatGPT? Are most people just still just using Google and are searching in the old ways? So I don't have the answer to that, but things are definitely moving fast and things are definitely changing fast Things that I think that have stayed the same are, I think in terms of being indexed by Google and getting into the conversation, it is very much still the way that it was always done.

GEO Still Needs Classic SEO

Speaker 12:38

There's a lot of-- I can't believe I've got this far in these two episodes already without mentioning GEO yet, the new buzzword. GEO, if you've not heard of it, Generative Engine Optimization. So that's optimizing to get into Claude, ChatGPT, Perplexity, et cetera I'm not gonna get into the debate over what we call the disciplines that we work in, but a lot still is those core things that the podcast talked about for the past three years, for all those 300 and whatever episodes. A lot of all that still counts in terms of getting yourself into indexes, 'cause at the end of the day, the AIs, the agents or the LLMs are just doing searches most of the time to find information. It may be searching their own index, it may be searching Google's index, it may be searching somebody else's index. But they all still work on the same principle. They've built an index, and when they want to look something up, they go to that index and get the results, and you wanna rank in those indexes to then become part of the conversation in the AI. So it's all still important, but it's where people interact with it is changing, and it's also the point at which people come into the actual picture as far as the website is c- is concerned is what's changing a lot more. A lot more of the initial research, the top of funnel stuff is happening at the agentic level now, and I think that's only going to grow. And y- I think the agents will get further and further down as people give them more and more power to do things. Right now, I think most purchases are still being done by a human on a webpage. That's why you see these studies like the Marie Haynes one, where they were showing that the conversion rate from, for people who are referred by an AI agent is three, four, five times. I can't read the example, but a lot higher than the conversion rate of people just coming through traditional search. But the number of people coming from those LLMs and AIs is much smaller, and that's because m- those people are doing most of their research away from your website, and it's actually the agent and LLMs that are d- that are on your website or referring to information you've published to help that person just make that purchase decision. So

Measurement Problems And Agent Tracking

Speaker 14:58

that's another change So what can you do as a website owner or an SEO or a consultant in this changing landscape? I think the first core thing is, as they would say in Hitchhiker's Guide to the Gal-- Hitchhiker's Guide to the Galaxy, "Don't panic." You do need a website still. You need to be indexed. You need to be crawled. You need up-to-date information published where the right people or the right agents or the right bots or the right crawlers can find it. So your website is still important How you measure your success is gonna be less about the number of actual clicks. Sales, which were always the most important thing. If you're selling something, sales are the im- are always going to be the number one metric. But you're gonna s- have to start looking in other places to be able to judge your performance, and it's going to be harder. Because some of these things are much harder to work out. For example, rank tracking Which has been a big thing over the last year. Everyone's trying to do prompt tracking. There is no such thing as a short tail anymore In AI, everything is long tail. People do not go to their AI and say, type in broadband if they're interested in changing broadband and then seeing what the first few results are and ending up on a comparison site. They are gonna talk to their AI and give a detailed description of why they're fed up with their current broadband provider, what they're looking for in a new broadband provider, where they live, how many people are in the house, the kind of things you use, you know. They... And they're gonna have a conversation with their AI, and then they'll get recommended, A, maybe a provider or, B, a pr- a comparison site to go and find more So you can't track the volume on those in the same way you could do with the old rank trackers when it was just 10 blue links and people made much shorter, s- more succinct phrases that they used when they were searching So tracking how your visibility stacks up compared to others is harder now. I've got some ideas for how we can do it, and I've experimented with some things, but at the moment there's no perfect solution to it You need to know how people are finding you in ways when they're not coming to your site, but their agents are. You need to be able to see when agents come into your site or when a crawler from the LLM is coming to index your site for its own index, or when LLM is doing a search on the fly to help answer a user's question, you need to know when you're being hit with them. And at the moment, for most people, there's no easy way to do it Because it doesn't appear in the analytics packages you're using at the moment. It doesn't appear with your Google Analytics, it doesn't appear in Clicky, it doesn't appear in places like that So you need to go somewhere new to find that information. And that's what we'll talk about more in the next episode, where I'll talk more about Salience and what we've built at salience.com, which helps you Find how AI agents, crawlers, bots are actually interacting with your site in a way that you can't... You just don't know at the moment if you don't look at those logs. So that's gonna be the next episode. Oh, and yeah, the other thing I wanted to mention is obviously Keywords People Use still exists, and it's still a fantastic product if I don't say so myself. And, that, that side of still needing to build content, still needing to answer people's questions is probably as important as ever because you still want to be seen to be the source, the authority in an area, and you've gotta have that content on your site. It might not be people reading it much anymore, might be agents, but those agents are reading it on behalf of someone at the other end, so you still need to be the source that is cited and which the agents lead people to make those purchases at the end. So yeah, you still need to do all those things, all that keyword research, all those... Like I said, all that previous old-fashioned SEO stuff still counts. Because if you just took your... If you decided, "Oh, SEO doesn't count anymore," and took your website off-offline, that's it. You would s- the AI agents wouldn't come to you anymore. You wouldn't get recommended. You still have to have that presence. You still have to have that content. You still have to be the authority to be cited and recommended by AI. It's just a, it's just the journey that is changing and the point at which people are getting involved is starting to move away from the top of funnel and more down to the bottom of funnel. Anyway, I think that's enough for today and about where things have come and where things have changed.

Robots.txt And Cloudflare Checks

Speaker 2 19:48

So yeah, I think the key thing to do is check your robots.txt, make sure you're not blocking the AI crawlers and the AI agents, because it's important to be in their conversation with them. And make sure if you're using something like Cloudflare, that's not blocking them for you. Because if you don't open yourselves up to visibility, then you're gonna s- see yourself more and more frozen out, especially by those more sophisticated searchers, who are the ones who are probably gonna be the, your best customers. Anyway, that's it for now. So I'll be back the next episode where we'll talk more about Salience how it can help how we've built it, and yeah, that's it. So until next time, take care, and I'll see you later.