Problems getting posts/ pages indexed in Google
I have recently got back in to doing some affiliate sites. One of the main problems I had was actually getting content indexed. I'd post stuff and Googlebot would not even come to visit. I actually have the problem with this site and maybe this post will never get indexed either.
I was on Linkedin today and stumbled upon a great post that seems to explain whats happening. I am not sure how to link to the post so I'll quote the best bits. Credit to Daniel Foley
So after analysing hundreds of sites over the last few months – more and more I am seeing huge swathes of issues when it comes to indexing content.
What I first thought to be a problem with Googlebot – actually looks more like Googlebots policy on indexing content of value seems to have changed dramatically.
It makes sense – with over a billion websites in the index and countless website owners churning out blog after blog, thin page after thin page – is that not a waste of crawl and processing power?
…
Creating blogs because “that's what people search for” isn't enough now.
You genuinely need to create content that brings something useful to the table – otherwise you'll find that your investment into content – was likely fruitless and a waste of time.
This rings very true to me. It seems like Google looks at a site and decides how good it is, decides whether its going to bother even sending Googlebot. Even using the instant indexing or manually submitting to the index via search console doesn't do anything. Google just seems to think your site is not worth bothering with so it will index a few pages but nothing new.
The old advice of keep blogging and writing regular blog posts is outdated. Even many prominent SEOs don't bother blogging on their own site anymore. Twitter threads and Linkedin posts seem to be the place where the audience is and that's probably where you should be posting.
I was watching a Webmaster Central video a few weeks back and John Mueller was asked about something similar. He said it can take months for Google to change its view on your site. In the meantime, you need to keep improving your site.
I guess that means get more links. I am reading between the lines here because how would Google ever know if your site had improved if its not even bothering spidering it?
I'll keep monitoring as its all I can do at the moment. Its quite heart-breaking producing high quality content for Googlebot to totally ignore it.
The request indexing button does very little. Googlebot won't bother coming for it. Even if it does, what are the chances of it ranking? Its all demoralising. On the plus side, I do have some theories that I will test out and luckily, some of my affiliate pages are indexed, so it helps me if no one else can get indexed after.
After I press publish, Googlebot will actually visit and hopefully index this post.
Ben has a BEng (Hons) in Computer Science and 20 years of experience in online marketing, specialising in SEO, lead generation and affiliate marketing. After spending over a decade as an igaming affiliate, he has decided to concentrate on GA4 training and SEO Audits.
