It very often happens that a pre-production site (test site) is indexed by search engines, following a bug or human error. How to restore the situation? This week, first let’s see what to do by default so that this kind of problem never occurs, we’ll see next week what to do if it occurs anyway…
When we create a new website or as part of a redesign/migration, most of the time we use a test site (called “preprod” for pre-production, therefore before going into production) on which we check that everything is going well. And once everything is in place, we move from this site to the real site, called “production”.
Of course, this pre-production site is not meant to be indexed by search engines like Google or Bing. Yet sometimes it happens, due to a bug or human error. You then need to correct this and de-index this intruder site, but this is not necessarily done in a very easy way. In short, you have to follow a procedure.
Already this week we will see, first of all, what to do at the start, during the creation of this pre-production site, to protect it from the visit of motor robots. Next week we will look at what to do if this was not set initially and the staging site ends up in the SERPs.
So you will know more by watching this video number 229:
Sorry, my preprod site is indexed, what should I do? – Video N°229 proposed by Olivier Andrieu (Abondance). Source: Plenty
Transcript of SEO video 229: “Oops, my pre-production site is indexed, what should I do? » :
“Hello and welcome to this video from 229th Abundance in which I wanted to talk to you about a topic that we often hear about on social networks in general and about which I have many questions: when you have a website test, a “preproduction” site – I go to explain what it is – when it is at some point, most often by accident, indexed by search engines, how does it react and get it out of Google Bing and others?
Yeah, what is a preprod site? This is a pre-production site. Of course you have a website that will be in production: www.example.fr here and you will have a test site, a site on which you will do all the tests to verify that what you want to implement then works well and will be called preprod.example .fr or test.example.fr or v4.example.fr, you give it the address you want and once it works well, which you are sure it works well, you put it into production on the real site that Internet users will see . Obviously the pre-production site, this test site, shouldn’t be indexed by search engines like Google or Bing. You can test the query [allinurl:] on engines like google and you’ll find a number of pre-production sites, which have no place there. There will not only be pre-prod sites, as there will also be articles that will talk about pre-prod sites and not all pre-prod sites are called “preprod point something” but that will allow you to identify some examples. A test site is not indexed in Google so at some point a bug or error occurred and the site found itself indexed which is not good.
So already in a first phase, and we will see the remedies in next week’s video, first of all what must be done so that the pre-prod site is not indexed, in order to be sure that, in the future, it will not be indexed when it shall we put online? First, at the global site level, we will make a robots.txt file, then your site address from preprod / robots.txt with the content, then User-agent:* – I am addressing all robots – Disallow : / – Crawling of this site is prohibited. So already normally, that should be enough.
We will do it again because the engines don’t always read the robots.txt, there could be a problem at this level, so we will double with a “noindex” meta robot tag in all HTML pages so meta robots content=noindex, there we are sure that if there is a problem with the robots.txt at least there is noindex that will be taken into account.
If you have PDF or Word or Excel or other short files that are not HTML, you can use the X-Robots-tag directive which you will set to noindex – watch video number 153 where I explain what this X-robots-tag is, this HTTP protocol directive. So here we’re at the overall site level, a robots.txt and at the level of each site element of the noindex, so we’re normally there, we’re shielded.
We are still about to complete it, why not finally you can do it anyway, by putting a password on the site. At that moment it is clear that there is a password so the robot doesn’t have the password so it won’t work and the site won’t be indexed. You can also create a white list, i.e. authorize access to the site for certain IP addresses only at the server level: developers, finally people who logically have access to this site. If the IP address is not in the white list, we can not log in, return a 403 error message or other.
So really there are a lot of things that can be done, they are all complementary so don’t hesitate to do this when creating a preprod site. So what if a pre-production site was indexed because an error occurred, something happened that caused the pre-production site to be re-indexed and again Is this something that happens very, very, very, very often? You will know next week as it will be the subject of the second video. If your pre-production site was indexed by mistake, what is the procedure to put in place: does it obviously exist? You will have to wait a short week to get access to video number 230 🙂
So thanks for following this video, I urge you to review already, looking forward to next week’s one, a little bit of everything I’ve talked about – it’s really a sampling of the videos I’ve already made on de-indexing, there are many others, so feel free to check out all of them on the Abondance site. Thanks for your attention, see you next week for the rest of this video, see you very soon and thanks for your loyalty. Thanks and goodbye 🙂”
#preprod #site #indexed #Abundance #SEO #video #N229 #SEO #news #search #engines
Comments
Post a Comment