The excessive market share that Google has in Europe, leads us to put ourselves at his feet as if this search engine were the owner of the Internet. Therefore, the aspiration of every webmaster is to manage their search results.
And, although Google does not stop being an external tool that you can not control at will, there are certain things you can do. For example, delete your pages from Google results or add new pages that are not yet in your index . Of course, Google is the one that decides when and how to show what it has in said index.
So this is what I want to explain to you today.
How to register your domain in Google Search Console
The first thing I recommend you do as a website owner is to register your site in the Google” search console> (former Google Webmaster Tools). Every webmaster who wants to appear (or stop appearing) on Google must do so to handle what the search engine is crawling and indexing.
1. Log in
Sign in with a Google account, typically a Gmail email.
2. Add the ownership of your website
Once inside, add your domain (Add a property button) with the included protocol (https: // or http: //).
3. Check the property
It’s about showing what your page is. What you can do by any of the means offered:
- Upload to the root directory of your website ( by FTP ), an HTML file that Search Console generates . (Method that I recommend)
- Insert an HTML tag in the source code
- Confirm the provider where your domain is registered
- Use Google Tag Manager
In a few days you will start to see all the statistics.
Now that you have access to a tool with enormous potential to monitor and manage Google’s crawling,” indexing and ranking process> let’s see how you can eliminate Google URLs through this tool …
How to remove URLs from the Google index?
Come now with the million dollar question: How to remove pages from Google results ?
I do not know if you know it, but Google is very capricious when it comes to placing pages in its index. And it is enough that you do not want a page to appear in your results , so that the uncle arrives and indexes it. This happens when you find a link that takes you to this page, either from another page of your website, from another external page or from the XML sitemap that you could include in your root directory.
The fact is that sometimes you find pages that you should not examine and add to your index .
And for what reason I will not want you to add them?
Well it can be one of the following:
- Pages generated automatically by malicious software in order to capture traffic to third pages. Attacks to shared servers are very common.
- Pages created at a specific time (eg to test “Hello World”) that no longer exist or will cease to exist and you must eliminate because the content has become obsolete .
- Empty content pages (eg attachment URLs generated by WordPress), which suppose a poor content ( Thin” content> ) on your website.
- Pages that are part of a private area and every time the robot tries to crawl it will find an error, wasting the Crawl” budget> .
We see what options you have in this case to eliminate them.
Remove the page or make sure it is not indexed again
Ok, now you know that you want to delete this page or pages from the Google index. But the first thing you have to do is make sure those URLs are not indexed again .
URLs that should not appear on your website
In the first 2 cases, what you have are pages that you do not want to be on your website and, therefore, that are not tracked back by the robots.
What you should do is remove them from your site and perform the timely redirection. The issue of redirects goes a long way, but I’m going to put the 2 main cases of redirects, in a very simple way:
- If the page has a new URL that replaces the old one : Redirection”> . For the robot to stop indexing the old one and start indexing the new one
- If the page is not going to exist again and does not have a new URL to replace it : Redirection”> . So that the robot does not waste time with that URL, not re-tracking it and removing it from the index when appropriate.
URLs that are in your site, but you do not want them to be indexed
But what happens if it is about pages that you want to keep, but do not want to index? Very simple, you have to tell the tracking robots by means of a Meta” robots> tag NoIndex” in your source code.>
This is something that we can configure very easily using the Yoast” seo plugin> . Either by type of page or individually through the Yoast box (telling you that we do not want to show it in the results).
Of course, make sure not to include these pages in the XML Sitemap of your website , because this would be an inconsistency and we do not want to drive the tracking robots crazy. If you do, you will get a notification of “Problems with index coverage”.
How to know which pages are indexed by Google
If you have many pages with these types of problems, you will have to collect them all to correct the error. For this you have 2 options:
- Download the pages of the Index coverage report.
You have to be registered in Google Search Console . You can get to select the option ” Test the New Search Console ” from the left side menu and click on ” Index coverage “. Or through this” link> . Once there you will find the different types of pages that the robot has tracked and tried to index. With different states, from each state you can download the table of up to 1,000 sample pages.
In the following link you can find more information about the coverage” reports of the index.>
- Use the site command:
You can also use one of the advanced” google search commands> , specifically “site” with your domain name, in the search engine. Example:
In order to extract all the URLs, I recommend using the free” extension of seoquake> , which allows you to export the results of a search in a CSV file .
As it only shows 10 results per page, you can change the Settings” of your searches> to show you up to 100 per page.
The idea is that you collect all the URLs in a file compatible with Excel so that you can easily stay with the ones you do not want to index , organize the information and streamline the process in the case of having a large number of URLs.
Un-index the Google page manually
We go with the most important thing of the article. Where can you remove a page from the Google index ?
Well, this is one of the reasons why I have recommended that you register in Google Search Console, because you will find a tool” to remove urls in the index section of google> , being logged in, once inside the property on which you are working.
After entering the URL and pressing Continue , you will have to say that in addition to hiding temporarily you want to remove it from its cache .
You can also access this function by requesting the removal of content from the results through the following URL, even if you are not registered in Gooogle Search Console or do not own the content :
Deindexing Google pages in a massive way
Well, you know how to remove a page from the Google index. But what if you want to delete a large list of pages from your site that you have extracted as I explained in the previous point and do not want to have to go one by one? Well, for this you have 2 options:
1. Use a Google Chrome extension
You can use the ” Bulk URL Removal Chrome Extension ” extension , which you have to download from Github” and proceed as luis l> explains in” this article> . This would be the fastest method, although I have to admit that I have not put it into practice yet. What I do recommend is that you do not happily block pages already crawled by Google in the robots.txt file, because this causes inconsistencies in the coverage of the index.
2. Make redirects 410 and send the new Sitemap
The least drastic option is to use the file of indexed pages downloaded previously and to use our dear friend Excel to semi-automate the redirects .
You only need to create a column where all the cells have the instruction “Redirect 410”, another column where all the cells have a space, a column with the URLs that we have previously extracted but without the domain (we do a “Replace” of all the cells where our domain appears eg https://luisrevuelto.es, nothing). And finally, use Excel’s “CONCATENATE” function to unite all these cells.
I give you an example with the hypothetical case that I would like to delete some of the pages that are right now on my site.
Dragging the first cell will have all the URLs to delete (OJO! Only those you want to delete), in the column “Redirects” to add to your htaccess as explained in the link that I told you above ( this”>
The next step is to send the updated XML Sitemap to Google , you can do it in the new” console here> .
Finally, we must send Google to our site through the Fetch” tool search console.>
This option is probably slower but more secure than the previous one.
How to add a URL to the Google index?
Now suppose we have the opposite case. We want Google to find and quickly index a page of our website.
For that we have the tool that I mentioned in the previous point “Explore as Google”. Where you can request to crawl, a specific URL or the entire website.
By clicking on the obtain and process button, we will get Google to examine the page that we indicate. And as soon as I finish examining it, a button will appear in which we can request its indexing. Immediately after, a window appears that indicates that it will be crawled again for indexing and that if we want its internal links to be tracked as well. If we want you to examine the entire site again, we can click on the second option.
If you come across pages that Google is not indexing as you should, you can use the ” URL” inspection> ” tool in the new Search Console, for more information.
All the doubts about the status of each URL, you can solve them by reading your More” information> link.
I hope that this article has served to clarify your doubts about how to remove content from Google. As well as at the time of accelerating the process of tracking and indexing. But if you have any, I’ll wait for you in the comments 😉