Post Views: 20 In the previous posts about the Robots Exclusion Protocol (REP) we explored what’s already possible to do with its various ...

Post Views: 49 Hello 2025! (Yeah, we know, time flies!) We’ve had some exciting plans in the works for Search Central Live (SCL) ...

Post Views: 61 With the robots.txt file, site owners have a simple way to control which parts of a website are accessible by ...

Post Views: 69 A long-standing tool for website owners, robots.txt has been in active use for over 30 years and is broadly supported ...

Post Views: 65 We’re very excited to announce that Search Central Live is going to Madrid for the first time on April 9! ...

Post Views: 59 On April 2, 2025 we’ll be in Johannesburg, South Africa for the very first Search Central Live event in Africa! ...

Post Views: 84 Every now and then we get questions about robots.txt, robots meta tags, and the control functionality that they offer. Following ...