If Cuil search results were not cool enough already, today I noticed some visits to this website coming from pages blocked in the robots.txt file for this site. What’s up with that Cuil?
Did you like this? If so, get FREE blog updates by email!
Cuil Ignoring Robots.txt
August 12th, 2008 · 2 Comments
Tags: Cool · Cuil · Not Cool · Robots.txt · SEM · SEO · Search Engine Marketing · Search Engines
2 responses so far ↓
1 Matt // Aug 14, 2008 at 2:04 pm
How did you see this?
2 Todd // Aug 15, 2008 at 10:10 am
Hi Matt: I just noticed some visitors to the site coming in from Cuil search results. They were landing on /blog/category/ pages, and all /blog/category/ pages are blocked in robots.txt for all search engines.
Leave a Comment