A tweak to the craigslist.org robots.txt file now bans all spidering, including that done by Google or Yahoo, from taking place. The popular online classifieds site Craigslist has apparently taken a harder line against having its content used by other sites. A thread on Search Engine Roundtable's boards was told by Craigslist to quit pulling its listings into Oodle. UPDATE! - It turns out that here. Add to document.write("Del.icio.us") | Yahoo My Web David Utter is a staff writer for Murdok covering technology and business.
Craigslist Stomps Out Spiders
1 views
Comments (0)
Please sign in to leave a comment.
No comments yet. Be the first to comment!