# robot.txt for sol.astro.virginia.edu # modified 7-Sep-2007 RJP # bars all bots from the following directories # Note, User-agent: * Disallow: /~eww6n/ Disallow: /dev/ Disallow: /local/ Disallow: /class/archive Disallow: /class/arras Disallow: /class/balbus Disallow: /class/bary Disallow: /class/bastian Disallow: /class/black Disallow: /class/bradley Disallow: /class/chevalier Disallow: /class/colley Disallow: /class/condon Disallow: /class/constandach Disallow: /class/fall2001 Disallow: /class/fall2002 Disallow: /class/hawley Disallow: /class/hicks Disallow: /class/howard Disallow: /class/ianna Disallow: /class/johnson Disallow: /class/kelsey Disallow: /class/kemper Disallow: /class/labs Disallow: /class/lab_signup Disallow: /class/li Disallow: /class/lister Disallow: /class/majewski Disallow: /class/mcdavid Disallow: /class/murphy #Disallow: /class/oconnell Disallow: /class/osten Disallow: /class/polak Disallow: /class/randall Disallow: /class/richards Disallow: /class/rood Disallow: /class/sarazin Disallow: /class/saslaw Disallow: /class/schiavon Disallow: /class/schmitt Disallow: /class/skrutskie Disallow: /class/spring2003 Disallow: /class/telobs Disallow: /class/thuan Disallow: /class/tolbert Disallow: /class/trammell #Disallow: /class/whittle Disallow: /class/xiluri Disallow: /reports/ # RJP 7-Sep-2007 # The following lines are of unknown origin, but I was worried that they #appear to allow the following bots access to our entire website, because # of the final, empty, Disallow statement # User-agent: ArchitextSpider # User-agent: InfoSeek Robot 1.0 # User-agent: Lycos # User-agent: Scooter/2.0 G.R.A.B. V1.1.0 # User-agent: Open Text Site Crawler V1.0 # User-agent: Slurp/2.0 # User-agent: WebCrawler/3.0 Robot libwww/5.0a # Disallow: