[Dspace-general] Problem with DSpace 1.5, 1.5.1 prevents indexing by search engines

[Robert Tansley]

To all users of DSpace 1.5 and DSpace 1.5.1:
These versions of DSpace ship with a bad robots.txt file that prevents
search engines such as Google Scholar or Yahoo from indexing any content on
a DSpace site. To check if this applies to you:
- Visit your site's robots.txt -- http://your_dspace_hostname.edu/robots.txt
- If you see the following line you have a bad robots.txt:

Disallow: /browse

It is important that you REMOVE this line from your robots.txt to ensure
that your DSpace instance is correctly indexed by search engines. More info
on ensuring your DSpace site is correctly indexed here:

http://wiki.dspace.org/index.php?title=Ensuring_your_instance_is_indexed

Robert Tansley / Google
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/dspace-general/attachments/20090205/db5c2db3/attachment.htm