You won't get entirely accurate numbers but you can get ballpark figures with e.g.<br><br>site:<a href="http://dspace.mit.edu">dspace.mit.edu</a> inurl:handle inurl:show=full<br><br>Basically this narrows things down to the "full item record" pages. Looks like there may be dups in there -- you could try some additional conditions.<br>
<br>For the number of bitstreams:<br><br>site:<a href="http://dspace.mit.edu">dspace.mit.edu</a> inurl:bitstream<br><br>Hope this helps<br><br>Rob<br><br><div class="gmail_quote">On Thu, Feb 19, 2009 at 05:47, Bram Luyten <span dir="ltr"><<a href="mailto:bluyten@gmail.com">bluyten@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Hi Rob,<br><br>I had a question somewhat related to robots.txt and they way how DSpace instances are being indexed by google.<br>
<br>As a part of the Google analytics - DSpace comparison that I've been running, I would like to analyse which repositories are being indexed best by Google, and how that impacts their number of visits.<br>
<br>As a first, very rough estimate, I searched for:<br><br>"site:<<repository url>>" to get an indication of how many useful pages were indexed. It was interesting to see that these numbers did not really corellate with visits to this repository.<br>
I assumed that for many repositories, different browse pages were being indexed, and that these indexed pages were not very useful to generate visits // expose the content.<br><br>In a second step, I tried to look for "site:<<repository url>>" -browse". The returned numbers were in some cases even less than half of the original number.<br>
But I realise this search is being too restrictive: because many pages include the word "browse" in their navigation bar, I'm probably excluding useful item pages etc in the search.<br><br>So my question is the following:<br>
which search query could I use in Google, to get the number of useful indexed pages in Google (item pages, bitstreams, collection & community pages) ?<br><br>Already an interesting finding from my research:<br>the 15 repositories already included in the research, get 60% of their visits through search engines (average calculated on the visits in december 2008). So even more reason to get exposure through search engines as optimized as possible.<br>
<br>best regards,<br><br>Bram<br><br clear="all">@mire NV<br>Romeinse Straat 18<br>3001 Heverlee<br>Belgium<br>+32 2 888 29 56<br><br><a href="http://www.atmire.com" target="_blank">http://www.atmire.com</a> - Institutional Repository Solutions<br>
<a href="http://www.togather.eu" target="_blank">http://www.togather.eu</a> - Before getting together, get Tog@ther <br>
<br><br><div class="gmail_quote"><div><div></div><div class="Wj3C7c">On Thu, Feb 5, 2009 at 10:21 PM, Robert Tansley <span dir="ltr"><<a href="mailto:roberttansley@google.com" target="_blank">roberttansley@google.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div></div><div class="Wj3C7c">
To all users of DSpace 1.5 and DSpace 1.5.1:<div><br></div><div>These versions of DSpace ship with a bad robots.txt file that prevents search engines such as Google Scholar or Yahoo from indexing any content on a DSpace site. To check if this applies to you: <div>
<br></div><div>- Visit your site's robots.txt -- <a href="http://your_dspace_hostname.edu/robots.txt" target="_blank">http://your_dspace_hostname.edu/robots.txt</a></div><div>- If you see the following line you have a bad robots.txt:</div>
<div><br></div><div>Disallow: /browse</div><div><br></div><div>It is important that you REMOVE this line from your robots.txt to ensure that your DSpace instance is correctly indexed by search engines. More info on ensuring your DSpace site is correctly indexed here:</div>
<div><br></div><div><a href="http://wiki.dspace.org/index.php?title=Ensuring_your_instance_is_indexed" target="_blank">http://wiki.dspace.org/index.php?title=Ensuring_your_instance_is_indexed</a><br></div><font color="#888888"><div>
<br></div><div>Robert Tansley / Google</div>
</font></div>
<br></div></div>------------------------------------------------------------------------------<br>
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)<br>
software. With Adobe AIR, Ajax developers can use existing skills and code to<br>
build responsive, highly engaging applications that combine the power of local<br>
resources and data with the reach of the web. Download the Adobe AIR SDK and<br>
Ajax docs to start building applications today-<a href="http://p.sf.net/sfu/adobe-com" target="_blank">http://p.sf.net/sfu/adobe-com</a><br>_______________________________________________<br>
DSpace-tech mailing list<br>
<a href="mailto:DSpace-tech@lists.sourceforge.net" target="_blank">DSpace-tech@lists.sourceforge.net</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/dspace-tech" target="_blank">https://lists.sourceforge.net/lists/listinfo/dspace-tech</a><br>
<br></blockquote></div><br>
</blockquote></div><br>