How to prevent very very large ccaches?

[Nicolas Williams]

On Tue, Jun 18, 2002 at 06:55:25PM -0400, Ken Raeburn wrote:
> Another thing to look at in the short term would be the stdio version
> of the ccache code.  Since it uses buffering, it wouldn't make nearly
> as many i/o calls.  Currently it's turned on by using "STDIO:" instead
> of "FILE:", but really I think it should be an implementation detail
> which one of the two is used for "FILE:" since AFAIK the contents
> should be the same regardless of the means used to access the disk
> file.

[You can't just turn on STDIO - that requires a patch I just posted.]

Given that patch the improvement I see of using STDIO over FILE is
tremendous. First off, my 20-at-a-time mass kvno run barely dents the load
average now. Second, on a quiet system, the difference between STDIO and
FILE, when using a very large ccache (1000 entries) is a FACTOR of TWO
(and a factor of TEN when using a ccache half that size).

I was using both of the patches I've posted when I got the results
mentioned above.

Wow!

All of which begs the question: why not make STDIO be the default
ccache type?

I did not expect the difference of impact on load average between FILE
and STDIO in my tests. I'm not sure how to explain that - in both cases
the ccache was being locked only once during the ccache search, and in
both cases I was using the kvno command to keep all other overhead low.


> Ken


Cheers,

Nico
-- 
-DISCLAIMER: an automatically appended disclaimer may follow. By posting-
-to a public e-mail mailing list I hereby grant permission to distribute-
-and copy this message.-

Visit our website at http://www.ubswarburg.com

This message contains confidential information and is intended only 
for the individual named.  If you are not the named addressee you 
should not disseminate, distribute or copy this e-mail.  Please 
notify the sender immediately by e-mail if you have received this 
e-mail by mistake and delete this e-mail from your system.

E-mail transmission cannot be guaranteed to be secure or error-free 
as information could be intercepted, corrupted, lost, destroyed, 
arrive late or incomplete, or contain viruses.  The sender therefore 
does not accept liability for any errors or omissions in the contents 
of this message which arise as a result of e-mail transmission.  If 
verification is required please request a hard-copy version.  This 
message is provided for informational purposes and should not be 
construed as a solicitation or offer to buy or sell any securities or 
related financial instruments.