<div dir="ltr">I wrote the sad story of a two days long thread in PageOneX :) <br><a href="https://github.com/numeroteca/pageonex/issues/179#issuecomment-30275654">https://github.com/numeroteca/pageonex/issues/179#issuecomment-30275654</a><br>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Dec 10, 2013 at 8:20 PM, pablo rey <span dir="ltr"><<a href="mailto:pablo@basurama.org" target="_blank">pablo@basurama.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Ok, I will add the steps to reproduce these errors in the already open issue <a href="https://github.com/numeroteca/pageonex/issues/179" target="_blank">https://github.com/numeroteca/pageonex/issues/179</a><br>
<br></div>
I don't think it is related to the cpu throttling, these problems were happening before.<br></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Dec 10, 2013 at 8:11 PM, Edward Platt <span dir="ltr"><<a href="mailto:elplatt@media.mit.edu" target="_blank">elplatt@media.mit.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Pablo can you create issues for these bugs in github and make sure they contain steps to reproduce? Let me know if you think we should revert the cpu throttling. I'll put a little time into fixing those errors.<div>
<br></div><div>-Ed</div></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Dec 10, 2013 at 12:49 PM, pablo rey <span dir="ltr"><<a href="mailto:pablo@basurama.org" target="_blank">pablo@basurama.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>My experience with those problems:<br><br>For long threads: Almost
every time I create or update a thread (add one day) I get a "500
Internal Server Error", after "creating background image grid" according
to the logs (sometime after "creating overlays for topics"). <br>When I
refresh the display view sometimes it works, but I guess many users
leave the page after that (I've found many broken threads with no even a
single area coded). Sometimes after 5 or 6 times of refreshing, the
display view works again.<br><br></div><div>For short threads (one or two days): it returns the error for the composite images <a href="http://pageonex.com/numeroteca/test-repeated/" target="_blank">http://pageonex.com/numeroteca/test-repeated/</a> It is not able to generate them 88 newspapers, one day).<br>
<br></div><div>I don't know if the problem for short threads is related to the
big images that imagemagick has to create. If that is the case, we
should limit the total height of those images: to avoid the tool from
crashing AND to provide a good overview of the news coverage without the
need of zooming out too much.<br></div><br>thanks Rafael for the hints!<br><br>ps:<a href="https://github.com/numeroteca/pageonex/issues/182" target="_blank"> Pagination in the thread list is not working again</a>, maybe related to lack of memory?<br>
</div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Dec 10, 2013 at 6:46 PM, Rafael Porres Molina <span dir="ltr"><<a href="mailto:rporres@gmail.com" target="_blank">rporres@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Request timeouts can cause internal server errors. You may look into that.<div><br></div><div>Regards,</div>
<div><br></div><div>Rafa</div></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">2013/12/10 Edward Platt <span dir="ltr"><<a href="mailto:elplatt@media.mit.edu" target="_blank">elplatt@media.mit.edu</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I haven't seen any cpu usage notifications, and the server hasn't crashed, so the throttling seems to be helping on that front.<div>
<br></div><div>The errors and broken images you see could be from the throttling or they could be a coincidence. Any idea how widespread/common those problems are? If it is throttling, the error likely occurs when the server would have otherwise crashed.</div>
<div><br></div><div>Obviously we want to avoid both, so there will still need to be some work around limiting the image size.</div><div><br></div><div>-Ed</div></div><div><div><div class="gmail_extra">
<br><br><div class="gmail_quote">
On Tue, Dec 10, 2013 at 7:41 AM, pablo rey <span dir="ltr"><<a href="mailto:pablo@basurama.org" target="_blank">pablo@basurama.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div><div><div><div>Great, Ed, how is this CPU throttling working?<br><br>I added one extra day to the thread <a href="http://pageonex.com/numeroteca/corrupcion-espana-diciembre-2013/" target="_blank">http://pageonex.com/numeroteca/corrupcion-espana-diciembre-2013/</a> and updated:<br>
<br>1st time: internal server error.<br><br></div><div>I reloaded:<br></div>2nd time: the newspaper images and overlays were broken.<br><br>I went back to edit and "updated" the thread<br></div>3rd time: it worked fine.<br>
<br><br></div></div></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Dec 9, 2013 at 11:12 PM, Edward Platt <span dir="ltr"><<a href="mailto:elplatt@media.mit.edu" target="_blank">elplatt@media.mit.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I've added cpu throttling to the imagemagick settings. Hopefully this will help stop the crashes, but keep an eye out for any weird behavior!<div>
<br></div><div>-Ed</div></div><div><div><div class="gmail_extra"><br>
<br><div class="gmail_quote">On Sun, Dec 8, 2013 at 12:14 PM, pablo rey <span dir="ltr"><<a href="mailto:pablo@basurama.org" target="_blank">pablo@basurama.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div><div><div><div>Hi Ed,<br></div>Prevent overloading would be great. <br>I think the server just went down again when editing the same thread.<br><br></div>Let's talk next week if there is something I can look into also.<span><font color="#888888"><br>
</font></span></div></div><span><font color="#888888">p</font></span><div><div><br><div><div><div><div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Dec 6, 2013 at 11:19 PM, Edward Platt <span dir="ltr"><<a href="mailto:elplatt@media.mit.edu" target="_blank">elplatt@media.mit.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hey Pablo!<div><br></div><div>The server's back up. We've been getting notifications about cpu and disk usage, probably from resizing images. Next week I'll look into whether there's an easy fix to prevent overloading the server on long/short threads.</div>
<div><br></div><div>-Ed</div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On Fri, Dec 6, 2013 at 5:30 AM, pablo rey <span dir="ltr"><<a href="mailto:pablo@basurama.org" target="_blank">pablo@basurama.org</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div><div dir="ltr"><div><div>Hi Ed and Rahul,<br>I was adding a new topic to the thread <a href="http://pageonex.com/numeroteca/corrupcion-espana-diciembre-2013/" target="_blank">http://pageonex.com/numeroteca/corrupcion-espana-diciembre-2013/</a> and the server went down.<br>
<br></div>I can not ssh.<br><br></div><div>Apart from the problem of lack of memory to calculate the background image grid for too many or too few images, I think there is another problem with too many overlays when it has to create the composites.<br>
</div><div><br></div>Please reboot over-9000 server!<br></div>
<br></div></div>_______________________________________________<br>
Pageonexdev mailing list<br>
<a href="mailto:Pageonexdev@mit.edu" target="_blank">Pageonexdev@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/pageonexdev" target="_blank">http://mailman.mit.edu/mailman/listinfo/pageonexdev</a><br>
<br></blockquote></div><span><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><div>Edward L. Platt<br></div><div>Civic Technology Programmer</div><div>MIT Center for Civic Media</div>
<div>E15-348</div><div><a href="http://civic.mit.edu" target="_blank">http://civic.mit.edu</a></div>
<div><a href="http://elplatt.com" target="_blank">http://elplatt.com</a></div><div><a href="http://twitter.com/elplatt" target="_blank">@elplatt</a></div></div>
</font></span></div>
<br>_______________________________________________<br>
Pageonexdev mailing list<br>
<a href="mailto:Pageonexdev@mit.edu" target="_blank">Pageonexdev@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/pageonexdev" target="_blank">http://mailman.mit.edu/mailman/listinfo/pageonexdev</a><br>
<br></blockquote></div><br></div></div></div></div></div></div></div></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><div>Edward L. Platt<br></div><div>Civic Technology Programmer</div><div>MIT Center for Civic Media</div><div>E15-348</div><div><a href="http://civic.mit.edu" target="_blank">http://civic.mit.edu</a></div>
<div><a href="http://elplatt.com" target="_blank">http://elplatt.com</a></div><div><a href="http://twitter.com/elplatt" target="_blank">@elplatt</a></div></div>
</div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><div>Edward L. Platt<br></div><div>Civic Technology Programmer</div><div>MIT Center for Civic Media</div><div>E15-348</div><div><a href="http://civic.mit.edu" target="_blank">http://civic.mit.edu</a></div>
<div><a href="http://elplatt.com" target="_blank">http://elplatt.com</a></div><div><a href="http://twitter.com/elplatt" target="_blank">@elplatt</a></div></div>
</div>
</div></div><br>_______________________________________________<br>
Pageonexdev mailing list<br>
<a href="mailto:Pageonexdev@mit.edu" target="_blank">Pageonexdev@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/pageonexdev" target="_blank">http://mailman.mit.edu/mailman/listinfo/pageonexdev</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
Pageonexdev mailing list<br>
<a href="mailto:Pageonexdev@mit.edu" target="_blank">Pageonexdev@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/pageonexdev" target="_blank">http://mailman.mit.edu/mailman/listinfo/pageonexdev</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
Pageonexdev mailing list<br>
<a href="mailto:Pageonexdev@mit.edu" target="_blank">Pageonexdev@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/pageonexdev" target="_blank">http://mailman.mit.edu/mailman/listinfo/pageonexdev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><div>Edward L. Platt<br></div><div>Civic Technology Programmer</div><div>MIT Center for Civic Media</div><div>E15-348</div><div><a href="http://civic.mit.edu" target="_blank">http://civic.mit.edu</a></div>
<div><a href="http://elplatt.com" target="_blank">http://elplatt.com</a></div><div><a href="http://twitter.com/elplatt" target="_blank">@elplatt</a></div></div>
</div>
</div></div><br>_______________________________________________<br>
Pageonexdev mailing list<br>
<a href="mailto:Pageonexdev@mit.edu" target="_blank">Pageonexdev@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/pageonexdev" target="_blank">http://mailman.mit.edu/mailman/listinfo/pageonexdev</a><br>
<br></blockquote></div><br></div>
</div></div></blockquote></div><br></div>