[PageOneX] [dev] D3 and layered png in display view: bugs

Thu Jun 6 13:55:08 EDT 2013

I fixed a bunch of these.

> PRIORITY Areas show overlaped, though in the coding view they are not:
I found and fixed the problem - it was an underlying data inconsistency issue that we need to address (but I've resolved it for now)

> PRIORITY The transparent png http://pageonex.com/assets/transparent.png used to put the map of links to the different front pages had a grey pixel. 
Did you touch tmp/restart.txt?

> + PRIORITY As it is mentioned in the commented code: rows and columns need padding
I fixed this

> + Uncoded images should render with less more opacity
I fixed this

> + Missing images: when an image could not be scraped they are displayed as white
I fixed this

> + In some 'old' threads are broken: the display view didn't work:
I pulled the production db locally, scraped images for each of those locally, and now they work (perhaps this was related to one of the other issues?) 

> + Some big threads give the following error, when rying to show
I believe this internal server error was a timeout by code taking too long.  My theory points to the poor coding in threadx.results - this method was making HUNDREDS of database calls to compute the data that feeds the datavis. I cleaned it up and it makes a lot less, so the server shouldn't timeout.

Rahul

On Jun 6, 2013, at 2:04 AM, pablo rey wrote:

> I was testing the last big improvements that Rahul and Ed have been working on:
> Changing the bar chart from google charts to d3
> Changing the array of images from an array of single img and areas as div, to a multilayered png.
> From google charts to d3
> It's a great improvement, that will help us build more interactive visualizations: by newspaper, stack bar charts...
> I listed some of the things that need to be fixed at https://github.com/numeroteca/pageonex/issues?milestone=10&page=1&state=open
> 
> From an array of single img and areas as div, to a multilayered png
> In the former version, there were too many http requests (too many big images) and which caused  a slow behavior in the browser. I made some tests and found a bunch of things to solve that I list bellow. I'll create issues as well, tomorrow.
> 
> Over all it looks great, and a great improvement in speed.
> 
> List of bugs and fixes in the display view:
> 
> + PRIORITY Areas show overlaped, though in the coding view they are not:
> Example: Here the layer: http://pageonex.com/assets/threads/40/42/code_87-fc41d8657f1ecd1e5f272a5f3f1695cc.png in this thread http://pageonex.com/matrushka/gezi-parki-protests-in-turkish-newspapers/ The day 6/04 Turkey - Millyet http://pageonex.com/matrushka/gezi-parki-protests-in-turkish-newspapers/coding/?i=milliyet-2013-06-04 (among others) is displayed wrongly in the graph.
> 
> Other layers with wrong overlaping of areas: 
> http://dev.pageonex.com/assets/threads/2/7/code_12.png
> http://pageonex.com/assets/threads/40/42/code_87-fc41d8657f1ecd1e5f272a5f3f1695cc.png
> http://pageonex.com/assets/threads/2/41/code_86-d92bdb3202ad158f598aa269fd384498.png
> 
> I am not able to find a pattern.
> 
> + PRIORITY The transparent png http://pageonex.com/assets/transparent.png used to put the map of links to the different front pages had a grey pixel. It created a grey aea in the top left of the array of images. I removed it 663849a, and solved it in dev. However theimage in production is not http://pageonex.com/assets/transparent.png but http://pageonex.com/assets/transparent-c86f133e8e3505b72c79262e6b82bcb6.png
> 
> + PRIORITY As it is mentioned in the commented code: rows and columns need padding. Now all newspaper are displayed without a pixel of separation. It will need to be variable: a pixel or 3 for long threads, 5-10 for short ones.
> 
> + Uncoded images should render with less more opacity, to show that they are not coded. With the new png version all lookthe same. Look at the example of the Sabah newspaper: https://twitter.com/numeroteca/status/342049105192837120/photo/1
> I think it is a good practice to help coder know what has not yet been coded. It was working well.
> 
> + Missing images: when an image could not be scraped they are displayed as white (not the former 404 message). Despite, we can still make those white spaces linkable. It is useful when user want to request that image again from the coding view.
> 
> + Background with all the newspapers' grid image too big: I downloaded the image and it weighted 5.5Mb. When I resaved it (recompressed?) again as png it was 2.2Mb (in another case it went from 3.2Mb to 1.2Mb), used compression 9 in gimp. Any way to make them be a bit lighter?
> 
> + In some 'old' threads are broken: the display view didn't work:
> Among others it happened in these ones:
>   http://pageonex.com/numeroteca/test-open-thread-for-media-lab-festival/
>   http://pageonex.com/numeroteca/corrupcion-spain-enero-2013/
>   http://pageonex.com/Sasha_Costanza-Chock/drop-the-i-word/
> 
> and they all gave the same error:
> 	Processing by CodingController#display as HTML
> 	  Parameters: {"username"=>"numeroteca", "thread_name"=>"test-open-thread-for-media-lab-festival"}
> 	  Rendered coding/display.html.erb within layouts/application (5.1ms)
> 	Completed 500 Internal Server Error in 10ms
> 
> 	ActionView::Template::Error (undefined method `[]' for nil:NilClass):
> 	    102: <div class="row">
> 	    103: 	<div class="span2">
> 	    104: 		<ul class='thread-coposite-labels'>
> 	    105: 			<% @img_map_info['row_info'].each do |media_id,info| %>
> 	    106: 				<li style="height:<%=info['height']%>px"><%=info['name']%></li>
> 	    107: 			<% end %>
> 	    108: 		</ul>
> 	  app/views/coding/display.html.erb:105:in `_app_views_coding_display_html_erb___3013141180728199937_28041120'
> 
> I found out that editing them and saving solved the problem. Why is this happening? Maybe it happens only to the ones that were copied from the development deployment?
> 
> 
> + Some big threads give the following error, when rying to show i the display view, like this one: http://pageonex.com/Sasha_Costanza-Chock/drop-the-i-word/ 
> 
> 	Internal Server Error
> 	The server encountered an internal error or misconfiguration and was unable to complete your request.
> 	Please contact the server administrator, [no address given] and inform them of the time the error occurred, and anything you might have done that may have caused the error.
> 	More information about this error may be available in the server error log.
> 	Apache/2.2.22 (Ubuntu) Server at pageonex.com Port 80
> 
> _______________________________________________
> Pageonexdev mailing list
> Pageonexdev at mit.edu
> http://mailman.mit.edu/mailman/listinfo/pageonexdev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/pageonexdev/attachments/20130606/32bae1d5/attachment.htm