[PageOneX] [dev] New round of development in PageOneX

pablo rey pablo at basurama.org
Wed Jul 25 10:52:09 EDT 2018 

I have not used it for a while, but I plan to use it for the last updates
we have developed recently.

On 25 July 2018 at 16:46, Rahul B <rahulb at media.mit.edu> wrote:

> Pablo do you currently use the dev installation of PageOneX on our
> civicdev server?
>
> Rahul
>
>
> On Mon, Jul 23, 2018 at 11:19 AM Rahul B <rahulb at media.mit.edu> wrote:
>
>> We are planning to deploy to production as a Docker image under Dokku (
>> details <http://dokku.viewdocs.io/dokku/deployment/methods/dockerfiles/>
>> ).
>>
>> I believe we run under Apache with Passenger right now. Since this will
>> now be a Docker image the setup script can choose whatever it wants.
>>
>> Rahul
>>
>> On Mon, Jul 23, 2018 at 11:11 AM Rafael Porres Molina <rporres at gmail.com>
>> wrote:
>>
>>> Hi everybody,
>>>
>>> I have a good part of the Dockerize feature to be reviewed. Please take
>>> a look: https://github.com/numeroteca/pageonex/pull/213
>>>
>>> What's missing:
>>>
>>> * Image mount. I will try to figure out how to control that part once I
>>> get to know better the app
>>> * CI: I will need to know how we plan to deploy this once it is
>>> containerized to see if something like Travis fits here to create the
>>> docker images under certain conditions, e.g tag
>>> * Run under a proper server. What are we currently using: Unicorn? Puma?
>>> Passenger?
>>>
>>> Cheers,
>>>
>>> Rafa
>>>
>>> El mié., 11 jul. 2018 a las 0:40, pablo rey (<pablo at basurama.org>)
>>> escribió:
>>>
>>>> I will review in a fresh install (without docker) if the migration
>>>> works and let you know if I have the same problem with the migration.
>>>>
>>>> The changes in config/database.yml will modify something from the
>>>> current way the tool works?
>>>>
>>>> On 9 July 2018 at 18:36, Rafael Porres Molina <rporres at gmail.com>
>>>> wrote:
>>>>
>>>>> See https://github.com/numeroteca/pageonex/issues/209 if you want to
>>>>> take a look to what I've done so far. There's a problem with a migration so
>>>>> I haven't been able to test it properly. I will continue working at some
>>>>> point in the following days.
>>>>>
>>>>> It is not finished, so I haven't done a pull request. But if you
>>>>> prefer that I do it so that you can take a look more easily, please let me
>>>>> know and I will create one (tagged with DO NOT MERGE)
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Rafa
>>>>>
>>>>>
>>>>>
>>>>> El lun., 9 jul. 2018 a las 10:22, Rafael Porres Molina (<
>>>>> rporres at gmail.com>) escribió:
>>>>>
>>>>>> I'll start taking a look and keep you posted.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Rafa
>>>>>>
>>>>>> El vie., 6 jul. 2018 a las 19:14, Rahul B (<rahulb at media.mit.edu>)
>>>>>> escribió:
>>>>>>
>>>>>>> Yes, a Docker container would work fine. Of course the problem there
>>>>>>> is we need persistent storage for the database and the images.  Both of
>>>>>>> these should be read from environment variables.
>>>>>>>
>>>>>>> I think this requires a change in the PageOneX configuration code.
>>>>>>> Rails already supports specifying a database via a `DATABASE_URL`
>>>>>>> environment variable
>>>>>>> <http://guides.rubyonrails.org/configuring.html#configuring-a-database>
>>>>>>> .
>>>>>>>
>>>>>>> For docker it looks like you can specific environment variables
>>>>>>> either on the command line individually or via a file (docs
>>>>>>> <https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file>
>>>>>>> ).
>>>>>>>
>>>>>>> With that in mind, I think the main changes to make are probably to
>>>>>>> a) Dockerize it
>>>>>>> b) change the image uploading to save images to the path the
>>>>>>> environment variable specifies
>>>>>>>
>>>>>>> Rahul
>>>>>>>
>>>>>>> On Fri, Jul 6, 2018 at 12:38 PM Rafael Porres Molina <
>>>>>>> rporres at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Rahul,
>>>>>>>>
>>>>>>>> I'm unsure if Pablo will be able to make it this late in the
>>>>>>>> spanish afternoon, so I will ask some quesitons here just to understand
>>>>>>>> better what's needed.
>>>>>>>>
>>>>>>>> I guess what it is needed now is to be able to run pageonex in a
>>>>>>>> Docker container, am I right?
>>>>>>>>
>>>>>>>> The other thing that is important is to be able to configure the
>>>>>>>> root of the image storage through an env variable so that you can do
>>>>>>>> something like
>>>>>>>>
>>>>>>>> docker run --env ROOT_IMAGE_STORAGE=/blah -v
>>>>>>>> /dokku/persistent/storage/pageonex:/blah ....
>>>>>>>>
>>>>>>>> I don't know how pageonex works internally so I don't know how easy
>>>>>>>> is to fix that. But would the following work?
>>>>>>>>
>>>>>>>> docker run -v  /dokku/persistent/storage/pageonex:/assets/images/
>>>>>>>> kiosko...
>>>>>>>>
>>>>>>>> or whatever combination around this? I guess that will depend on
>>>>>>>> where the application writes the assets once is running inside the
>>>>>>>> container...
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Rafa
>>>>>>>>
>>>>>>>>
>>>>>>>> El vie., 6 jul. 2018 a las 18:12, Rahul B (<rahulb at media.mit.edu>)
>>>>>>>> escribió:
>>>>>>>>
>>>>>>>>> Hi - I have 20 mins now, or in about an hour.  If you are still
>>>>>>>>> available you can find me on Skype at civic-rahulb.
>>>>>>>>>
>>>>>>>>> Rahul
>>>>>>>>>
>>>>>>>>> On Fri, Jul 6, 2018 at 6:36 AM pablo rey <pablo at basurama.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hey Rahul, Let's chat when you are available today. Rafa said
>>>>>>>>>> that he will join. He can better help with the containers and image storage
>>>>>>>>>> location thing better than me.
>>>>>>>>>> Ping me.
>>>>>>>>>>
>>>>>>>>>> On 5 July 2018 at 15:53, Rahul B <rahulb at media.mit.edu> wrote:
>>>>>>>>>>
>>>>>>>>>>> Yes - you can deploy a Docker image via Dokku (details
>>>>>>>>>>> <http://dokku.viewdocs.io/dokku/deployment/methods/dockerfiles/>).
>>>>>>>>>>> That said, deploying as a Heroic build pack is a little simpler.
>>>>>>>>>>>
>>>>>>>>>>> Regarding the images, the feature suggestion I made (to specify
>>>>>>>>>>> the storage location via an environment variable), will let us move the
>>>>>>>>>>> image storage to the larger hard drive.  That is precisely the goal.
>>>>>>>>>>>
>>>>>>>>>>> If you'd like to chat I have time today or tomorrow morning
>>>>>>>>>>> (Boston time).
>>>>>>>>>>>
>>>>>>>>>>> Rahul
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jul 4, 2018 at 7:07 AM pablo rey <pablo at basurama.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hey Rafa, that'd be great. Thanks!
>>>>>>>>>>>>
>>>>>>>>>>>> Let's see what Rahul says.
>>>>>>>>>>>> Are Docker and Dokku interchangeable?
>>>>>>>>>>>>
>>>>>>>>>>>> On 4 July 2018 at 12:06, Rafael Porres Molina <
>>>>>>>>>>>> rporres at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hey Pablo,
>>>>>>>>>>>>>
>>>>>>>>>>>>> This next Monday I can have some hours helping you if you need
>>>>>>>>>>>>> to move the application to Docker. Prior to that, if you want we can set a
>>>>>>>>>>>>> meeting with Rahul so that the time that I can work on the project is more
>>>>>>>>>>>>> focused.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Let me know if this works for you.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Rafa
>>>>>>>>>>>>>
>>>>>>>>>>>>> El mié., 4 jul. 2018 a las 0:25, pablo rey (<
>>>>>>>>>>>>> pablo at basurama.org>) escribió:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Rahul,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks for the good news, happy to continue at civic media
>>>>>>>>>>>>>> servers.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We have modest plans for this round of development, let's see
>>>>>>>>>>>>>> how we do.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I am not familiarized with Dokku and containerized, how
>>>>>>>>>>>>>> difficult is to change to that mode of deployment?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regarding image storage, remember you tried in November'17.
>>>>>>>>>>>>>> You wrote me off the list "I'm going to copy the giant folder of kiosko
>>>>>>>>>>>>>> images to the bigger drive I mentioned again, but this time with a flag to
>>>>>>>>>>>>>> maintain all the permissions. Then perhaps we can schedule a time where I
>>>>>>>>>>>>>> can flip it to use the larger hard drive and you can make a thread and test
>>>>>>>>>>>>>> some old ones to tell me if it works.  If it does, all good and I'll delete
>>>>>>>>>>>>>> the giant folder on the main drive that is running low on space.  If it
>>>>>>>>>>>>>> doesn't work then I can just quickly put it back the way it was and think
>>>>>>>>>>>>>> harder about it." I think that caused some problems of image
>>>>>>>>>>>>>> missing
>>>>>>>>>>>>>> <https://github.com/numeroteca/pageonex/issues/203#issuecomment-361261644>,
>>>>>>>>>>>>>> still unsolved.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You suggest that Dokku's persistent storage will enable that
>>>>>>>>>>>>>> change in storage. We'll need your help to make that shift.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>> pablo
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 3 July 2018 at 15:29, Rahul B <rahulb at media.mit.edu>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Pleasure to meet you Juanjo.  Glad to see some new life
>>>>>>>>>>>>>>> breathing into the project.  We host it here and the civic media servers,
>>>>>>>>>>>>>>> and I confirmed with Ethan that he is willing to continue to do so.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> * We are using RVM, but are switching to a containerized
>>>>>>>>>>>>>>> deployment system soon.  That means it'd be best to think about
>>>>>>>>>>>>>>> Heroku-style deployment (like PageOneX originally did).
>>>>>>>>>>>>>>> * We will be using Dokku <http://dokku.viewdocs.io/dokku/>
>>>>>>>>>>>>>>> for containerized deployment to our servers, we you will be able to deploy
>>>>>>>>>>>>>>> with a `git push` (once we set keys up). We are migrating our hosted
>>>>>>>>>>>>>>> applications to this mode now, so this is super good timing!
>>>>>>>>>>>>>>> * Image storage is the only complicated thing to think
>>>>>>>>>>>>>>> about... right now the `assets/images/kiosko` folder is 115GB (!).  It'd be
>>>>>>>>>>>>>>> great if you could prioritize a feature change that lets us specify the
>>>>>>>>>>>>>>> root image storage path via a environment variable.  That would allow us to
>>>>>>>>>>>>>>> use the Dokku persistent storage path
>>>>>>>>>>>>>>> <https://github.com/dokku/dokku/blob/master/docs/advanced-usage/persistent-storage.md> to
>>>>>>>>>>>>>>> specify a storage point on our giant 10TB network drive while still
>>>>>>>>>>>>>>> supporting containerized deployment, rather than having it fill up our root
>>>>>>>>>>>>>>> drive (as it does now)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Sound good?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Rahul
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Jul 3, 2018 at 4:03 AM pablo rey <pablo at basurama.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hey!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We are back after a long time. We have a few enhancements
>>>>>>>>>>>>>>>> and things to fix in our road map (https://github.com/
>>>>>>>>>>>>>>>> numeroteca/pageonex/projects/1). they are basic features
>>>>>>>>>>>>>>>> to extend the use for an easier and deeper analysis (drawn areas would be
>>>>>>>>>>>>>>>> able to store more data, not only the topic). Juanjo Bazán will be
>>>>>>>>>>>>>>>> developing in RoR.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Rahul, we have a few doubts to solve regarding deployment
>>>>>>>>>>>>>>>> process:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    - We are planning to update the ruby version that it's
>>>>>>>>>>>>>>>>    used. I guess that should be no problem if we are using RVM. Could you
>>>>>>>>>>>>>>>>    confirm?
>>>>>>>>>>>>>>>>    - Another related question to the gems. Which is the
>>>>>>>>>>>>>>>>    deployment process in dev and production servers? is there an script that
>>>>>>>>>>>>>>>>    executes the bundle/migrations of the database or everything is installed
>>>>>>>>>>>>>>>>    "manually" or  something like capistrano is used?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Down the road, but not yet in the listed in the project, we
>>>>>>>>>>>>>>>> are planning to update ruby, gems and bootstrap.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If you want to join, just tell!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> ------------------------------------------
>>>>>>>>>>>>>>> Rahul Bhargava
>>>>>>>>>>>>>>> http://rahul.connectionlab.org
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> ------------------------------------------
>>>>>>>>>>> Rahul Bhargava
>>>>>>>>>>> http://rahul.connectionlab.org
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>> ------------------------------------------
>>>>>>>>> Rahul Bhargava
>>>>>>>>> http://rahul.connectionlab.org
>>>>>>>>>
>>>>>>>> --
>>>>>>> ------------------------------------------
>>>>>>> Rahul Bhargava
>>>>>>> http://rahul.connectionlab.org
>>>>>>>
>>>>>>
>>>> --
>> ------------------------------------------
>> Rahul Bhargava
>> http://rahul.connectionlab.org
>>
> --
> ------------------------------------------
> Rahul Bhargava
> http://rahul.connectionlab.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/pageonexdev/attachments/20180725/f912b9d7/attachment-0001.html

More information about the Pageonexdev mailing list