[PageOneX] [dev] New round of development in PageOneX

Rafael Porres Molina rporres at gmail.com
Mon Jul 23 11:11:04 EDT 2018 

Hi everybody,

I have a good part of the Dockerize feature to be reviewed. Please take a
look: https://github.com/numeroteca/pageonex/pull/213

What's missing:

* Image mount. I will try to figure out how to control that part once I get
to know better the app
* CI: I will need to know how we plan to deploy this once it is
containerized to see if something like Travis fits here to create the
docker images under certain conditions, e.g tag
* Run under a proper server. What are we currently using: Unicorn? Puma?
Passenger?

Cheers,

Rafa

El mié., 11 jul. 2018 a las 0:40, pablo rey (<pablo at basurama.org>) escribió:

> I will review in a fresh install (without docker) if the migration works
> and let you know if I have the same problem with the migration.
>
> The changes in config/database.yml will modify something from the current
> way the tool works?
>
> On 9 July 2018 at 18:36, Rafael Porres Molina <rporres at gmail.com> wrote:
>
>> See https://github.com/numeroteca/pageonex/issues/209 if you want to
>> take a look to what I've done so far. There's a problem with a migration so
>> I haven't been able to test it properly. I will continue working at some
>> point in the following days.
>>
>> It is not finished, so I haven't done a pull request. But if you prefer
>> that I do it so that you can take a look more easily, please let me know
>> and I will create one (tagged with DO NOT MERGE)
>>
>> Cheers,
>>
>> Rafa
>>
>>
>>
>> El lun., 9 jul. 2018 a las 10:22, Rafael Porres Molina (<
>> rporres at gmail.com>) escribió:
>>
>>> I'll start taking a look and keep you posted.
>>>
>>> Cheers,
>>>
>>> Rafa
>>>
>>> El vie., 6 jul. 2018 a las 19:14, Rahul B (<rahulb at media.mit.edu>)
>>> escribió:
>>>
>>>> Yes, a Docker container would work fine. Of course the problem there is
>>>> we need persistent storage for the database and the images.  Both of these
>>>> should be read from environment variables.
>>>>
>>>> I think this requires a change in the PageOneX configuration code.
>>>> Rails already supports specifying a database via a `DATABASE_URL`
>>>> environment variable
>>>> <http://guides.rubyonrails.org/configuring.html#configuring-a-database>
>>>> .
>>>>
>>>> For docker it looks like you can specific environment variables either
>>>> on the command line individually or via a file (docs
>>>> <https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file>
>>>> ).
>>>>
>>>> With that in mind, I think the main changes to make are probably to
>>>> a) Dockerize it
>>>> b) change the image uploading to save images to the path the
>>>> environment variable specifies
>>>>
>>>> Rahul
>>>>
>>>> On Fri, Jul 6, 2018 at 12:38 PM Rafael Porres Molina <rporres at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Rahul,
>>>>>
>>>>> I'm unsure if Pablo will be able to make it this late in the spanish
>>>>> afternoon, so I will ask some quesitons here just to understand better
>>>>> what's needed.
>>>>>
>>>>> I guess what it is needed now is to be able to run pageonex in a
>>>>> Docker container, am I right?
>>>>>
>>>>> The other thing that is important is to be able to configure the root
>>>>> of the image storage through an env variable so that you can do something
>>>>> like
>>>>>
>>>>> docker run --env ROOT_IMAGE_STORAGE=/blah -v
>>>>> /dokku/persistent/storage/pageonex:/blah ....
>>>>>
>>>>> I don't know how pageonex works internally so I don't know how easy is
>>>>> to fix that. But would the following work?
>>>>>
>>>>> docker run -v  /dokku/persistent/storage/pageonex:/
>>>>> assets/images/kiosko...
>>>>>
>>>>> or whatever combination around this? I guess that will depend on where
>>>>> the application writes the assets once is running inside the container...
>>>>>
>>>>> Regards,
>>>>>
>>>>> Rafa
>>>>>
>>>>>
>>>>> El vie., 6 jul. 2018 a las 18:12, Rahul B (<rahulb at media.mit.edu>)
>>>>> escribió:
>>>>>
>>>>>> Hi - I have 20 mins now, or in about an hour.  If you are still
>>>>>> available you can find me on Skype at civic-rahulb.
>>>>>>
>>>>>> Rahul
>>>>>>
>>>>>> On Fri, Jul 6, 2018 at 6:36 AM pablo rey <pablo at basurama.org> wrote:
>>>>>>
>>>>>>> Hey Rahul, Let's chat when you are available today. Rafa said that
>>>>>>> he will join. He can better help with the containers and image storage
>>>>>>> location thing better than me.
>>>>>>> Ping me.
>>>>>>>
>>>>>>> On 5 July 2018 at 15:53, Rahul B <rahulb at media.mit.edu> wrote:
>>>>>>>
>>>>>>>> Yes - you can deploy a Docker image via Dokku (details
>>>>>>>> <http://dokku.viewdocs.io/dokku/deployment/methods/dockerfiles/>).
>>>>>>>> That said, deploying as a Heroic build pack is a little simpler.
>>>>>>>>
>>>>>>>> Regarding the images, the feature suggestion I made (to specify the
>>>>>>>> storage location via an environment variable), will let us move the image
>>>>>>>> storage to the larger hard drive.  That is precisely the goal.
>>>>>>>>
>>>>>>>> If you'd like to chat I have time today or tomorrow morning (Boston
>>>>>>>> time).
>>>>>>>>
>>>>>>>> Rahul
>>>>>>>>
>>>>>>>> On Wed, Jul 4, 2018 at 7:07 AM pablo rey <pablo at basurama.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hey Rafa, that'd be great. Thanks!
>>>>>>>>>
>>>>>>>>> Let's see what Rahul says.
>>>>>>>>> Are Docker and Dokku interchangeable?
>>>>>>>>>
>>>>>>>>> On 4 July 2018 at 12:06, Rafael Porres Molina <rporres at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hey Pablo,
>>>>>>>>>>
>>>>>>>>>> This next Monday I can have some hours helping you if you need to
>>>>>>>>>> move the application to Docker. Prior to that, if you want we can set a
>>>>>>>>>> meeting with Rahul so that the time that I can work on the project is more
>>>>>>>>>> focused.
>>>>>>>>>>
>>>>>>>>>> Let me know if this works for you.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>
>>>>>>>>>> Rafa
>>>>>>>>>>
>>>>>>>>>> El mié., 4 jul. 2018 a las 0:25, pablo rey (<pablo at basurama.org>)
>>>>>>>>>> escribió:
>>>>>>>>>>
>>>>>>>>>>> Hi Rahul,
>>>>>>>>>>>
>>>>>>>>>>> Thanks for the good news, happy to continue at civic media
>>>>>>>>>>> servers.
>>>>>>>>>>>
>>>>>>>>>>> We have modest plans for this round of development, let's see
>>>>>>>>>>> how we do.
>>>>>>>>>>>
>>>>>>>>>>> I am not familiarized with Dokku and containerized, how
>>>>>>>>>>> difficult is to change to that mode of deployment?
>>>>>>>>>>>
>>>>>>>>>>> Regarding image storage, remember you tried in November'17.  You
>>>>>>>>>>> wrote me off the list "I'm going to copy the giant folder of kiosko images
>>>>>>>>>>> to the bigger drive I mentioned again, but this time with a flag to
>>>>>>>>>>> maintain all the permissions. Then perhaps we can schedule a time where I
>>>>>>>>>>> can flip it to use the larger hard drive and you can make a thread and test
>>>>>>>>>>> some old ones to tell me if it works.  If it does, all good and I'll delete
>>>>>>>>>>> the giant folder on the main drive that is running low on space.  If it
>>>>>>>>>>> doesn't work then I can just quickly put it back the way it was and think
>>>>>>>>>>> harder about it." I think that caused some problems of image
>>>>>>>>>>> missing
>>>>>>>>>>> <https://github.com/numeroteca/pageonex/issues/203#issuecomment-361261644>,
>>>>>>>>>>> still unsolved.
>>>>>>>>>>>
>>>>>>>>>>> You suggest that Dokku's persistent storage will enable that
>>>>>>>>>>> change in storage. We'll need your help to make that shift.
>>>>>>>>>>>
>>>>>>>>>>> Best,
>>>>>>>>>>> pablo
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 3 July 2018 at 15:29, Rahul B <rahulb at media.mit.edu> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello,
>>>>>>>>>>>>
>>>>>>>>>>>> Pleasure to meet you Juanjo.  Glad to see some new life
>>>>>>>>>>>> breathing into the project.  We host it here and the civic media servers,
>>>>>>>>>>>> and I confirmed with Ethan that he is willing to continue to do so.
>>>>>>>>>>>>
>>>>>>>>>>>> * We are using RVM, but are switching to a containerized
>>>>>>>>>>>> deployment system soon.  That means it'd be best to think about
>>>>>>>>>>>> Heroku-style deployment (like PageOneX originally did).
>>>>>>>>>>>> * We will be using Dokku <http://dokku.viewdocs.io/dokku/> for
>>>>>>>>>>>> containerized deployment to our servers, we you will be able to deploy with
>>>>>>>>>>>> a `git push` (once we set keys up). We are migrating our hosted
>>>>>>>>>>>> applications to this mode now, so this is super good timing!
>>>>>>>>>>>> * Image storage is the only complicated thing to think about...
>>>>>>>>>>>> right now the `assets/images/kiosko` folder is 115GB (!).  It'd be great if
>>>>>>>>>>>> you could prioritize a feature change that lets us specify the root image
>>>>>>>>>>>> storage path via a environment variable.  That would allow us to use the Dokku
>>>>>>>>>>>> persistent storage path
>>>>>>>>>>>> <https://github.com/dokku/dokku/blob/master/docs/advanced-usage/persistent-storage.md> to
>>>>>>>>>>>> specify a storage point on our giant 10TB network drive while still
>>>>>>>>>>>> supporting containerized deployment, rather than having it fill up our root
>>>>>>>>>>>> drive (as it does now)
>>>>>>>>>>>>
>>>>>>>>>>>> Sound good?
>>>>>>>>>>>>
>>>>>>>>>>>> Rahul
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jul 3, 2018 at 4:03 AM pablo rey <pablo at basurama.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hey!
>>>>>>>>>>>>>
>>>>>>>>>>>>> We are back after a long time. We have a few enhancements and
>>>>>>>>>>>>> things to fix in our road map (
>>>>>>>>>>>>> https://github.com/numeroteca/pageonex/projects/1). they are
>>>>>>>>>>>>> basic features to extend the use for an easier and deeper analysis (drawn
>>>>>>>>>>>>> areas would be able to store more data, not only the topic). Juanjo Bazán
>>>>>>>>>>>>> will be developing in RoR.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Rahul, we have a few doubts to solve regarding deployment
>>>>>>>>>>>>> process:
>>>>>>>>>>>>>
>>>>>>>>>>>>>    - We are planning to update the ruby version that it's
>>>>>>>>>>>>>    used. I guess that should be no problem if we are using RVM. Could you
>>>>>>>>>>>>>    confirm?
>>>>>>>>>>>>>    - Another related question to the gems. Which is the
>>>>>>>>>>>>>    deployment process in dev and production servers? is there an script that
>>>>>>>>>>>>>    executes the bundle/migrations of the database or everything is installed
>>>>>>>>>>>>>    "manually" or  something like capistrano is used?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Down the road, but not yet in the listed in the project, we
>>>>>>>>>>>>> are planning to update ruby, gems and bootstrap.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If you want to join, just tell!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> p
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>> ------------------------------------------
>>>>>>>>>>>> Rahul Bhargava
>>>>>>>>>>>> http://rahul.connectionlab.org
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Pageonexdev mailing list
>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>
>>>>>>>> --
>>>>>>>> ------------------------------------------
>>>>>>>> Rahul Bhargava
>>>>>>>> http://rahul.connectionlab.org
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>> ------------------------------------------
>>>>>> Rahul Bhargava
>>>>>> http://rahul.connectionlab.org
>>>>>>
>>>>> --
>>>> ------------------------------------------
>>>> Rahul Bhargava
>>>> http://rahul.connectionlab.org
>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/pageonexdev/attachments/20180723/9c7d038c/attachment-0001.html

More information about the Pageonexdev mailing list