[PageOneX] [dev] New round of development in PageOneX
Rahul B
rahulb at media.mit.edu
Mon Jul 23 11:19:30 EDT 2018
We are planning to deploy to production as a Docker image under Dokku (
details <http://dokku.viewdocs.io/dokku/deployment/methods/dockerfiles/>).
I believe we run under Apache with Passenger right now. Since this will now
be a Docker image the setup script can choose whatever it wants.
Rahul
On Mon, Jul 23, 2018 at 11:11 AM Rafael Porres Molina <rporres at gmail.com>
wrote:
> Hi everybody,
>
> I have a good part of the Dockerize feature to be reviewed. Please take a
> look: https://github.com/numeroteca/pageonex/pull/213
>
> What's missing:
>
> * Image mount. I will try to figure out how to control that part once I
> get to know better the app
> * CI: I will need to know how we plan to deploy this once it is
> containerized to see if something like Travis fits here to create the
> docker images under certain conditions, e.g tag
> * Run under a proper server. What are we currently using: Unicorn? Puma?
> Passenger?
>
> Cheers,
>
> Rafa
>
> El mié., 11 jul. 2018 a las 0:40, pablo rey (<pablo at basurama.org>)
> escribió:
>
>> I will review in a fresh install (without docker) if the migration works
>> and let you know if I have the same problem with the migration.
>>
>> The changes in config/database.yml will modify something from the current
>> way the tool works?
>>
>> On 9 July 2018 at 18:36, Rafael Porres Molina <rporres at gmail.com> wrote:
>>
>>> See https://github.com/numeroteca/pageonex/issues/209 if you want to
>>> take a look to what I've done so far. There's a problem with a migration so
>>> I haven't been able to test it properly. I will continue working at some
>>> point in the following days.
>>>
>>> It is not finished, so I haven't done a pull request. But if you prefer
>>> that I do it so that you can take a look more easily, please let me know
>>> and I will create one (tagged with DO NOT MERGE)
>>>
>>> Cheers,
>>>
>>> Rafa
>>>
>>>
>>>
>>> El lun., 9 jul. 2018 a las 10:22, Rafael Porres Molina (<
>>> rporres at gmail.com>) escribió:
>>>
>>>> I'll start taking a look and keep you posted.
>>>>
>>>> Cheers,
>>>>
>>>> Rafa
>>>>
>>>> El vie., 6 jul. 2018 a las 19:14, Rahul B (<rahulb at media.mit.edu>)
>>>> escribió:
>>>>
>>>>> Yes, a Docker container would work fine. Of course the problem there
>>>>> is we need persistent storage for the database and the images. Both of
>>>>> these should be read from environment variables.
>>>>>
>>>>> I think this requires a change in the PageOneX configuration code.
>>>>> Rails already supports specifying a database via a `DATABASE_URL`
>>>>> environment variable
>>>>> <http://guides.rubyonrails.org/configuring.html#configuring-a-database>
>>>>> .
>>>>>
>>>>> For docker it looks like you can specific environment variables either
>>>>> on the command line individually or via a file (docs
>>>>> <https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file>
>>>>> ).
>>>>>
>>>>> With that in mind, I think the main changes to make are probably to
>>>>> a) Dockerize it
>>>>> b) change the image uploading to save images to the path the
>>>>> environment variable specifies
>>>>>
>>>>> Rahul
>>>>>
>>>>> On Fri, Jul 6, 2018 at 12:38 PM Rafael Porres Molina <
>>>>> rporres at gmail.com> wrote:
>>>>>
>>>>>> Hi Rahul,
>>>>>>
>>>>>> I'm unsure if Pablo will be able to make it this late in the spanish
>>>>>> afternoon, so I will ask some quesitons here just to understand better
>>>>>> what's needed.
>>>>>>
>>>>>> I guess what it is needed now is to be able to run pageonex in a
>>>>>> Docker container, am I right?
>>>>>>
>>>>>> The other thing that is important is to be able to configure the root
>>>>>> of the image storage through an env variable so that you can do something
>>>>>> like
>>>>>>
>>>>>> docker run --env ROOT_IMAGE_STORAGE=/blah -v
>>>>>> /dokku/persistent/storage/pageonex:/blah ....
>>>>>>
>>>>>> I don't know how pageonex works internally so I don't know how easy
>>>>>> is to fix that. But would the following work?
>>>>>>
>>>>>> docker run -v /dokku/persistent/storage/pageonex:/
>>>>>> assets/images/kiosko...
>>>>>>
>>>>>> or whatever combination around this? I guess that will depend on
>>>>>> where the application writes the assets once is running inside the
>>>>>> container...
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Rafa
>>>>>>
>>>>>>
>>>>>> El vie., 6 jul. 2018 a las 18:12, Rahul B (<rahulb at media.mit.edu>)
>>>>>> escribió:
>>>>>>
>>>>>>> Hi - I have 20 mins now, or in about an hour. If you are still
>>>>>>> available you can find me on Skype at civic-rahulb.
>>>>>>>
>>>>>>> Rahul
>>>>>>>
>>>>>>> On Fri, Jul 6, 2018 at 6:36 AM pablo rey <pablo at basurama.org> wrote:
>>>>>>>
>>>>>>>> Hey Rahul, Let's chat when you are available today. Rafa said that
>>>>>>>> he will join. He can better help with the containers and image storage
>>>>>>>> location thing better than me.
>>>>>>>> Ping me.
>>>>>>>>
>>>>>>>> On 5 July 2018 at 15:53, Rahul B <rahulb at media.mit.edu> wrote:
>>>>>>>>
>>>>>>>>> Yes - you can deploy a Docker image via Dokku (details
>>>>>>>>> <http://dokku.viewdocs.io/dokku/deployment/methods/dockerfiles/>).
>>>>>>>>> That said, deploying as a Heroic build pack is a little simpler.
>>>>>>>>>
>>>>>>>>> Regarding the images, the feature suggestion I made (to specify
>>>>>>>>> the storage location via an environment variable), will let us move the
>>>>>>>>> image storage to the larger hard drive. That is precisely the goal.
>>>>>>>>>
>>>>>>>>> If you'd like to chat I have time today or tomorrow morning
>>>>>>>>> (Boston time).
>>>>>>>>>
>>>>>>>>> Rahul
>>>>>>>>>
>>>>>>>>> On Wed, Jul 4, 2018 at 7:07 AM pablo rey <pablo at basurama.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hey Rafa, that'd be great. Thanks!
>>>>>>>>>>
>>>>>>>>>> Let's see what Rahul says.
>>>>>>>>>> Are Docker and Dokku interchangeable?
>>>>>>>>>>
>>>>>>>>>> On 4 July 2018 at 12:06, Rafael Porres Molina <rporres at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hey Pablo,
>>>>>>>>>>>
>>>>>>>>>>> This next Monday I can have some hours helping you if you need
>>>>>>>>>>> to move the application to Docker. Prior to that, if you want we can set a
>>>>>>>>>>> meeting with Rahul so that the time that I can work on the project is more
>>>>>>>>>>> focused.
>>>>>>>>>>>
>>>>>>>>>>> Let me know if this works for you.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> Rafa
>>>>>>>>>>>
>>>>>>>>>>> El mié., 4 jul. 2018 a las 0:25, pablo rey (<pablo at basurama.org>)
>>>>>>>>>>> escribió:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Rahul,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the good news, happy to continue at civic media
>>>>>>>>>>>> servers.
>>>>>>>>>>>>
>>>>>>>>>>>> We have modest plans for this round of development, let's see
>>>>>>>>>>>> how we do.
>>>>>>>>>>>>
>>>>>>>>>>>> I am not familiarized with Dokku and containerized, how
>>>>>>>>>>>> difficult is to change to that mode of deployment?
>>>>>>>>>>>>
>>>>>>>>>>>> Regarding image storage, remember you tried in November'17.
>>>>>>>>>>>> You wrote me off the list "I'm going to copy the giant folder of kiosko
>>>>>>>>>>>> images to the bigger drive I mentioned again, but this time with a flag to
>>>>>>>>>>>> maintain all the permissions. Then perhaps we can schedule a time where I
>>>>>>>>>>>> can flip it to use the larger hard drive and you can make a thread and test
>>>>>>>>>>>> some old ones to tell me if it works. If it does, all good and I'll delete
>>>>>>>>>>>> the giant folder on the main drive that is running low on space. If it
>>>>>>>>>>>> doesn't work then I can just quickly put it back the way it was and think
>>>>>>>>>>>> harder about it." I think that caused some problems of image
>>>>>>>>>>>> missing
>>>>>>>>>>>> <https://github.com/numeroteca/pageonex/issues/203#issuecomment-361261644>,
>>>>>>>>>>>> still unsolved.
>>>>>>>>>>>>
>>>>>>>>>>>> You suggest that Dokku's persistent storage will enable that
>>>>>>>>>>>> change in storage. We'll need your help to make that shift.
>>>>>>>>>>>>
>>>>>>>>>>>> Best,
>>>>>>>>>>>> pablo
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 3 July 2018 at 15:29, Rahul B <rahulb at media.mit.edu> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Pleasure to meet you Juanjo. Glad to see some new life
>>>>>>>>>>>>> breathing into the project. We host it here and the civic media servers,
>>>>>>>>>>>>> and I confirmed with Ethan that he is willing to continue to do so.
>>>>>>>>>>>>>
>>>>>>>>>>>>> * We are using RVM, but are switching to a containerized
>>>>>>>>>>>>> deployment system soon. That means it'd be best to think about
>>>>>>>>>>>>> Heroku-style deployment (like PageOneX originally did).
>>>>>>>>>>>>> * We will be using Dokku <http://dokku.viewdocs.io/dokku/>
>>>>>>>>>>>>> for containerized deployment to our servers, we you will be able to deploy
>>>>>>>>>>>>> with a `git push` (once we set keys up). We are migrating our hosted
>>>>>>>>>>>>> applications to this mode now, so this is super good timing!
>>>>>>>>>>>>> * Image storage is the only complicated thing to think
>>>>>>>>>>>>> about... right now the `assets/images/kiosko` folder is 115GB (!). It'd be
>>>>>>>>>>>>> great if you could prioritize a feature change that lets us specify the
>>>>>>>>>>>>> root image storage path via a environment variable. That would allow us to
>>>>>>>>>>>>> use the Dokku persistent storage path
>>>>>>>>>>>>> <https://github.com/dokku/dokku/blob/master/docs/advanced-usage/persistent-storage.md> to
>>>>>>>>>>>>> specify a storage point on our giant 10TB network drive while still
>>>>>>>>>>>>> supporting containerized deployment, rather than having it fill up our root
>>>>>>>>>>>>> drive (as it does now)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sound good?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Rahul
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Jul 3, 2018 at 4:03 AM pablo rey <pablo at basurama.org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hey!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We are back after a long time. We have a few enhancements and
>>>>>>>>>>>>>> things to fix in our road map (
>>>>>>>>>>>>>> https://github.com/numeroteca/pageonex/projects/1). they are
>>>>>>>>>>>>>> basic features to extend the use for an easier and deeper analysis (drawn
>>>>>>>>>>>>>> areas would be able to store more data, not only the topic). Juanjo Bazán
>>>>>>>>>>>>>> will be developing in RoR.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Rahul, we have a few doubts to solve regarding deployment
>>>>>>>>>>>>>> process:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - We are planning to update the ruby version that it's
>>>>>>>>>>>>>> used. I guess that should be no problem if we are using RVM. Could you
>>>>>>>>>>>>>> confirm?
>>>>>>>>>>>>>> - Another related question to the gems. Which is the
>>>>>>>>>>>>>> deployment process in dev and production servers? is there an script that
>>>>>>>>>>>>>> executes the bundle/migrations of the database or everything is installed
>>>>>>>>>>>>>> "manually" or something like capistrano is used?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Down the road, but not yet in the listed in the project, we
>>>>>>>>>>>>>> are planning to update ruby, gems and bootstrap.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If you want to join, just tell!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>> ------------------------------------------
>>>>>>>>>>>>> Rahul Bhargava
>>>>>>>>>>>>> http://rahul.connectionlab.org
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> ------------------------------------------
>>>>>>>>> Rahul Bhargava
>>>>>>>>> http://rahul.connectionlab.org
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>> ------------------------------------------
>>>>>>> Rahul Bhargava
>>>>>>> http://rahul.connectionlab.org
>>>>>>>
>>>>>> --
>>>>> ------------------------------------------
>>>>> Rahul Bhargava
>>>>> http://rahul.connectionlab.org
>>>>>
>>>>
>> --
------------------------------------------
Rahul Bhargava
http://rahul.connectionlab.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/pageonexdev/attachments/20180723/28915bdc/attachment-0001.html
More information about the Pageonexdev
mailing list