[PageOneX] [dev] New round of development in PageOneX
Rahul B
rahulb at media.mit.edu
Wed Jul 25 10:46:23 EDT 2018
Pablo do you currently use the dev installation of PageOneX on our civicdev
server?
Rahul
On Mon, Jul 23, 2018 at 11:19 AM Rahul B <rahulb at media.mit.edu> wrote:
> We are planning to deploy to production as a Docker image under Dokku (
> details <http://dokku.viewdocs.io/dokku/deployment/methods/dockerfiles/>).
>
> I believe we run under Apache with Passenger right now. Since this will
> now be a Docker image the setup script can choose whatever it wants.
>
> Rahul
>
> On Mon, Jul 23, 2018 at 11:11 AM Rafael Porres Molina <rporres at gmail.com>
> wrote:
>
>> Hi everybody,
>>
>> I have a good part of the Dockerize feature to be reviewed. Please take a
>> look: https://github.com/numeroteca/pageonex/pull/213
>>
>> What's missing:
>>
>> * Image mount. I will try to figure out how to control that part once I
>> get to know better the app
>> * CI: I will need to know how we plan to deploy this once it is
>> containerized to see if something like Travis fits here to create the
>> docker images under certain conditions, e.g tag
>> * Run under a proper server. What are we currently using: Unicorn? Puma?
>> Passenger?
>>
>> Cheers,
>>
>> Rafa
>>
>> El mié., 11 jul. 2018 a las 0:40, pablo rey (<pablo at basurama.org>)
>> escribió:
>>
>>> I will review in a fresh install (without docker) if the migration works
>>> and let you know if I have the same problem with the migration.
>>>
>>> The changes in config/database.yml will modify something from the
>>> current way the tool works?
>>>
>>> On 9 July 2018 at 18:36, Rafael Porres Molina <rporres at gmail.com> wrote:
>>>
>>>> See https://github.com/numeroteca/pageonex/issues/209 if you want to
>>>> take a look to what I've done so far. There's a problem with a migration so
>>>> I haven't been able to test it properly. I will continue working at some
>>>> point in the following days.
>>>>
>>>> It is not finished, so I haven't done a pull request. But if you prefer
>>>> that I do it so that you can take a look more easily, please let me know
>>>> and I will create one (tagged with DO NOT MERGE)
>>>>
>>>> Cheers,
>>>>
>>>> Rafa
>>>>
>>>>
>>>>
>>>> El lun., 9 jul. 2018 a las 10:22, Rafael Porres Molina (<
>>>> rporres at gmail.com>) escribió:
>>>>
>>>>> I'll start taking a look and keep you posted.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Rafa
>>>>>
>>>>> El vie., 6 jul. 2018 a las 19:14, Rahul B (<rahulb at media.mit.edu>)
>>>>> escribió:
>>>>>
>>>>>> Yes, a Docker container would work fine. Of course the problem there
>>>>>> is we need persistent storage for the database and the images. Both of
>>>>>> these should be read from environment variables.
>>>>>>
>>>>>> I think this requires a change in the PageOneX configuration code.
>>>>>> Rails already supports specifying a database via a `DATABASE_URL`
>>>>>> environment variable
>>>>>> <http://guides.rubyonrails.org/configuring.html#configuring-a-database>
>>>>>> .
>>>>>>
>>>>>> For docker it looks like you can specific environment variables
>>>>>> either on the command line individually or via a file (docs
>>>>>> <https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file>
>>>>>> ).
>>>>>>
>>>>>> With that in mind, I think the main changes to make are probably to
>>>>>> a) Dockerize it
>>>>>> b) change the image uploading to save images to the path the
>>>>>> environment variable specifies
>>>>>>
>>>>>> Rahul
>>>>>>
>>>>>> On Fri, Jul 6, 2018 at 12:38 PM Rafael Porres Molina <
>>>>>> rporres at gmail.com> wrote:
>>>>>>
>>>>>>> Hi Rahul,
>>>>>>>
>>>>>>> I'm unsure if Pablo will be able to make it this late in the spanish
>>>>>>> afternoon, so I will ask some quesitons here just to understand better
>>>>>>> what's needed.
>>>>>>>
>>>>>>> I guess what it is needed now is to be able to run pageonex in a
>>>>>>> Docker container, am I right?
>>>>>>>
>>>>>>> The other thing that is important is to be able to configure the
>>>>>>> root of the image storage through an env variable so that you can do
>>>>>>> something like
>>>>>>>
>>>>>>> docker run --env ROOT_IMAGE_STORAGE=/blah -v
>>>>>>> /dokku/persistent/storage/pageonex:/blah ....
>>>>>>>
>>>>>>> I don't know how pageonex works internally so I don't know how easy
>>>>>>> is to fix that. But would the following work?
>>>>>>>
>>>>>>> docker run -v /dokku/persistent/storage/pageonex:/
>>>>>>> assets/images/kiosko...
>>>>>>>
>>>>>>> or whatever combination around this? I guess that will depend on
>>>>>>> where the application writes the assets once is running inside the
>>>>>>> container...
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Rafa
>>>>>>>
>>>>>>>
>>>>>>> El vie., 6 jul. 2018 a las 18:12, Rahul B (<rahulb at media.mit.edu>)
>>>>>>> escribió:
>>>>>>>
>>>>>>>> Hi - I have 20 mins now, or in about an hour. If you are still
>>>>>>>> available you can find me on Skype at civic-rahulb.
>>>>>>>>
>>>>>>>> Rahul
>>>>>>>>
>>>>>>>> On Fri, Jul 6, 2018 at 6:36 AM pablo rey <pablo at basurama.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hey Rahul, Let's chat when you are available today. Rafa said that
>>>>>>>>> he will join. He can better help with the containers and image storage
>>>>>>>>> location thing better than me.
>>>>>>>>> Ping me.
>>>>>>>>>
>>>>>>>>> On 5 July 2018 at 15:53, Rahul B <rahulb at media.mit.edu> wrote:
>>>>>>>>>
>>>>>>>>>> Yes - you can deploy a Docker image via Dokku (details
>>>>>>>>>> <http://dokku.viewdocs.io/dokku/deployment/methods/dockerfiles/>).
>>>>>>>>>> That said, deploying as a Heroic build pack is a little simpler.
>>>>>>>>>>
>>>>>>>>>> Regarding the images, the feature suggestion I made (to specify
>>>>>>>>>> the storage location via an environment variable), will let us move the
>>>>>>>>>> image storage to the larger hard drive. That is precisely the goal.
>>>>>>>>>>
>>>>>>>>>> If you'd like to chat I have time today or tomorrow morning
>>>>>>>>>> (Boston time).
>>>>>>>>>>
>>>>>>>>>> Rahul
>>>>>>>>>>
>>>>>>>>>> On Wed, Jul 4, 2018 at 7:07 AM pablo rey <pablo at basurama.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hey Rafa, that'd be great. Thanks!
>>>>>>>>>>>
>>>>>>>>>>> Let's see what Rahul says.
>>>>>>>>>>> Are Docker and Dokku interchangeable?
>>>>>>>>>>>
>>>>>>>>>>> On 4 July 2018 at 12:06, Rafael Porres Molina <rporres at gmail.com
>>>>>>>>>>> > wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hey Pablo,
>>>>>>>>>>>>
>>>>>>>>>>>> This next Monday I can have some hours helping you if you need
>>>>>>>>>>>> to move the application to Docker. Prior to that, if you want we can set a
>>>>>>>>>>>> meeting with Rahul so that the time that I can work on the project is more
>>>>>>>>>>>> focused.
>>>>>>>>>>>>
>>>>>>>>>>>> Let me know if this works for you.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Rafa
>>>>>>>>>>>>
>>>>>>>>>>>> El mié., 4 jul. 2018 a las 0:25, pablo rey (<pablo at basurama.org>)
>>>>>>>>>>>> escribió:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Rahul,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for the good news, happy to continue at civic media
>>>>>>>>>>>>> servers.
>>>>>>>>>>>>>
>>>>>>>>>>>>> We have modest plans for this round of development, let's see
>>>>>>>>>>>>> how we do.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am not familiarized with Dokku and containerized, how
>>>>>>>>>>>>> difficult is to change to that mode of deployment?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regarding image storage, remember you tried in November'17.
>>>>>>>>>>>>> You wrote me off the list "I'm going to copy the giant folder of kiosko
>>>>>>>>>>>>> images to the bigger drive I mentioned again, but this time with a flag to
>>>>>>>>>>>>> maintain all the permissions. Then perhaps we can schedule a time where I
>>>>>>>>>>>>> can flip it to use the larger hard drive and you can make a thread and test
>>>>>>>>>>>>> some old ones to tell me if it works. If it does, all good and I'll delete
>>>>>>>>>>>>> the giant folder on the main drive that is running low on space. If it
>>>>>>>>>>>>> doesn't work then I can just quickly put it back the way it was and think
>>>>>>>>>>>>> harder about it." I think that caused some problems of image
>>>>>>>>>>>>> missing
>>>>>>>>>>>>> <https://github.com/numeroteca/pageonex/issues/203#issuecomment-361261644>,
>>>>>>>>>>>>> still unsolved.
>>>>>>>>>>>>>
>>>>>>>>>>>>> You suggest that Dokku's persistent storage will enable that
>>>>>>>>>>>>> change in storage. We'll need your help to make that shift.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> pablo
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 3 July 2018 at 15:29, Rahul B <rahulb at media.mit.edu> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Pleasure to meet you Juanjo. Glad to see some new life
>>>>>>>>>>>>>> breathing into the project. We host it here and the civic media servers,
>>>>>>>>>>>>>> and I confirmed with Ethan that he is willing to continue to do so.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> * We are using RVM, but are switching to a containerized
>>>>>>>>>>>>>> deployment system soon. That means it'd be best to think about
>>>>>>>>>>>>>> Heroku-style deployment (like PageOneX originally did).
>>>>>>>>>>>>>> * We will be using Dokku <http://dokku.viewdocs.io/dokku/>
>>>>>>>>>>>>>> for containerized deployment to our servers, we you will be able to deploy
>>>>>>>>>>>>>> with a `git push` (once we set keys up). We are migrating our hosted
>>>>>>>>>>>>>> applications to this mode now, so this is super good timing!
>>>>>>>>>>>>>> * Image storage is the only complicated thing to think
>>>>>>>>>>>>>> about... right now the `assets/images/kiosko` folder is 115GB (!). It'd be
>>>>>>>>>>>>>> great if you could prioritize a feature change that lets us specify the
>>>>>>>>>>>>>> root image storage path via a environment variable. That would allow us to
>>>>>>>>>>>>>> use the Dokku persistent storage path
>>>>>>>>>>>>>> <https://github.com/dokku/dokku/blob/master/docs/advanced-usage/persistent-storage.md> to
>>>>>>>>>>>>>> specify a storage point on our giant 10TB network drive while still
>>>>>>>>>>>>>> supporting containerized deployment, rather than having it fill up our root
>>>>>>>>>>>>>> drive (as it does now)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sound good?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Rahul
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Jul 3, 2018 at 4:03 AM pablo rey <pablo at basurama.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hey!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We are back after a long time. We have a few enhancements
>>>>>>>>>>>>>>> and things to fix in our road map (
>>>>>>>>>>>>>>> https://github.com/numeroteca/pageonex/projects/1). they
>>>>>>>>>>>>>>> are basic features to extend the use for an easier and deeper analysis
>>>>>>>>>>>>>>> (drawn areas would be able to store more data, not only the topic). Juanjo
>>>>>>>>>>>>>>> Bazán will be developing in RoR.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Rahul, we have a few doubts to solve regarding deployment
>>>>>>>>>>>>>>> process:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - We are planning to update the ruby version that it's
>>>>>>>>>>>>>>> used. I guess that should be no problem if we are using RVM. Could you
>>>>>>>>>>>>>>> confirm?
>>>>>>>>>>>>>>> - Another related question to the gems. Which is the
>>>>>>>>>>>>>>> deployment process in dev and production servers? is there an script that
>>>>>>>>>>>>>>> executes the bundle/migrations of the database or everything is installed
>>>>>>>>>>>>>>> "manually" or something like capistrano is used?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Down the road, but not yet in the listed in the project, we
>>>>>>>>>>>>>>> are planning to update ruby, gems and bootstrap.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If you want to join, just tell!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> ------------------------------------------
>>>>>>>>>>>>>> Rahul Bhargava
>>>>>>>>>>>>>> http://rahul.connectionlab.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> ------------------------------------------
>>>>>>>>>> Rahul Bhargava
>>>>>>>>>> http://rahul.connectionlab.org
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>> ------------------------------------------
>>>>>>>> Rahul Bhargava
>>>>>>>> http://rahul.connectionlab.org
>>>>>>>>
>>>>>>> --
>>>>>> ------------------------------------------
>>>>>> Rahul Bhargava
>>>>>> http://rahul.connectionlab.org
>>>>>>
>>>>>
>>> --
> ------------------------------------------
> Rahul Bhargava
> http://rahul.connectionlab.org
>
--
------------------------------------------
Rahul Bhargava
http://rahul.connectionlab.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/pageonexdev/attachments/20180725/a5ab4505/attachment-0001.html
More information about the Pageonexdev
mailing list