[PageOneX] [dev] New round of development in PageOneX
Rahul B
rahulb at media.mit.edu
Wed Jul 25 11:27:53 EDT 2018
OK - it is going to go down for a few days while we wipe it clean and
update it to the latest Ubuntu. It is the dev server, so I'm assuming
there is no data the needs preserving on it - correct?
After that it will be ready for you to test out containerized deployment
on. That'll probably happen around 8/6.
Rahul
On Wed, Jul 25, 2018 at 10:52 AM pablo rey <pablo at basurama.org> wrote:
> I have not used it for a while, but I plan to use it for the last updates
> we have developed recently.
>
> On 25 July 2018 at 16:46, Rahul B <rahulb at media.mit.edu> wrote:
>
>> Pablo do you currently use the dev installation of PageOneX on our
>> civicdev server?
>>
>> Rahul
>>
>>
>> On Mon, Jul 23, 2018 at 11:19 AM Rahul B <rahulb at media.mit.edu> wrote:
>>
>>> We are planning to deploy to production as a Docker image under Dokku (
>>> details <http://dokku.viewdocs.io/dokku/deployment/methods/dockerfiles/>
>>> ).
>>>
>>> I believe we run under Apache with Passenger right now. Since this will
>>> now be a Docker image the setup script can choose whatever it wants.
>>>
>>> Rahul
>>>
>>> On Mon, Jul 23, 2018 at 11:11 AM Rafael Porres Molina <rporres at gmail.com>
>>> wrote:
>>>
>>>> Hi everybody,
>>>>
>>>> I have a good part of the Dockerize feature to be reviewed. Please take
>>>> a look: https://github.com/numeroteca/pageonex/pull/213
>>>>
>>>> What's missing:
>>>>
>>>> * Image mount. I will try to figure out how to control that part once I
>>>> get to know better the app
>>>> * CI: I will need to know how we plan to deploy this once it is
>>>> containerized to see if something like Travis fits here to create the
>>>> docker images under certain conditions, e.g tag
>>>> * Run under a proper server. What are we currently using: Unicorn?
>>>> Puma? Passenger?
>>>>
>>>> Cheers,
>>>>
>>>> Rafa
>>>>
>>>> El mié., 11 jul. 2018 a las 0:40, pablo rey (<pablo at basurama.org>)
>>>> escribió:
>>>>
>>>>> I will review in a fresh install (without docker) if the migration
>>>>> works and let you know if I have the same problem with the migration.
>>>>>
>>>>> The changes in config/database.yml will modify something from the
>>>>> current way the tool works?
>>>>>
>>>>> On 9 July 2018 at 18:36, Rafael Porres Molina <rporres at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> See https://github.com/numeroteca/pageonex/issues/209 if you want to
>>>>>> take a look to what I've done so far. There's a problem with a migration so
>>>>>> I haven't been able to test it properly. I will continue working at some
>>>>>> point in the following days.
>>>>>>
>>>>>> It is not finished, so I haven't done a pull request. But if you
>>>>>> prefer that I do it so that you can take a look more easily, please let me
>>>>>> know and I will create one (tagged with DO NOT MERGE)
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Rafa
>>>>>>
>>>>>>
>>>>>>
>>>>>> El lun., 9 jul. 2018 a las 10:22, Rafael Porres Molina (<
>>>>>> rporres at gmail.com>) escribió:
>>>>>>
>>>>>>> I'll start taking a look and keep you posted.
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Rafa
>>>>>>>
>>>>>>> El vie., 6 jul. 2018 a las 19:14, Rahul B (<rahulb at media.mit.edu>)
>>>>>>> escribió:
>>>>>>>
>>>>>>>> Yes, a Docker container would work fine. Of course the problem
>>>>>>>> there is we need persistent storage for the database and the images. Both
>>>>>>>> of these should be read from environment variables.
>>>>>>>>
>>>>>>>> I think this requires a change in the PageOneX configuration code.
>>>>>>>> Rails already supports specifying a database via a `DATABASE_URL`
>>>>>>>> environment variable
>>>>>>>> <http://guides.rubyonrails.org/configuring.html#configuring-a-database>
>>>>>>>> .
>>>>>>>>
>>>>>>>> For docker it looks like you can specific environment variables
>>>>>>>> either on the command line individually or via a file (docs
>>>>>>>> <https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file>
>>>>>>>> ).
>>>>>>>>
>>>>>>>> With that in mind, I think the main changes to make are probably to
>>>>>>>> a) Dockerize it
>>>>>>>> b) change the image uploading to save images to the path the
>>>>>>>> environment variable specifies
>>>>>>>>
>>>>>>>> Rahul
>>>>>>>>
>>>>>>>> On Fri, Jul 6, 2018 at 12:38 PM Rafael Porres Molina <
>>>>>>>> rporres at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Rahul,
>>>>>>>>>
>>>>>>>>> I'm unsure if Pablo will be able to make it this late in the
>>>>>>>>> spanish afternoon, so I will ask some quesitons here just to understand
>>>>>>>>> better what's needed.
>>>>>>>>>
>>>>>>>>> I guess what it is needed now is to be able to run pageonex in a
>>>>>>>>> Docker container, am I right?
>>>>>>>>>
>>>>>>>>> The other thing that is important is to be able to configure the
>>>>>>>>> root of the image storage through an env variable so that you can do
>>>>>>>>> something like
>>>>>>>>>
>>>>>>>>> docker run --env ROOT_IMAGE_STORAGE=/blah -v
>>>>>>>>> /dokku/persistent/storage/pageonex:/blah ....
>>>>>>>>>
>>>>>>>>> I don't know how pageonex works internally so I don't know how
>>>>>>>>> easy is to fix that. But would the following work?
>>>>>>>>>
>>>>>>>>> docker run -v /dokku/persistent/storage/pageonex:/
>>>>>>>>> assets/images/kiosko...
>>>>>>>>>
>>>>>>>>> or whatever combination around this? I guess that will depend on
>>>>>>>>> where the application writes the assets once is running inside the
>>>>>>>>> container...
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> Rafa
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> El vie., 6 jul. 2018 a las 18:12, Rahul B (<rahulb at media.mit.edu>)
>>>>>>>>> escribió:
>>>>>>>>>
>>>>>>>>>> Hi - I have 20 mins now, or in about an hour. If you are still
>>>>>>>>>> available you can find me on Skype at civic-rahulb.
>>>>>>>>>>
>>>>>>>>>> Rahul
>>>>>>>>>>
>>>>>>>>>> On Fri, Jul 6, 2018 at 6:36 AM pablo rey <pablo at basurama.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hey Rahul, Let's chat when you are available today. Rafa said
>>>>>>>>>>> that he will join. He can better help with the containers and image storage
>>>>>>>>>>> location thing better than me.
>>>>>>>>>>> Ping me.
>>>>>>>>>>>
>>>>>>>>>>> On 5 July 2018 at 15:53, Rahul B <rahulb at media.mit.edu> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Yes - you can deploy a Docker image via Dokku (details
>>>>>>>>>>>> <http://dokku.viewdocs.io/dokku/deployment/methods/dockerfiles/>).
>>>>>>>>>>>> That said, deploying as a Heroic build pack is a little simpler.
>>>>>>>>>>>>
>>>>>>>>>>>> Regarding the images, the feature suggestion I made (to specify
>>>>>>>>>>>> the storage location via an environment variable), will let us move the
>>>>>>>>>>>> image storage to the larger hard drive. That is precisely the goal.
>>>>>>>>>>>>
>>>>>>>>>>>> If you'd like to chat I have time today or tomorrow morning
>>>>>>>>>>>> (Boston time).
>>>>>>>>>>>>
>>>>>>>>>>>> Rahul
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jul 4, 2018 at 7:07 AM pablo rey <pablo at basurama.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hey Rafa, that'd be great. Thanks!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Let's see what Rahul says.
>>>>>>>>>>>>> Are Docker and Dokku interchangeable?
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 4 July 2018 at 12:06, Rafael Porres Molina <
>>>>>>>>>>>>> rporres at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hey Pablo,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This next Monday I can have some hours helping you if you
>>>>>>>>>>>>>> need to move the application to Docker. Prior to that, if you want we can
>>>>>>>>>>>>>> set a meeting with Rahul so that the time that I can work on the project is
>>>>>>>>>>>>>> more focused.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Let me know if this works for you.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Rafa
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> El mié., 4 jul. 2018 a las 0:25, pablo rey (<
>>>>>>>>>>>>>> pablo at basurama.org>) escribió:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Rahul,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks for the good news, happy to continue at civic media
>>>>>>>>>>>>>>> servers.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We have modest plans for this round of development, let's
>>>>>>>>>>>>>>> see how we do.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I am not familiarized with Dokku and containerized, how
>>>>>>>>>>>>>>> difficult is to change to that mode of deployment?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regarding image storage, remember you tried in November'17.
>>>>>>>>>>>>>>> You wrote me off the list "I'm going to copy the giant folder of kiosko
>>>>>>>>>>>>>>> images to the bigger drive I mentioned again, but this time with a flag to
>>>>>>>>>>>>>>> maintain all the permissions. Then perhaps we can schedule a time where I
>>>>>>>>>>>>>>> can flip it to use the larger hard drive and you can make a thread and test
>>>>>>>>>>>>>>> some old ones to tell me if it works. If it does, all good and I'll delete
>>>>>>>>>>>>>>> the giant folder on the main drive that is running low on space. If it
>>>>>>>>>>>>>>> doesn't work then I can just quickly put it back the way it was and think
>>>>>>>>>>>>>>> harder about it." I think that caused some problems of
>>>>>>>>>>>>>>> image missing
>>>>>>>>>>>>>>> <https://github.com/numeroteca/pageonex/issues/203#issuecomment-361261644>,
>>>>>>>>>>>>>>> still unsolved.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You suggest that Dokku's persistent storage will enable that
>>>>>>>>>>>>>>> change in storage. We'll need your help to make that shift.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>> pablo
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 3 July 2018 at 15:29, Rahul B <rahulb at media.mit.edu>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Pleasure to meet you Juanjo. Glad to see some new life
>>>>>>>>>>>>>>>> breathing into the project. We host it here and the civic media servers,
>>>>>>>>>>>>>>>> and I confirmed with Ethan that he is willing to continue to do so.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> * We are using RVM, but are switching to a containerized
>>>>>>>>>>>>>>>> deployment system soon. That means it'd be best to think about
>>>>>>>>>>>>>>>> Heroku-style deployment (like PageOneX originally did).
>>>>>>>>>>>>>>>> * We will be using Dokku <http://dokku.viewdocs.io/dokku/>
>>>>>>>>>>>>>>>> for containerized deployment to our servers, we you will be able to deploy
>>>>>>>>>>>>>>>> with a `git push` (once we set keys up). We are migrating our hosted
>>>>>>>>>>>>>>>> applications to this mode now, so this is super good timing!
>>>>>>>>>>>>>>>> * Image storage is the only complicated thing to think
>>>>>>>>>>>>>>>> about... right now the `assets/images/kiosko` folder is 115GB (!). It'd be
>>>>>>>>>>>>>>>> great if you could prioritize a feature change that lets us specify the
>>>>>>>>>>>>>>>> root image storage path via a environment variable. That would allow us to
>>>>>>>>>>>>>>>> use the Dokku persistent storage path
>>>>>>>>>>>>>>>> <https://github.com/dokku/dokku/blob/master/docs/advanced-usage/persistent-storage.md> to
>>>>>>>>>>>>>>>> specify a storage point on our giant 10TB network drive while still
>>>>>>>>>>>>>>>> supporting containerized deployment, rather than having it fill up our root
>>>>>>>>>>>>>>>> drive (as it does now)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Sound good?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Rahul
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Jul 3, 2018 at 4:03 AM pablo rey <
>>>>>>>>>>>>>>>> pablo at basurama.org> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hey!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We are back after a long time. We have a few enhancements
>>>>>>>>>>>>>>>>> and things to fix in our road map (
>>>>>>>>>>>>>>>>> https://github.com/numeroteca/pageonex/projects/1). they
>>>>>>>>>>>>>>>>> are basic features to extend the use for an easier and deeper analysis
>>>>>>>>>>>>>>>>> (drawn areas would be able to store more data, not only the topic). Juanjo
>>>>>>>>>>>>>>>>> Bazán will be developing in RoR.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Rahul, we have a few doubts to solve regarding deployment
>>>>>>>>>>>>>>>>> process:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> - We are planning to update the ruby version that it's
>>>>>>>>>>>>>>>>> used. I guess that should be no problem if we are using RVM. Could you
>>>>>>>>>>>>>>>>> confirm?
>>>>>>>>>>>>>>>>> - Another related question to the gems. Which is the
>>>>>>>>>>>>>>>>> deployment process in dev and production servers? is there an script that
>>>>>>>>>>>>>>>>> executes the bundle/migrations of the database or everything is installed
>>>>>>>>>>>>>>>>> "manually" or something like capistrano is used?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Down the road, but not yet in the listed in the project,
>>>>>>>>>>>>>>>>> we are planning to update ruby, gems and bootstrap.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If you want to join, just tell!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> p
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> ------------------------------------------
>>>>>>>>>>>>>>>> Rahul Bhargava
>>>>>>>>>>>>>>>> http://rahul.connectionlab.org
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Pageonexdev mailing list
>>>>>>>>>>>>> Pageonexdev at mit.edu
>>>>>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/pageonexdev
>>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> ------------------------------------------
>>>>>>>>>>>> Rahul Bhargava
>>>>>>>>>>>> http://rahul.connectionlab.org
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>> ------------------------------------------
>>>>>>>>>> Rahul Bhargava
>>>>>>>>>> http://rahul.connectionlab.org
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>> ------------------------------------------
>>>>>>>> Rahul Bhargava
>>>>>>>> http://rahul.connectionlab.org
>>>>>>>>
>>>>>>>
>>>>> --
>>> ------------------------------------------
>>> Rahul Bhargava
>>> http://rahul.connectionlab.org
>>>
>> --
>> ------------------------------------------
>> Rahul Bhargava
>> http://rahul.connectionlab.org
>>
>
> --
------------------------------------------
Rahul Bhargava
http://rahul.connectionlab.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/pageonexdev/attachments/20180725/fea28324/attachment-0001.html
More information about the Pageonexdev
mailing list