Rik Goldman - Sat, 2010/12/25 - 09:55

Update: Atteached is first draft of TKLPatch intended to patch LAPP 11.1 to produce Sahana Eden with Postgres Db backend. The conf script is derived from a script for OSGeo LiveDVD Eden install; the skeleton was provided by Fran Boon, it was revised and subsequently adapted by Maurice Quarles, Jerel Moses, Steven Robinson, and me. Another revision is necessary to provide admin scripts in /usr/local/bin. ISO derived from the patch is available at http://9while9.com.

Update: Attached is first and hopefully last edition of a TKLPatch  intended to patch Core 11.0 to produce a Sahana Eden Lite (Headless) appliance. The default password for the web2py server is admin.

Eden-Lite-Client serves as a headless server as well; however, the user may drop out of confconsole, login, and startx to start LXDE and work with Eden in Firefox. The web2py administrator password is Eden.

At http://9while9.com is a patch to produce from Core 11.0 an ISO with a Sahana Eden develoment environment. Script is bulky because it includes Pydev. Eclipse installs with first boot - along with LXDM. (LXDM prevented the security updates, the Eclipse install, and root password dialog.) WIth that in mind, I haven't attached it below.

If eclipse is installed with the conf script, the patch process doesn't produce and ISO or clean up after itself. So Eclipse installs on first boot. I hope I can resolve this with readers' feedback.

Earlier:

This thread originated here: http://www.turnkeylinux.org/forum/general/20101213/tklpatch-sahana-agast... . Sahana-Eden is FOSS humanitarian disaster/crisis management server technology from the Sahana Foundation.

I've just released for evaluation by the mentoring devs a Sahana-Eden development environment, per blueprint at http://eden.sahanafoundation.org/wiki/BluePrintlivecdinstaller.

It's based on TKL's Core 11.0, which I found at SourceForge (http://sourceforge.net/projects/turnkeylinux/files/). I scripted the install of the packages and much of the configuration of Web2py and Sahana-Eden; I chose to use LXDE for gui interface - it's svelte and efficient.

Then I did the bit that was foreign to me: configured Eclipse to work with Web2py. Added bookmarks to Firefox. Then preconfigured irssi to connect to #sahana-eden on freenode (icing). Made sure to install sudo and give the user account sudo access, and add the user to the web2py group.

Build notes, credentials, usage notes, and the download for VirtualBox are available at http://9while9.com.

The result, I hope, is a fully preconfigured development platform for Sahana-Eden; at the same time, it serves as a full-fledged Sahana-Eden server. Waiting to hear how it's received.

Noted: Core still hangs when it wants to update udev.

Forum: 
General
Tags: 
tklpatch
humanitarian
disaster
development
eden
sahana

Rik Goldman - Sun, 2010/12/26 - 20:31

I'd love to use TKLpatch for the Sahana-Eden dev environment. Manually, the steps involved are these:

  1. Install PyDev through Eclipse:
    1. Help menu > Install New Software
    2. Work with: http://pydev.org/updates
    3. Add, OK
    4. tick PyDev (can exclude the optional PyDev Mylyn Integration), Next, Next, Accept, Finish
    5. tick to trust certificate, Restart now
  2. Set up the Python Interpreter:
    1. Window > Preferences > PyDev > Interpreter - Python
    2. New, Browse to python26/python.exe, OK, OK
  3. Add a new PyDev Project:
    1. File > New > Project > PyDev Project > Next
    2. untick Use Default
    3. Browse to the web2py directory
    4. Give the project a name: "web2py", Next
    5. untick "Create default 'src' folder and add it to the pythonpath"
  4. Debug Configuration: Run > Debug Configurations > Python Run
    1. Name: Sahana Eden
    2. Project: Browse to eden
    3. Main Module: Browse to web2py.py
    4. Arguments tab: Program Arguments: -a password
    5. Apply, Debug

To do this with TKLPatch, I grok that overlays are required. Does anyone know what to put in the overlays folder, and what if anything can be done from the conf script?

Enjoy the rest of the year,

Rik

Jeremy Davis's picture

Jeremy Davis - Mon, 2010/12/27 - 00:19

As per usual, my comments are prefixed with the reminder that I'm no expert in these things and if someone else has a better idea, don't be shy :)

According to this doc installing PyDev is as simple as downloading the zip file from SourceForge and unzipping it to eclipse/dropins. The zip could either be included in the overlay and unzipped by the conf file of the patch, or downloaded and unzipped by the conf file. The downside of this method is that it obviously won't be easily updated. The other steps of installing PyDev (from step 1 in your post - which should make it more easily updated) should be able to be added to a config file somewhere. This doc may help with some ideas around where the config file(s) may be found.

I would imagine that steps 2, 3 & 4 should also adjust config file values somewhere (perhaps the above link will help here too?). It should be as simple as configuring Eclipse manually and then copying the relevant config file(s) to the overlay folder in your patch.

Rik Goldman - Mon, 2010/12/27 - 01:42

I figured something you describe must be the case, but I didn't know where to start. You've lent me a great hand with the links - because of what your eyes saw and mine didn't, i can use wget for pydev and unzip to the destination you found for me. Thanks for giving me a start! First three revisions were half scripted, half configured manually. Here's looking to a patch for the next revision.

Sincerest and most enthusiastic thanks!

Rik

Rik Goldman - Wed, 2010/12/29 - 00:31

The dev environment for Eden is certainly the most challenging patch I've been involved with. Maybe because I don't have a team of students during the winter break. A lot of new things - a python framework for starters. Add various bzr commands, a tangled mess of overlays, adding a repo to source.list, a lot of sed statements I didn't have the skill for before, some wget, shuffling the contents of zip archives, manipulating services. Quite an exercise. A very rewarding challenge while contributing a very worthy project.

I've also completed a TKLPatch for a lightweight eden that relies on web2py with rocket and sqlite - designed to serve headlessly, or to be an Eden workstation. As soon as it's revised, I'll attach it to the original post. On the way to the Eden patches is a web2py patch. I know from the Web2py Google group that a patch for web2py has been brought up in the past. Is this something that was left dangling, or was the whole idea scrubbed? Is it something I should look to attach to the o.p.?

Web2py is a challenge for me itself, because no one seems to agree on where it should go in the filesystem (I'm being mentored to package, so I'm going to have to come to terms with this sooner or later). For those interested, here's the developer's take on the subject. Definitely a tangled knot.

Jeremy Davis's picture

Jeremy Davis - Wed, 2010/12/29 - 01:30

Personally sed does my head in. I have tried reading about it but I just can't quite get my head around it. I have managed to use it a few times but it has always come from reverse engineering something close and being amazed when it actually works as intended!

As for adding repos, rather than edit sources.list I prefer to create a new list file with the new repo. If you drop a text file name.list in /etc/apt/sources.list.d (with the contents being the appropriate repo info). Not only does that make it easier to add a new source using TKLPatch (just put the file in overlay - no sed required) IMO it also makes it easier to see whats happening. BTW it appears that is the default behaviour of Ubuntu when you add a new repo via the GUI.

I don't know enough about Web2py to make a useful comment, but if you pretty much have to make it anyway, why not?

As for filesystem locations, again I don't really know enough to comment. I'm sure Alon, Liraz and Adrian would have some good ideas. I think its great that you're learning about packaging. I have thought that developing and maintaining a package could potentially be a great class/student project. Again another opportunity for open source to get a boost from education, another chance for students to learn in the real world on a real project! :)

Rik Goldman - Thu, 2010/12/30 - 17:50

After a brief exchange with Massimo, the web2py dev, I'm set to work with students on a TKLPatch for web2py. I seeking a collaborator to write the script to set the password for web2py on install - inspired by Alon's post. I know where the script goes in the patch, and I know what it needs to do, but I have no idea how to do it (besides not knowing Python, I also don't know where web2py stores its admin password or what kind of hash, if any, it uses. So I put out a call for collaboration on the web2py google group here: http://groups.google.com/group/web2py/browse_thread/thread/b75d1dadac6c710c.

With luck, we'll produce a patch that will put Alon's post to good use.

(Intent right now is to patch the LAPP stack - if anyone has suggestions about where to put web2py in the filesystem, I'd be eager the hear them.)

Rik

Liraz Siri's picture

Liraz Siri - Tue, 2011/01/04 - 07:45
This is a niche application that wasn't even on our map before Rik started looking into it, but I'm deeply obliged that he has. This is such important work and it's inspiring to think that we could apply open source to its aid.

Regarding the practicalities, we'd like to explore adding Sahana-Eden to part 2 of the release but thanks to the multiple patches we seem to be suffering from an embarrassment of riches. On the face of it, the "lite" headless patch would seem to be the best fit for TurnKey's existing roster of solutions, if only for the sake of consistency with the rest of the library. OTOH, I won't pretend to understand how disaster relief efforts work in practice, the constraints they face and ultimately - what sort of TurnKey based solution would be most helpful "in the field".

It would be useful to understand more about that. I'm also somewhat confused regarding the usage scenario for the Sahana-Eden development environment...

Fran's picture

Fran - Tue, 2011/01/04 - 13:19

For inclusion in the main repo, I'd guess the upcoming multi-user LAPP version would make most sense.

The Lite version is mainly designed as a single-user Offline client to a LAPP server install.

The Dev version is designed to get devs running quickly - this is hopefully on a training course, but can also be for rapid customisation during an emergency deployment.

Thanks for your help with giving us an environment to build these useful tools :)

Fran (Sahana Eden PMC Chair)

Rik Goldman - Tue, 2011/01/04 - 15:23

Realized it would be neighborly to put build notes in the thread for comments and discussion. These are from the first build in a VM prototype. I'll revise when I have the moments. All this and more documentation out our project home.

Build Notes for Sahana Eden Lite (headless)

Overlays

  • /usr/local/bin/update_eden (bzr upgrade, takes optional argument)
  • /usr/local/bin/update_web2py (bzr updgrade, takes optional argument)
  • /usr/local/bin/import (import Eden models to web2py)
  • /etc/confconsole/services.txt (add web2py and Eden)

Set Hostname

# set hostname
HOSTNAME=Sahana-Eden
echo "$HOSTNAME" > /etc/hostname
sed -i "s|127.0.1.1 \(.*\)|127.0.1.1 $HOSTNAME|" /etc/hosts

Install Dependencies

# Install dependences
apt-get update
DEBIAN_FRONTEND=noninteractive apt-get -y \
    -o DPkg::Options::=--force-confdef \
    -o DPkg::Options::=--force-confold \
    install python-lxml python-reportlab python-xlwt python-shapely bzr libgeos-3.1.0

Add User Web2py

#Add and Modify Users
useradd -d web2py
useradd -d -p eden eden
usermod -g web2py:web2py eden

Get Web2py and Eden

#bzr checkout for web2py and eden
bzr checkout --lightweight -r 2717 lp:~mdipierro/web2py/devel /home/web2py
bzr checkout --lightweight -r 1585 lp:sahana-eden /home/web2py/applications/eden

Update Startup Scripts

#handle web2py service as overlay or copy to /etc/rc.d/
cp /home/web2py/scripts/web2py.ubuntu.sh /etc/init.d/web2py
chmod +x /etc/init.d/web2py
#If not using overlay, use sed statements to set username, path to web2py daemon, and startup parameters
update-rc.d web2py defaults

Set Permissions

#Set permissions: be sure base dir is /home/web2py/
#anything wrong with chown -R web2py:web2py /home/web2py/* ?
chown web2py ~web2py/applications/admin/cron
chown web2py ~web2py/applications/admin/cron/crontab
chown web2py ~web2py/applications/admin/cache/
chown web2py ~web2py/applications/admin/databases/
chown web2py ~web2py/applications/admin/errors/
chown web2py ~web2py/applications/admin/sessions/
chown web2py ~web2py/applications/admin/uploads/
cd web2py/applications
chown web2py eden/cron
chown web2py eden/cache/
chown web2py eden/databases/
chown web2py eden/errors/
chown web2py eden/sessions/
chown web2py eden/static/img/markers
chown web2py eden/static/scripts/tools
chown web2py eden/static/styles/S3/sahana.css
chown web2py eden/static/styles/S3/sahana.min.css
chown web2py eden/uploads/
mkdir eden/uploads/gis_cache
mkdir eden/uploads/images
mkdir eden/uploads/tracks
chown web2py eden/uploads/gis_cache
chown web2py eden/uploads/images
chown web2py eden/uploads/tracks

Install Python-Tweepy

echo "deb http://ppa.launchpad.net/chris-lea/python-tweepy/ubuntu lucid main" >> /etc/apt/sources.list.d/sources.list
echo "deb-src http://ppa.launchpad.net/chris-lea/python-tweepy/ubuntu lucid main" >> /etc/apt/sources.list.d/sources.list
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C7917B12
apt-get update
apt-get install python-tweepy

Copy Deployment Templates

cp /home/web2py/applications/eden/deployment_templates/models/000_config.py /home/web2py/applications/eden/models/
cp /home/web2py/applications/eden/deployment_templates/cron/crontab /home/web2py/applications/eden/cron/

Import Eden

#must be executed from the web2py folder
python /home/web2py/web2py.py -N -S eden -M #doesn't work consistently, so repeat.
python /home/web2py/web2py.py -N -S eden -M # doesn't work consistently

Credentials

Web2py Admin Password: admin (set in /etc/init.d/web2py with sed)

Ubuntu accounts:

Username: root

Password: root

Username: web2py

Password: none

Username: dev

Password: eden

Rik Goldman - Sat, 2011/01/29 - 21:57

A draft of a TKLPatch for LAPP 11.1 to produce Sahana-Eden with postgres backend has been attached above. I plan to revise to incorporate administrative scripts in /usr/local/bin. This was a chance to practice test conditions and heredocs in bash, so there are some lines weighing down the bash script unnecessarily but in the name of a learning opportunity. The script was adapted from the one used for the Eden install to the OSGeo liveDVD (original skeleton by Fran Boon, elaborated and revised by Maurice Quarles, Jerel Moses, Steven Robinson, and me). This should, if I understand correctly, finish up the blueprint for the sahana foundation at http://eden.sahanafoundation.org/wiki/BluePrintlivecdinstaller. Feedback welcome!

Rik Goldman - Mon, 2011/02/14 - 01:20

Sahana-Eden over TKL's LAPP 11.0 is complete and attached to the original post. I have some admin scripts to add to /usr/local/bin, but aside from that it's ready to go. Let me know if you still have interest in incorporating this into 11.1 release, if there's still time. I'll rush to get the admin scripts added to overlay.

All the best during these busy times,

Rik

Alon Swartz's picture

Alon Swartz - Mon, 2011/02/14 - 08:38

So what ever still needs to be done, go for it. We'll be releasing part II RC's in batches of probably 5 for testing by the community, so we'll push Sahana Eden to the end of the list until you confirm that the TKLPatch is ready to go.

Rik Goldman - Mon, 2011/03/14 - 01:27

I've finally gotten the scripts into usr/local/bin for Eden LAPP patch. Problem: Geraldo has moved and I can't get it successfully incorporated into the patch.

Otherwise, it's good to go (attached with original post)

Add new comment