Adrian Moya's picture

This TKLPatch will convert turnkey-core-beta-lucid-x86 image to a Gitorious appliance setup to work in a private network. I'm not a git user yet but I'm learning quick (to be a cool rails dev). That said, it's important for me that if any community member has experience with git, please give this appliance a test drive and feel free to post bugs / recommendations. 

IMPORTANT NOTE: you must add gitorious.localdomain to your OS hosts file to access this appliance using this url. Hosts file is located:
  Windows: C:\windows\system32\drivers\etc\hosts
  Linux: /etc/hosts
  Mac: ???
  
Just add a line with the appliance IP and the hostname. Ex:
        192.168.1.100   gitorious.localdomain
 

Features:

Deployed on apache using:
  Ruby Enterprise 1.8.7
  Passenger 2.2.15
Apache ActiveMQ
Postfix MTA 
            https://gitorious.localdomain/
Pass: turnkey
 

What it does:

 
1. Sets hostname to gitorious.localdomain
 
# Set hostname
HOSTNAME="gitorious"
echo "$HOSTNAME" > /etc/hostname
sed -i "s|127.0.1.1 \(.*\)|127.0.1.1 $HOSTNAME $HOSTNAME.localdomain|" /etc/hosts
hostname gitorious
 
2. Updates packages information
 
apt-get update
 
3. Download and installs Ruby Enterprise & RubyGems 1.3.7. Creates symlinks to usr/bin
 
# Install Ruby Enterprise and RubyGems
wget http://rubyforge.org/frs/download.php/71100/ruby-enterprise_1.8.7-2010.02_i386_ubuntu10.04.deb
dpkg -i ruby-enterprise_1.8.7-2010.02_i386_ubuntu10.04.deb
rm ruby-enterprise_1.8.7-2010.02_i386_ubuntu10.04.deb
ln -s /usr/local/bin/ruby /usr/bin/ruby
wget http://production.cf.rubygems.org/rubygems/rubygems-1.3.7.tgz
tar xf rubygems-1.3.7.tgz
cd rubygems-1.3.7/
ruby setup.rb
ln -s /usr/local/bin/gem /usr/bin/gem
cd /root/
rm -Rf rubygems-1.3.7/
4. Install apache, git, openjdk (for ActiveMQ), sphinx, mysql-server, Xsendfile apache module (needed by gitorious to work with tars), apg (for random key creation), Memcached for better performance, and other requirements. Gitorious uses sphinx to accelerate database searches, through a gem called ultrasphinx. 
 
# Install required packages
install build-essential apache2-mpm-prefork apache2-prefork-dev libapr1-dev libaprutil1-dev git-core git-svn git-doc git-cvs sphinxsearch postfix memcached mysql-server imagemagick libmagickwand-dev aspell libaspell-dev libapache2-mod-xsendfile jsvc openjdk-6-jre-headless apg
5. Install passenger and apache modules. Configure passenger to use Ruby Enterprise.
 
gem install --no-ri --no-rdoc --version 2.2.15 passenger
/usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.15/bin/passenger-install-apache2-module -a
echo "LoadModule passenger_module /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.15/ext/apache2/mod_passenger.so" > /etc/apache2/mods-available/passenger.load
echo "<IfModule mod_passenger.c>" > /etc/apache2/mods-available/passenger.conf
echo "   PassengerRoot /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.15" >> /etc/apache2/mods-available/passenger.conf
echo "   PassengerRuby /usr/local/bin/ruby" >> /etc/apache2/mods-available/passenger.conf
echo "</IfModule>" >> /etc/apache2/mods-available/passenger.conf
a2enmod passenger
a2enmod rewrite  
a2enmod ssl 
echo "ServerName gitorious.localdomain" > /etc/apache2/httpd.conf
service apache2 restart
6. Download & install Active MQ. Configured to use STOMP as comunication protocol. To use as a service, activemq file is overlayed to /etc/init.d
# Install ActiveMQ
wget -O apache-activemq-5.3.2-bin.tar.gz http://www.carfab.com/apachesoftware/activemq/apache-activemq/5.3.2/apache-activemq-5.3.2-bin.tar.gz
tar xf apache-activemq-5.3.2-bin.tar.gz -C /usr/share/
mv /usr/share/apache-activemq-5.3.2/ /usr/share/activemq
rm apache-activemq-5.3.2-bin.tar.gz
cp /usr/share/activemq/conf/activemq-stomp.xml /usr/share/activemq/conf/activemq.xml
chmod +x /etc/init.d/activemq
7. Install required gems. All gems installed without documentation as this is a production system. Also made a symlink to rake in /usr/bin
# Install required gems
gem install rack --no-rdoc --no-ri --version 1.0.1
gem install rails --no-rdoc --no-ri --version 2.3.8
gem install rdiscount --version "= 1.3.1.1" --no-ri --no-rdoc
gem install stomp --version "= 1.1" --no-ri --no-rdoc
gem install ruby-openid --version ">= 2.0.4" --no-ri --no-rdoc
gem install --no-ri --no-rdoc ultrasphinx raspell mysql rake chronic rmagick json ruby-hmac geoip daemons echoe ruby-yadis ruby-openid mime-types diff-lcs
ln -s /usr/local/lib/ruby/gems/1.8/gems/rake-0.8.7/bin/rake /usr/bin/rake
8. Create user and group git. Added apache user (www-data) to git group.
 
#Create git user and group
adduser --system --home /var/www/gitorious/ --no-create-home --group --shell /bin/bash git   
usermod -a -G git www-data
 
9. Install gitorious by cloning the gitorious repository. Also configured the 3 main files (database.yml, broker.yml, and gitorios.yml, which was previously overlayed to /root/ and it's completed in this step adding the cookie_secret property as recommended by gitorious (using apg)
 
# Install gitorious
git clone git://gitorious.org/gitorious/mainline.git /var/www/gitorious
cd /var/www/gitorious

echo "production:" > config/broker.yml
echo "  adapter: stomp" >> config/broker.yml

echo "production:" > config/database.yml
echo "  adapter: mysql" >> config/database.yml
echo "  database: gitorious_production" >> config/database.yml
echo "  username: root" >> config/database.yml
echo "  password: " >> config/database.yml
echo "  host: localhost" >> config/database.yml
echo "  encoding: utf8" >> config/database.yml

export SECRET=`apg -m 64 | tr -d '\n'`
cat /root/gitorious.yml | sed \
  -e "s/cookie_secret\:.*$/cookie_secret\: $SECRET/g" \
  > /var/www/gitorious/config/gitorious.yml
rm /root/gitorious.yml
10. Migrate production database, bootstrap ultrasphinx gem, and create initial user/pass. The script /root/create_admin_turnkey is a copy of script/create_admin script, modified to add the admin@example.org user and pass turnkey. 
 
sed -i 's/0.0.0.0/localhost/' config/ultrasphinx/default.base
RAILS_ENV=production rake db:create
RAILS_ENV=production rake db:migrate
RAILS_ENV=production rake ultrasphinx:bootstrap
RAILS_ENV=production /root/create_admin_turnkey
rm /root/create_admin_turnkey
 
11. Create various directories needed by the app and set correct ownership and permissions. Also create the gitorious symlink in /usr/bin.
mkdir -p /var/git/git-repos
mkdir -p /var/git/tarballs-cache
mkdir -p /var/git/tarballs-work
mkdir /var/www/gitorious/tmp/pids
mkdir /var/www/gitorious/.ssh    
touch /var/www/gitorious/.ssh/authorized_keys    
chown -R git:git /var/www/gitorious
chown -R git:git /var/git
chmod -R ug+rwx /var/git
chmod 700 /var/www/gitorious/.ssh    
chmod 600 /var/www/gitorious/.ssh/authorized_keys
ln -s /var/www/gitorious/script/gitorious /usr/bin
 
12. Disable default apache website and enable gitorious and gitorious-ssl websites (previously overlayed)
 
a2dissite default
a2ensite gitorious    
a2ensite gitorious-ssl
service apache2 reload
13.Configure overlayed scripts to start at system startup
 
#Make init scripts start on boot
chown git:git /etc/init.d/git-ultrasphinx
chown git:git /etc/init.d/git-daemon
chown git:git /etc/init.d/git-poller
chmod 755 /etc/init.d/git-ultrasphinx /etc/init.d/git-daemon /etc/init.d/git-poller
update-rc.d git-daemon defaults  
update-rc.d git-ultrasphinx defaults  
update-rc.d git-poller defaults
update-rc.d activemq defaults
14. Stop services and cleanup apt cache.
 
#Stop services and cleanup
service mysql stop
service apache2 stop
cleanup_apt

 

Enjoy it! Please send some feedback in this forum. 

Forum: 
Adrian Moya's picture

Hi Rtreffer, thanks for trying the patch. Could you please provide more info on the patching process? yo can do it by issuing the following command:

tklpatch turnkey-core-beta-lucid-x86.iso gitorious.tar.gz | tee tklgit.log

And attach tklgit.log file to see what's happening during the patching process. 

Adrian Moya's picture

Looking at the output you posted, the errors start to show from the moment of migrating Gitorious database 


(Couldn't create database for {"encoding"=>"utf8", "username"=>"root", "adapter"=>"mysql", "database"=>"gitorious_production", "host"=>"localhost", "password"=>nil}, charset: utf8, collation: utf8_unicode_ci (if you set the charset manually, make sure you have a matching collation)
(in /var/www/gitorious)
rake aborted!
Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)

I'll run again this patch in my dev environment. Mysql server seems to have started correctly, as seen in line:

1514: mysql start/running, process 21744

The user home is not created at the moment of the git user creation, and it also doesn't exists at that moment, but after that, the gitorious repository is being cloned at /var/www/gitorious creating that home directory for the user. So that's not the problem, I could create the user after cloning Gitorious and the warning would disappear. I'll update it. Let me check and I'll come back to you in a moment. 

Just for curiosity: did you upgrade tklpatch to its lastest version? Are you patching from within Turnkey Core Lucid or in another environment? 

This is indeed a very complex patch. So many things I was going crazy with each test. Thanks for trying it. 

Jeremy Davis's picture

So I was off track but I appreciate you taking the time to explain where :)

I haven't actually tested your patch, just sticking my newb nose in and getting a bit down and dirty with stuff way over my head! I had a scan through Guest's error code link (above) and found that bit (there may be more?) I did notice that there were references to sub dirs of /var/www/gitorious after the errors but thought perhaps the MySQL failure was somehow related to the path error (not sure how though).

The appliance looks really cool and I will defiantely have a go at it soon hopefully. I think it could be a really cool tool for projects I work on, both at home and at work. Oh so many things I want to do and so little time!

I reckon you'd be going crazy with it! I can handle a bit of basic code/commandline but serious stuff like that does my head in!

Adrian Moya's picture

Here's my log when running the patch: http://pastebin.com/vBbtZkqY

Take a look at this differences:

Yours (starting mysql-server) lines  1514-1515

mysql start/running, process 21744
 

Mine (starting mysql-server) lines 798-801 (fewer lines because I only posted tklpatch-apply output)

 

* Starting MySQL database server mysqld       [157G
[151G[ OK ]
 * Checking for corrupt, not cleanly closed and upgrade needing tables.
The other difference, of course, is during the migration of the DB. Take a look at my output in lines 1339-2153
 
(in /var/www/gitorious)
(in /var/www/gitorious)
==  CreateUsers: migrating ====================================================
-- create_table("users", {:force=>true})
   -> 0.0365s
-- add_index(:users, :login)
   -> 0.0183s
   .... blah blah blah
And finally line 2153:
Admin user created successfully.
 
So there may be something different in your patching environment. I'm using tklpatch version 
 
	dpkg -l tklpatch
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                              Version                           Description
+++-=================================-=================================-==================================================================================
ii  tklpatch                          0.93+5+ga2bd433                   TurnKey Linux Customization Mechanism
Also, try again in a clean (recently booted) system. It could be that your previous attempts left some locked file in the fs that could be causing problems. I hope this helps, please post your results.
Liraz Siri's picture

Finally had a chance to sit down and investigate this patch a bit more thoroughly.

First off, excellent work. This is a complex patch and it looks like a lot of work went into getting everything right. The only things that caught my attention were a few violations of the FHS (Filesystem Hierarchy Standard). This is the standard Debian-based systems are advised to follow with respect to what belongs where (e.g., data, configuration files, local binaries, stuff controlled by package management, etc.).

For example:

  • Re-linking /usr/bin/ruby to a path in /usr/local. Ideally you would do that as a last resort if you can't achieve the same effect by manipulating PATH in the environment.
  • Putting various helper scripts in /root. I'd put them into /tmp if you want to delete them later (I.e., they're temporary), or if it's useful to keep them around /usr/local/bin.

In case you're wondering if this technical bureaucracy serves any other point besides slowing down system integrators, yes it actually does make sense when you take package management into consideration. If you replace /usr/bin/ruby for example, the next package update for ruby will overwrite your change and break the system. Also, if you're making an efficient backup of the system (e.g., TKLBAM) you can save a lot of space by just backing up those directories which contain your local customizations / data and recreating the rest of the system by syncing package management states.

Adrian Moya's picture

First be sure that your host fqdn is properly set. Issue the following command in a terminal: hostname -f (or cat /etc/hosts) correct the appropiate entries if needed. (check step 1 if you need help)

Second, change "ServerName gitorious.localdomain" in /etc/apache2/httpd.conf to myurl.com. You can do this with

sed -i "s/ServerName gitorious.localdomain/ServerName myurl.com/" /etc/apache2/httpd.conf

Third, you have to change the following settings in /var/www/gitorious/config/gitorious.yml:

gitorious_client_host: gitorious.localdomain
gitorious_host: gitorious.localdomain

You can do both changes with:

sed -i "s/gitorious.localdomain/myurl.com/" /var/www/gitorious/config/gitorious.yml

Be sure to backup all files in case anything goes wrong. Please tell me if this works for you. If you loose repos is normal, as gitorious is able to server in multiple domains, and when you make a change, it won't reconize the other domain projects/repos. At least this is what I've read. Thanks for testing the appliance, your feedback is welcome. 

P.S. Restart apache once all changes are made:

service apache2 restart

Adrian Moya's picture

Be warned that public mode will let anyone open an account and create repos. Be sure that's what you want. Private mode means an administrator create accounts/repos. I'll think I'll add a helper script for this

Adrian Moya's picture

It is very possible that this patch become an official appliance, for the new release of Turnkey Linux based on Ubuntu Lucid. But there is no official date when this is going to happen, the Core Devs are in the middle of producing a second beta of the Core, then they'll update the existing appliances, and finally convert some of the patches to official appliances. I'm hoping too this one gets in. 

I encourage you to make a donation if it is in your possibilities, but cannot guarantee that would speed things up, as the team is small and the work to be done is immense. But it surely helps the project to be maintained and this kind of appliances to be finished and available to you and all the community. 

Keep in touch for the evolution of this patch and let's hope it's ready before the year ends. 

Todd Forsberg's picture

I've been following this thread in hopes of the full appliance appearing.  Just wondering if there are any updates on this?  I also, would really LOVE to see this as a full appliance.  GREAT WORK ADRIAN!

-Todd Forsberg

Adrian Moya's picture

Be patient, Turnkey-lucid already have been published and next week a version with some fixes will be available. The second part of the releases will include Gitorious. I'm also waiting for some of my other patches to get to the second batch of releases, I know the tkl guys are working hard to make this happen. 

Meanwhile, have you tryed this version? There's still time for some feedback before the final appliance goes live. 

Reid Ellis's picture

Hi there,

I was going to try applying your patches on the released turnkey-core-11.1-lucid but it's been 9 days since your "be patient" post, so I'm thinking I should just wait for the Gitorious appliance to show up.

Is there a beta release we can help test?

Reid

Jeremy Davis's picture

But soon is relative. Our beloved TKL devs do not like to give clear timeframes as they don't like to dissapoint. Whilst you could try it out (by patching TKL Core) it would be better to wait if you plan to put it into production.

Reid Ellis's picture

I am running "tklpatch turnkey-core-11.1-lucid-x86.iso gitorious.tar.gz" even as I type...

I'll report here if there are any problems.

Reid

Reid Ellis's picture

I uploaded a screenshot to Flickr:

http://www.flickr.com/photos/clith/5388084035/sizes/o/

Looks like I got the exact same result as a_c_m back on Jan 2 <https://img.skitch.com/20110102-duj7ifmku3uafbcph3sde37qqd.jpg>. No patch update over the last 4 weeks?

Reid

Jeremy Davis's picture

I've got no idea why it's doing that (well out of my league). Not sure whether Adrian has plans to update his patch or not though seeing as the TKL guys have started working on Part 2 of the v11 release (which will include Gitorious). I'm guessing from his comments above that he may not be updating it, if that's the case then you'll either need to try debugging it yourself, or wait for the official release.

Adrian Moya's picture

Please do the following: when applying the patch, add an extra > gitorious.log and post it to check for errors during patch.

 

tklpatch turnkey-core-11.1-lucid-x86.iso gitorious.tar.gz > gitorious.log

Also: you can try what the error is suggesting, that is, go to /var/www/gitorious and run 

rake gems:install

This patch was done a while ago, and I haven't tested with the stable releases of turnkey-core. I'm also not updating it as the core tkl-devs will release this appliance soon as part2 of turnkey-lucid. But I have no problem giving you some support in the meantime. 

Reid Ellis's picture

 

It was missing the "hoe" gem, so I did a "gem install hoe" and now I'm getting a screen saying:
 
We are sorry, but something went wrong.
We have been notified about this issue and we will take a look at it shortly.
 
I looked in /var/log/apache2/error.log and it's saying:
 
sendmail: fatal: git(107): No recipient addresses found in message header
postdrop: warning: stdin: unexpected EOF in data, record type 78 length 124
postdrop: fatal: uid=107: malformed input
 
I will try re-doing the patch and save the output for you.
 
Oh, I also did a "rake gems:install" but actually, since there was only a production environment, I had to do "env RAILS_ENV=production rake gems:install" instead :-)
 
Reid
Reid Ellis's picture

(Posted separately to avoid scrunching into the right side of the page!)

I have saved the output from using zsh to execute:

(date; tklpatch turnkey-core-11.1-lucid-x86.iso gitorious.tar.gz; date) |& tee tklpatch-out.txt

You can see it at http://rae.pastebin.com/1AzyJUEK

There are some errors that start around line 2628 and 2634 that might be the problem.

Reid

Adrian Moya's picture

It surprises me it worked at the end for you. For what I can see, the system is unable to start apache as port 80 is in use (1920). Did you run this patch over the previously patched iso? I'll make my test here anyway.

Also, the patch was unable to clone gitorious (2633-2636)

And then, all errors are because of the /var/www/gitorious directory not exists (2637-2668).

Let me run this one locally and I'll come back to you.

Reid Ellis's picture

I always started with a clean ISO, so that wasn't it.

I'm ok for now, since I got it working with the beta release of the base ISO.

Reid

Reid Ellis's picture

Thinking it might be a beta-vs-release issue, I tried using turnkey-core-beta-lucid-x86. However, I also had to do the following:

% export RAILS_ENV=production
% rake gems:install ; # installed "hoe"
% rake db:create
% rake db:migrate
% rake ultrasphinx:bootstrap
% crontab -e

* * * * * cd /var/www/gitorious && /usr/bin/rake ultrasphinx:index RAILS_ENV=production

% /etc/init.d/apache2 restart
% ruby script/create_admin

To get script/console working (for deleiting users, etc) you will also need libreadline

% apt-get install libreadline5

After all that, I had a running Gitorious. I'm not sure I needed *all* of it - I was copying it off of a Gitorious setup page at http://gitorious.org/gitorious/pages/UbuntuInstallation ; I suspect I needed to do some of that with the release version. Maybe I'll give it a try tomorrow.

Reid

[updates:
- did a minor edit on order of commands - had a duplicated db:migreate (it failed because I needed to do the gems:install, after which I did db:migrate again)
- added libreadline5 ]

[Much later update:
- if you change the hostname, along with editing config/citorious.yml be sure to edit /etc/hosts if the domain name is not a real one for the machine. I keep hitting this and forgetting! ]

Reid Ellis's picture

The web front-end seems to work fine, but git operations are failing. Creating a repository never finishes, with the spiner perpetually spinning, and any attempt to access the repository failing as well.

Trying to use the ssh method of access results in being asked for "git"s password.

So.. it looks like more work is needed.

Oh, I did perform the same commands on the release'd base OS as on the beta and I got the same successul results. So it seems that when the OS boots, a number of gitorious initialization steps need to be performed. Perhaps my problems relate to more steps that need to be followed?

Reid

Reid Ellis's picture

I figured out that the problem with git operations was that ~git/script/poller wasn't running. When I ran it manually like this, everything worked:

% cd ~git
% export RAILS_ENV=production
% script/poller run

There is a script at /etc/init.d/git-poller that runs this as a daemon on startup, so after a reboot, it ran automatically and now everything is running correctly. So yay!

Reid

Adrian Moya's picture

This patch is getting old, probably need some updating. But the official appliance is comming soon, is just a matter of waiting. 

Jeremy Davis's picture

So still have to wait for the TKL core devs (Alon & Liraz). As Adrian (the dev of this patch) says just above, this patch is getting quite old now so may require some tweaking to get it to work properly.

Good luck.

Scott Beardsley's picture

Hello, 

I modified the gitorious patch so that it will build a functioning gitorious server from the turnkey-core-11.1-lucid-x86.iso image.

I modified it to use: 

  • ruby-enterprise_1.8.7-2011.03_1386
  • rubygems-1.4.1 (to install hoe)
  • apache-activemq-5.5.0-bin
  • bundler (to install required gems for gitorious)

I also modified it to use bundle exec to perform the rake commands.

I modified git-poller to use "bundle exec" because a gem thats required was the incorrect version and the poller would just shut down.  This was causing the ssh_keys to never be activated and the repository to never be created.

However, I did notice that my call to "/root/create_admin_turnkey" is out of order and it needs to be run manually.  Tomorrow I will try to fix and test that problem.

I wasn't sure how to post files in the comments, feel free to download the file from http://dl.dropbox.com/u/193989/gitorious-11.1-lucid-x86.tar.gz and give it a shot.

If you have any problems let me know.

Jeremy Davis's picture

You can only attach files to the first post of a new thread so just start a new thread and attach your updated patch to it. Then just post a link at the bottom of this thread pointing to your new thread.

Scott Beardsley's picture

Thanks Jeremy, 

I will create a new thread when I think I have all the bugs worked out of the patch.

I am currently reading the webmin module development documentation.  I am considering creating a webmin module to include in the patch so that we can look in webmin to see the poller, sphinx, git-daemon and activemq service status. I think it would also be nice to be able to change the hostname, rotate the logs, etc. all from the webmin module.

Jeremy Davis's picture

That'd be great if you developed a custom Webmin module. The only thing that springs to mind would be when you develop this perhaps consider rather than making just one module; create 2 modules: One with Gitorious specific settings and another a more generic module - that could potentially be included with Core (and thus all TKL appliances).

Scott Beardsley's picture

I renamed the tar file before uploading it, I guess you cant do that 

I have renamed the patch folder. I will bundle, test and push it out to my dropbox shortly.

Update:  I guess my employer blocks the port that apt-get uses.. so I can't test the change until this evening.

In the meantime try:

  • mv gitorious-11.1-lucid-x86.tar.gz gitorious-lucid.tar.gz 
  • tklpatch turnkey-core-11.1-lucid-x86.iso gitorious-lucid.tar.gz

or 

  • tar -xf gitorious-11.1-lucid-x86.tar.gz
  • tklpatch-bundle gitorious-lucid
  • tklpatch turnkey-core-11.1-lucid-x86.iso gitorious-lucid.tar.gz
Scott Beardsley's picture

You should be able to run it by specifying the rails environment

RAILS_ENV=production /root/create_admin_turnkey

Also, I just pushed an updated version to dropbox at the same url.

http://dl.dropbox.com/u/193989/gitorious-11.1-lucid-x86.tar.gz

It should fix the folder structure problem and take care of creating the admin for you as well. (although I haven't had a chance to test it)

Scott Beardsley's picture

Can you give me the details of what its complaining about?  I am not an expert, but I will see what I can figure out.

Scott Beardsley's picture

So there were a couple small problems with that last one.  I have pushed a new version up to my dropbox at the same url:  http://dl.dropbox.com/u/193989/gitorious-11.1-lucid-x86.tar.gz

Just to make sure that I am not doing anything out of the ordinary.. here are the steps I used to patch and build my system.

Here are the steps I use to patch the iso:

  1. Start the turnkey-core-11.1-lucid-x86 VM that I got from here
  2. ssh root@turnkey-core-ip-address
  3. apt-get update
  4. apt-get install tklpatch
  5. mkdir /work
  6. cd /work
  7. tklpatch turnkey-core-11.1-lucid-x86.iso gitorious-11.1-lucid-x86.tar.gz
  8. exit (exit the ssh session)
  9. scp root@turnkey-core-ip-address:/work/turnkey-core-11.1-lucid-x86-patched.iso ~/Documents/Parallels/turnkey-gitorious-11.1-lucid-x86.iso

 

Then I create a new Parallels Virtual Machine booting from the ISO and follow the installation screens.
 
When it has started and I am presented with the Turnkey Configuration Console
  • I open chrome and browse to gitorious.localdomain
  • I log in with 'admin@example.org' and 'turnkey'
  • After the terms of service screen
    • I click on Adminstration
    • I click on Create New User and add my user
    • I then suspend the Admin login and logout
  • Then I login with my user and
    • I click on ~myuser
    • I click on "go to your dashboard"
    • I click on "Manage SSH Keys"
    • I click on "Add SSH Key"
    • I add my key and then refresh the page until the key becomes active.
    • I click on "Projects"
    • I click on "Create a new project"
    • I add the project and git repository following the steps

All of those steps worked fine for me, however I did notice that the sphinx service was not running.  Its because /var/www/gitorious/db/sphinx/* is owned by root.

To fix this problem:

  • ssh root@gitorious.localdomain
  • chown git:git /var/www/gitorious/db/sphinx/
  • chown git:git /var/www/gitorious/db/sphinx/*
  • /etc/init.d/git-ultrasphinx start

After I verified everything is working then I change my hostname to match my fully qualified domain name as Adrian outlined above.

I figure some people will not want this to be an internet facing appliance.  If that is the case, then I recommend changing the hostname to whatever you want and then setup ZeroConf to enable browsing by name without the need to hardcode the hostfile.

ZeroConf is as easy as apt-get install avahi-daemon.  Then you can access your appliance by browsing to https://hostname.local without the need to modify your hosts file or adding a dns server.

Hope all of this works for you

Scott

*******************  You shouldn't need any of this  *********************

Here are some common problems that I have ran into while messing around with gitorious the last 2 days.

  1. ruby error  undefined method `[]' for false:FalseClass
    • usually caused by empty or missing yml files.
  2. SSH Key never activates or Repository never gets created
    • usually caused by poller not running. (shouldn't happen.. I modified /etc/init.d/git-poller to use bundler to start the service which should ensure the correct gems)
  3. misc other issues... just double check the yml files and the permissions.  /var/www/gitorious/** should all be owned by git:git
Reid Ellis's picture

After all this time, all these patches, and the promises of an official appliance .. is there any news on it? Maybe an upload to the SourceForge Turnkey project? If not yet, is there anything we can do to help?

 

P.S. Is there a way to add more tags to this post? Like maybe "git" and/or "gitorious"?

Scott Beardsley's picture

I would be more than happy to push my iso to the Turnkey Community sf project. Who do I contact to give them the iso?

Adrian Moya's picture

First of all thanks for taking time to upgrade the patch and contribute back. That's what makes opensource great! You seem to know very well about the app, contrary to my initial patch, which was a bit of a try-and-fail patch. I'll point you places where you can contribute with the project:

- Webmin modules would be a great addition, if they are simple to implement. But for this appliance, I think it would be more important to code an inithook script to configure the admin password. You can use the create_admin_turnkey as a base for the implementation, and maybe add the hostname setting. For information on this, you can check the inithooks documentation in the site.

- Feel free to update the dev's wiki page for this patch. Update it to point to the new thread were you'll attach your final version of the patch. Also, you gave a lot of useful documentation on how to setup the appliance once is running. That would make a great addition to the documentation wiki. Add it now to the dev's page until the official appliance is out. With this update, surely the core team will have less to worry when implementing the official appliance. BTW, they are currently working hard to make the project a bit more open and accept more outside collaboration, so the lucid releases second batch will probably take some time.

- When you feel your patch is complete, you can contact Jed or myself, and we'll happily create the iso and upload it to the community appliances. I'll suggest Jed right now as my laptop died and I've been a bit with my hands tied up, but I'll hope to get a new one by the end of july. 

Again, thanks for your contribution, it's nice to see someone jump in and give a hand. If you have further questions the general forum is the right place to ask.

Jeremy Davis's picture

So no word on when, although considering part 2 of v11.x release is well overdue hopefully we'll see it sometime soonish.

Reid Ellis's picture

I'm trying to create another Gitorious server (the first is running nicely under VirtualBox on an iMac).

First try didn't go well. I'm using Scott Beardsley's latest patch at http://dl.dropbox.com/u/193989/gitorious-11.1-lucid-x86.tar.gz. There seems to be no /var/www/gitorious directory -- I think the "git clone" failed.

Is there any means for these scripts to handle errors like this gracefully? i.e. tell the user "Can't connect to git server, patch failed"?

Reid Ellis's picture

The "create_admin_turnkey" script in root's home directory doesn't work any more. Gitorious now stores encrypted passwords with salt. So now I have to figure out how to make the password encrypted so that the admin user can be created.

Reid Ellis's picture

Took me a while to figure out I needed to run this to upgrade my gitorious MySQL database.

bundle exec rake db:migrate:up VERSION=20110223130355

That's specific to the version of gitorious I was at (20110223130355). egad.

Reid Ellis's picture

I've finished a successful test run of moving our Gitorious instance to a new install of Gitorious (on machine "newgitorious") using this patch. Here are the key steps I used:

As root on the old VM/machine:

mysqldump --all-databases --add-drop-database --add-drop-table --add-locks | gzip > dev-sql-dump.gz
scp dev-sql-dump.gz root@newgitorious:
rsync -az -e ssh --delete /var/git/ newgitorious:/var/git

On the newgitorious machine:

As root:

apt-get install avahi-daemon
zcat dev-sql-dump.gz | mysql

avahi-daemon allowed me to use ZeroConf so I could use the hostname newgitorious.local without any mucking about with /etc/hosts or similar. I think this should be included with all TKL distributions. Using "applianceName.localhost" is sub-optimal to say the least.

As user "git":

bundle exec rake db:migrate:up VERSION=20110223130355

As mentioned before, this is related to the specific version of Gitorious I was running on the old machine. I really wish the Gitorious dev team would update their real revision number more often. By now it should really be at version 2.7 or something.

Hope this all helps someone else.

Reid Ellis's picture

Hi Bill,

The version number was the least obvious part of all this. It involves the directory db/migrate. On the old machine, the last file in there was 20100316124803_create_archived_events.rb. Looking on the new machine, the very next file was 20110223130355_add_suspended_at_to_projects.rb - I ended up using the "20110223130355" string as my argument to "bundle exec rake db:migrate:up" since otherwise, Rails didn't seem able to migrate. I would have expected to simply do a "bundle exec rake db:migrate" and have everything taken care of, but alas, such was not the case.

I found the "bundle exec rake db:migrate:up" command through a series of google searches frankly.

Hope this helps,

Reid

Reid Ellis's picture

The rsync command took care of copying over the git repositories, and the only other thing I used was mysqldump. I added flags so that I could do it over and over until I got it right though.

One thing that may not be finished are the ssh keys. I think there is a rails command needed to push them into ~git/.ssh ...

Reid

Adrian Moya's picture

I wouldn't say TKL don't have an interest in appliances. What I would say is that they became overwhelmed with the work. I asked the main team to be more open for the project's health, after all, it's an opensource project. What I understood is that the main infraestructure for creating appliances isn't community friendly, and they decided to make an effort to enhance the whole thing so they can be more open to contributions in the future. 

The fact is that as the second batch of appliances (a lot of them almost a year old since they got published), there is no info on timeline/features being developed. I'm sure they are working, and I expect to see the results by the Precise Pangolin release. Meanwhile, we can upload them to the community site.  

I also suggested a site dedicated to tklpatches with a voting system, as I understand it's going to be impossible to review/officialize all patches into appliances, but at least as you suggest, give a space to community to post appliances and be reviewed by other community members. I'm sure we can push as community to see some of these things done, or start something ourselves and later we could get the official support.

Alon Swartz's picture

We are definitely interested in expanding the TurnKey library, but as Adrian mentioned we decided to focus our efforts on moving our build infrastructure to the cloud, which will make it much easier to open up the development process to others and ease our current bottle neck.

We also have plans/ideas of creating a more "developers" orientated section for developing and sharing appliances (and later implement reviews/voting/etc).

BTW, I just came across GitLab (open source clone of GitHub). Would be interesting to see how it compares to Gitorious.

Andrew Stewart's picture

both on the GitLab appliance, more more importantly on the idea of 'Community TKLs'.  

Liraz Siri's picture

Adrian Moya, that nice chap who developed the Gitorious TKLPatch recommended we do Gitlab after using both. He said he found Gitlab more powerful. I think it's based on Gitorious BTW...

Add new comment