You are here
Adrian Moya - Tue, 2010/08/10 - 20:53
This TKLPatch will convert turnkey-core-beta-lucid-x86 image to a Gitorious appliance setup to work in a private network. I'm not a git user yet but I'm learning quick (to be a cool rails dev). That said, it's important for me that if any community member has experience with git, please give this appliance a test drive and feel free to post bugs / recommendations.
IMPORTANT NOTE: you must add gitorious.localdomain to your OS hosts file to access this appliance using this url. Hosts file is located:
Windows: C:\windows\system32\drivers\etc\hosts
Linux: /etc/hosts
Mac: ???
Just add a line with the appliance IP and the hostname. Ex:
192.168.1.100 gitorious.localdomain
Features:
Deployed on apache using:
Ruby Enterprise 1.8.7
Passenger 2.2.15
Apache ActiveMQ
Postfix MTA
Access url: http://gitorious.localdomain/
User: admin@example.org
Pass: turnkey
What it does:
1. Sets hostname to gitorious.localdomain
# Set hostname HOSTNAME="gitorious" echo "$HOSTNAME" > /etc/hostname sed -i "s|127.0.1.1 \(.*\)|127.0.1.1 $HOSTNAME $HOSTNAME.localdomain|" /etc/hosts hostname gitorious
2. Updates packages information
apt-get update
3. Download and installs Ruby Enterprise & RubyGems 1.3.7. Creates symlinks to usr/bin
# Install Ruby Enterprise and RubyGems wget http://rubyforge.org/frs/download.php/71100/ruby-enterprise_1.8.7-2010.02_i386_ubuntu10.04.deb dpkg -i ruby-enterprise_1.8.7-2010.02_i386_ubuntu10.04.deb rm ruby-enterprise_1.8.7-2010.02_i386_ubuntu10.04.deb ln -s /usr/local/bin/ruby /usr/bin/ruby wget http://production.cf.rubygems.org/rubygems/rubygems-1.3.7.tgz tar xf rubygems-1.3.7.tgz cd rubygems-1.3.7/ ruby setup.rb ln -s /usr/local/bin/gem /usr/bin/gem cd /root/ rm -Rf rubygems-1.3.7/
4. Install apache, git, openjdk (for ActiveMQ), sphinx, mysql-server, Xsendfile apache module (needed by gitorious to work with tars), apg (for random key creation), Memcached for better performance, and other requirements. Gitorious uses sphinx to accelerate database searches, through a gem called ultrasphinx.
# Install required packages install build-essential apache2-mpm-prefork apache2-prefork-dev libapr1-dev libaprutil1-dev git-core git-svn git-doc git-cvs sphinxsearch postfix memcached mysql-server imagemagick libmagickwand-dev aspell libaspell-dev libapache2-mod-xsendfile jsvc openjdk-6-jre-headless apg
5. Install passenger and apache modules. Configure passenger to use Ruby Enterprise.
gem install --no-ri --no-rdoc --version 2.2.15 passenger /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.15/bin/passenger-install-apache2-module -a echo "LoadModule passenger_module /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.15/ext/apache2/mod_passenger.so" > /etc/apache2/mods-available/passenger.load echo "<IfModule mod_passenger.c>" > /etc/apache2/mods-available/passenger.conf echo " PassengerRoot /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.15" >> /etc/apache2/mods-available/passenger.conf echo " PassengerRuby /usr/local/bin/ruby" >> /etc/apache2/mods-available/passenger.conf echo "</IfModule>" >> /etc/apache2/mods-available/passenger.conf a2enmod passenger a2enmod rewrite a2enmod ssl echo "ServerName gitorious.localdomain" > /etc/apache2/httpd.conf service apache2 restart
6. Download & install Active MQ. Configured to use STOMP as comunication protocol. To use as a service, activemq file is overlayed to /etc/init.d
# Install ActiveMQ wget -O apache-activemq-5.3.2-bin.tar.gz http://www.carfab.com/apachesoftware/activemq/apache-activemq/5.3.2/apache-activemq-5.3.2-bin.tar.gz tar xf apache-activemq-5.3.2-bin.tar.gz -C /usr/share/ mv /usr/share/apache-activemq-5.3.2/ /usr/share/activemq rm apache-activemq-5.3.2-bin.tar.gz cp /usr/share/activemq/conf/activemq-stomp.xml /usr/share/activemq/conf/activemq.xml chmod +x /etc/init.d/activemq
7. Install required gems. All gems installed without documentation as this is a production system. Also made a symlink to rake in /usr/bin
# Install required gems gem install rack --no-rdoc --no-ri --version 1.0.1 gem install rails --no-rdoc --no-ri --version 2.3.8 gem install rdiscount --version "= 1.3.1.1" --no-ri --no-rdoc gem install stomp --version "= 1.1" --no-ri --no-rdoc gem install ruby-openid --version ">= 2.0.4" --no-ri --no-rdoc gem install --no-ri --no-rdoc ultrasphinx raspell mysql rake chronic rmagick json ruby-hmac geoip daemons echoe ruby-yadis ruby-openid mime-types diff-lcs ln -s /usr/local/lib/ruby/gems/1.8/gems/rake-0.8.7/bin/rake /usr/bin/rake
8. Create user and group git. Added apache user (www-data) to git group.
#Create git user and group adduser --system --home /var/www/gitorious/ --no-create-home --group --shell /bin/bash git usermod -a -G git www-data
9. Install gitorious by cloning the gitorious repository. Also configured the 3 main files (database.yml, broker.yml, and gitorios.yml, which was previously overlayed to /root/ and it's completed in this step adding the cookie_secret property as recommended by gitorious (using apg)
# Install gitorious git clone git://gitorious.org/gitorious/mainline.git /var/www/gitorious cd /var/www/gitorious echo "production:" > config/broker.yml echo " adapter: stomp" >> config/broker.yml echo "production:" > config/database.yml echo " adapter: mysql" >> config/database.yml echo " database: gitorious_production" >> config/database.yml echo " username: root" >> config/database.yml echo " password: " >> config/database.yml echo " host: localhost" >> config/database.yml echo " encoding: utf8" >> config/database.yml export SECRET=`apg -m 64 | tr -d '\n'` cat /root/gitorious.yml | sed \ -e "s/cookie_secret\:.*$/cookie_secret\: $SECRET/g" \ > /var/www/gitorious/config/gitorious.yml rm /root/gitorious.yml
10. Migrate production database, bootstrap ultrasphinx gem, and create initial user/pass. The script /root/create_admin_turnkey is a copy of script/create_admin script, modified to add the admin@example.org user and pass turnkey.
sed -i 's/0.0.0.0/localhost/' config/ultrasphinx/default.base RAILS_ENV=production rake db:create RAILS_ENV=production rake db:migrate RAILS_ENV=production rake ultrasphinx:bootstrap RAILS_ENV=production /root/create_admin_turnkey rm /root/create_admin_turnkey
11. Create various directories needed by the app and set correct ownership and permissions. Also create the gitorious symlink in /usr/bin.
mkdir -p /var/git/git-repos mkdir -p /var/git/tarballs-cache mkdir -p /var/git/tarballs-work mkdir /var/www/gitorious/tmp/pids mkdir /var/www/gitorious/.ssh touch /var/www/gitorious/.ssh/authorized_keys chown -R git:git /var/www/gitorious chown -R git:git /var/git chmod -R ug+rwx /var/git chmod 700 /var/www/gitorious/.ssh chmod 600 /var/www/gitorious/.ssh/authorized_keys ln -s /var/www/gitorious/script/gitorious /usr/bin
12. Disable default apache website and enable gitorious and gitorious-ssl websites (previously overlayed)
a2dissite default a2ensite gitorious a2ensite gitorious-ssl service apache2 reload
13.Configure overlayed scripts to start at system startup
#Make init scripts start on boot chown git:git /etc/init.d/git-ultrasphinx chown git:git /etc/init.d/git-daemon chown git:git /etc/init.d/git-poller chmod 755 /etc/init.d/git-ultrasphinx /etc/init.d/git-daemon /etc/init.d/git-poller update-rc.d git-daemon defaults update-rc.d git-ultrasphinx defaults update-rc.d git-poller defaults update-rc.d activemq defaults
14. Stop services and cleanup apt cache.
#Stop services and cleanup service mysql stop service apache2 stop cleanup_apt
Enjoy it! Please send some feedback in this forum.
Forum:
Somehow doesn't work for
Somehow doesn't work for me.... First run on a fresh unsquashfs fails with "passanger.so not found", consecutive runs fail with "can't connect to mysql through socket"....
Please give more info to help you
Hi Rtreffer, thanks for trying the patch. Could you please provide more info on the patching process? yo can do it by issuing the following command:
And attach tklgit.log file to see what's happening during the patching process.
I'll try that, I've so far
I'll try that, I've so far patched an unsquashfs'ed 10root.squashfs with tklapply-patch /mnt/ /home/root/gitorious/
Direkt tklpatch is slightly better, here is the output: http://pastebin.com/G8yTqg8j
I should point out that the
I should point out that the mysql user creation fails, this is critical as it prevents any logins.
Just had a quick browse
Warning: The home dir /var/www/gitorious/ you specified can't be accessed: No such file or directory Adding system user `git' (UID 107) ... Adding new group `git' (GID 111) ... Adding new user `git' (UID 107) with group `git' ... Not creating home directory `/var/www/gitorious/'. Initialized empty Git repository in /var/www/gitorious/.git/ Couldn't create database for {"encoding"=>"utf8", "username"=>"root", "adapter"=>"mysql", "database"=>"gitorious_production", "host"=>"localhost", "password"=>nil}, charset: utf8, collation: utf8_unicode_ci (if you set the charset manually, make sure you have a matching collation) (in /var/www/gitorious) rake aborted! Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)Could it be as simple as the fact its not creating /var/www/gitorious ? Why would it not be creating the dir? I'm assuming that it should be created as the user 's git home dir?
Sorry more questions than answers :)
Problem solved after re-installing mysql
Re-install mysql-server and mysql-client using this command:
and then some libraries you need to install to make MySQL available to ruby:
This all solved my problem. Try it !!! :)
The problem is at the MySQL database creation.
Looking at the output you posted, the errors start to show from the moment of migrating Gitorious database
I'll run again this patch in my dev environment. Mysql server seems to have started correctly, as seen in line:
1514: mysql start/running, process 21744The user home is not created at the moment of the git user creation, and it also doesn't exists at that moment, but after that, the gitorious repository is being cloned at /var/www/gitorious creating that home directory for the user. So that's not the problem, I could create the user after cloning Gitorious and the warning would disappear. I'll update it. Let me check and I'll come back to you in a moment.
Just for curiosity: did you upgrade tklpatch to its lastest version? Are you patching from within Turnkey Core Lucid or in another environment?
This is indeed a very complex patch. So many things I was going crazy with each test. Thanks for trying it.
I saw the MySQL err and thought it may have related to path err
So I was off track but I appreciate you taking the time to explain where :)
I haven't actually tested your patch, just sticking my newb nose in and getting a bit down and dirty with stuff way over my head! I had a scan through Guest's error code link (above) and found that bit (there may be more?) I did notice that there were references to sub dirs of /var/www/gitorious after the errors but thought perhaps the MySQL failure was somehow related to the path error (not sure how though).
The appliance looks really cool and I will defiantely have a go at it soon hopefully. I think it could be a really cool tool for projects I work on, both at home and at work. Oh so many things I want to do and so little time!
I reckon you'd be going crazy with it! I can handle a bit of basic code/commandline but serious stuff like that does my head in!
It's working for me
Here's my log when running the patch: http://pastebin.com/vBbtZkqY
Take a look at this differences:
Yours (starting mysql-server) lines 1514-1515
Mine (starting mysql-server) lines 798-801 (fewer lines because I only posted tklpatch-apply output)
(in /var/www/gitorious) (in /var/www/gitorious) == CreateUsers: migrating ==================================================== -- create_table("users", {:force=>true}) -> 0.0365s -- add_index(:users, :login) -> 0.0183s .... blah blah blahThanks for the suggestions,
Thanks for the suggestions, I'm retrying the patch right now.
mysql is indeed installed on the host but was offline (ps aux|grep [m]y -> no results), I've just rebooted and will try to patch again. Host OS is maverick right now, with tklpatch 0.93 (from the lucid repository, no +5-ga... though).
Awesome work. A few minor comments
Finally had a chance to sit down and investigate this patch a bit more thoroughly.
First off, excellent work. This is a complex patch and it looks like a lot of work went into getting everything right. The only things that caught my attention were a few violations of the FHS (Filesystem Hierarchy Standard). This is the standard Debian-based systems are advised to follow with respect to what belongs where (e.g., data, configuration files, local binaries, stuff controlled by package management, etc.).
For example:
In case you're wondering if this technical bureaucracy serves any other point besides slowing down system integrators, yes it actually does make sense when you take package management into consideration. If you replace /usr/bin/ruby for example, the next package update for ruby will overwrite your change and break the system. Also, if you're making an efficient backup of the system (e.g., TKLBAM) you can save a lot of space by just backing up those directories which contain your local customizations / data and recreating the rest of the system by syncing package management states.
Ruby Enterprise Edition in BrightBox.Net APT Repository
There is actually no need at all to install Ruby Enterprise Edition by hand and symlinking it. BrightBox (a Ruby on Rails hosting company) provides an APT Repository at http://APT.BrightBox.Net/ with Ruby Enterprise Edition and many Rails-related Debian packages (like the Passenger module for Apache or a patched version of NGinX with the Passenger module NGinX). Their REE package installs into
/usr/binjust like any other Debian package. And it provides therubypackage, which means you can simply install it withaptitude install rubyand it will get REE, since you can be pretty much guaranteed that BrightBox’s packages will always be newer than Debian’s.This is awesome
This is a very good tutorial after searching ing google just too many times for a working recipe for gitorious finally.
Excellent work man!
Thanks!
I know what you're talking about google! I read almost 3 or 4 to come to this. Just remember that being a TKLPatch, some files must be overlayed (copied) to the file system before running this steps. You could wait to see if this patch gets converted to an official appliance in the library, which will make it easier to install.
How can I apply the patch?
How can I apply the patch?
There is no documentation about it but it's fairly easy
I checked the documentation and it seems there's no tutorial to apply a patch. But it's easy, follow the directions of JedMeister in this thread http://www.turnkeylinux.org/forum/general/20100819/turnkey-linux-and-tklpatch and if you find yourself lost feel free to ask for more help.
Wrote a tutorial on how to apply a patch
I wrote a tutorial on how to apply a TKLPatch. Seems lots of people have had trouble applying patches so that (along with Adrian's comment) made me suspect the existing documentation is inadequate. Hope this will clear things up.
Certainly doesn't hurt to tidy it up a bit
But why the recommendation to uncompress prior to application of patch? Personally I think it would be a better idea to clarify in the patch construction and the importance of using the tklpatch-bundle command to create your patch. This way the potential pitfalls for users applying the patch are avoided and it makes it less daunting for TKL/Linux newbs (all you have to do is tklpatch tklimage.iso tklpatch.tar.gz). Perhaps it'd be better to create a new file extension for tklpatches such as .tklp or .patch (or similar etc)? Obviously it'd still be a .tar.gz behind the scenes, just with an alternate extension. I know its only one small line of code, I just think newbs can be turned off pretty easily.
Also for ease of use (and reduce the likelihood of issues) I'd be inclined to keep the tutorial really basic with simple instructions for patching ISOs. All other options can be listed in the full documentation. Basically keep the 'quick ref for newbs' really basic and clean, with fuller explanation held back for the 'full' documentation.
Also while I'm meandering off topic all over the forums tonight (my time) I'd like to add that I'd really love to see some sort of TKLPatch database/table type page where all the patches are in the one place and basically outlined (like they are/were on the dev wiki) but with them available for download irght there too. This could also mention if they have been published as an appliance or not and users could add info on which patches they had successfully applied to which appliances (sort of mix and match as many have asked for here previously). I would suggest that this usage scenario not be 'officially' supported but troubleshooting etc could be welcome in the forums for those interested (similar to what Proxmox do for certain features they do not wish to support - discussion is welcome on the forums, but the devs rarely involve themselves).
Removed the extract section
The extra doc page was definitely needed. I also noticed the "extract tklpatch" part, and agree it's not necessary. We don't want to encourage bundling and patching of tklpatches adhoc style (that's why tklpatch-bundle exists, and patching from the tarball as well).
Weird errors during installation
I get these errors when the install procedure reaches something between 70 and 80% idk..
http://image-upload.de/image/rIoIsV/f315cac258.png
After waiting some time, the install finished though. And it "seems" to work. But I wanted to mention it.
I used this command, sorry I don't have the tklgit.log anymore :(
What was your patching environment?
Maybe it has something to do with your patching environment, do you have a postfix running on the system where you applied the patch? It's preferably to patch in an isolate environment (a VM with TKL Lucid Core) to prevent errors. You can also try (if you have a postfix running) to stop the service before patching.
Just for peace of mind, it's better to repatch without errors so if you get into any issue in the future, we don't suspect of these errors during patching.
Thanks for reporting!
Almost works: I've repatched it
I've repatched it in the same vm (no postfix) and it works now.
hmm I encounter one problem when I actually want to use gitorious in production.
So when I enter "http://myurl.com" it redirects me to "http://gitorious.localdomain".
Could you help me solving that please?
Yes, check these points:
First be sure that your host fqdn is properly set. Issue the following command in a terminal: hostname -f (or cat /etc/hosts) correct the appropiate entries if needed. (check step 1 if you need help)
Second, change "ServerName gitorious.localdomain" in /etc/apache2/httpd.conf to myurl.com. You can do this with
Third, you have to change the following settings in /var/www/gitorious/config/gitorious.yml:
You can do both changes with:
Be sure to backup all files in case anything goes wrong. Please tell me if this works for you. If you loose repos is normal, as gitorious is able to server in multiple domains, and when you make a change, it won't reconize the other domain projects/repos. At least this is what I've read. Thanks for testing the appliance, your feedback is welcome.
P.S. Restart apache once all changes are made:
Found it.
OK. I found it. For all others you have to edit /var/www/gitorious/config/gitorious.yml
Thanks!
Be warned that public mode will let anyone open an account and create repos. Be sure that's what you want. Private mode means an administrator create accounts/repos. I'll think I'll add a helper script for this
For what its worth, i would
For what its worth, i would LOVE to see this as a appliance, perhaps as part of an updated http://www.turnkeylinux.org/revision-control
Would a bounty or donation help speed that along? Does it have a estimate on when it might be turned into a full appliance?
Hi a_c_m
It is very possible that this patch become an official appliance, for the new release of Turnkey Linux based on Ubuntu Lucid. But there is no official date when this is going to happen, the Core Devs are in the middle of producing a second beta of the Core, then they'll update the existing appliances, and finally convert some of the patches to official appliances. I'm hoping too this one gets in.
I encourage you to make a donation if it is in your possibilities, but cannot guarantee that would speed things up, as the team is small and the work to be done is immense. But it surely helps the project to be maintained and this kind of appliances to be finished and available to you and all the community.
Keep in touch for the evolution of this patch and let's hope it's ready before the year ends.
Any Updates?
I've been following this thread in hopes of the full appliance appearing. Just wondering if there are any updates on this? I also, would really LOVE to see this as a full appliance. GREAT WORK ADRIAN!
-Todd Forsberg
Soon...
Be patient, Turnkey-lucid already have been published and next week a version with some fixes will be available. The second part of the releases will include Gitorious. I'm also waiting for some of my other patches to get to the second batch of releases, I know the tkl guys are working hard to make this happen.
Meanwhile, have you tryed this version? There's still time for some feedback before the final appliance goes live.
are we there yet?
Hi there,
I was going to try applying your patches on the released turnkey-core-11.1-lucid but it's been 9 days since your "be patient" post, so I'm thinking I should just wait for the Gitorious appliance to show up.
Is there a beta release we can help test?
Reid
Hopefully soon
But soon is relative. Our beloved TKL devs do not like to give clear timeframes as they don't like to dissapoint. Whilst you could try it out (by patching TKL Core) it would be better to wait if you plan to put it into production.
couldn't wait.. :-)
I am running "tklpatch turnkey-core-11.1-lucid-x86.iso gitorious.tar.gz" even as I type...
I'll report here if there are any problems.
Reid
Gitorious fails to run
I uploaded a screenshot to Flickr:
http://www.flickr.com/photos/clith/5388084035/sizes/o/
Looks like I got the exact same result as a_c_m back on Jan 2 <https://img.skitch.com/20110102-duj7ifmku3uafbcph3sde37qqd.jpg>. No patch update over the last 4 weeks?
Reid
That sucks :(
I've got no idea why it's doing that (well out of my league). Not sure whether Adrian has plans to update his patch or not though seeing as the TKL guys have started working on Part 2 of the v11 release (which will include Gitorious). I'm guessing from his comments above that he may not be updating it, if that's the case then you'll either need to try debugging it yourself, or wait for the official release.
Sorry I've been busy today to follow your comments
Please do the following: when applying the patch, add an extra > gitorious.log and post it to check for errors during patch.
tklpatch turnkey-core-11.1-lucid-x86.iso gitorious.tar.gz > gitorious.log
Also: you can try what the error is suggesting, that is, go to /var/www/gitorious and run
rake gems:install
This patch was done a while ago, and I haven't tested with the stable releases of turnkey-core. I'm also not updating it as the core tkl-devs will release this appliance soon as part2 of turnkey-lucid. But I have no problem giving you some support in the meantime.
Tried installing missing gem
How to fix sendmail: fatal: git(107): No recipient addresses err
Hello,
While registering a user in gitorious web panel got 500 internal server error.
I looked in /var/log/apache2/error.log and it's saying:
I applied this patch to and
I applied this patch to and installed turnkey-core-beta-lucid-x86 and it worked fine. Great work!
It applied cleanly to the new
It applied cleanly to the new core-11, but when you try to view gitorious.localdomain you get : https://skitch.com/a-c-m/r8m9j/ruby-on-rails-application-could-not-be-st...
Output from tklpatch
(Posted separately to avoid scrunching into the right side of the page!)
I have saved the output from using zsh to execute:
(date; tklpatch turnkey-core-11.1-lucid-x86.iso gitorious.tar.gz; date) |& tee tklpatch-out.txt
You can see it at http://rae.pastebin.com/1AzyJUEK
There are some errors that start around line 2628 and 2634 that might be the problem.
Reid
There are lots of errors in this log
It surprises me it worked at the end for you. For what I can see, the system is unable to start apache as port 80 is in use (1920). Did you run this patch over the previously patched iso? I'll make my test here anyway.
Also, the patch was unable to clone gitorious (2633-2636)
And then, all errors are because of the /var/www/gitorious directory not exists (2637-2668).
Let me run this one locally and I'll come back to you.
Nope, patches were always clean
I always started with a clean ISO, so that wasn't it.
I'm ok for now, since I got it working with the beta release of the base ISO.
Reid
Got it working with the beta..
Thinking it might be a beta-vs-release issue, I tried using turnkey-core-beta-lucid-x86. However, I also had to do the following:
% export RAILS_ENV=production
% rake gems:install ; # installed "hoe"
% rake db:create
% rake db:migrate
% rake ultrasphinx:bootstrap
% crontab -e
* * * * * cd /var/www/gitorious && /usr/bin/rake ultrasphinx:index RAILS_ENV=production
% /etc/init.d/apache2 restart
% ruby script/create_admin
To get script/console working (for deleiting users, etc) you will also need libreadline
% apt-get install libreadline5
After all that, I had a running Gitorious. I'm not sure I needed *all* of it - I was copying it off of a Gitorious setup page at http://gitorious.org/gitorious/pages/UbuntuInstallation ; I suspect I needed to do some of that with the release version. Maybe I'll give it a try tomorrow.
Reid
[updates:
- did a minor edit on order of commands - had a duplicated db:migreate (it failed because I needed to do the gems:install, after which I did db:migrate again)
- added libreadline5 ]
[Much later update:
- if you change the hostname, along with editing config/citorious.yml be sure to edit /etc/hosts if the domain name is not a real one for the machine. I keep hitting this and forgetting! ]
not quite as successful as I had thought
The web front-end seems to work fine, but git operations are failing. Creating a repository never finishes, with the spiner perpetually spinning, and any attempt to access the repository failing as well.
Trying to use the ssh method of access results in being asked for "git"s password.
So.. it looks like more work is needed.
Oh, I did perform the same commands on the release'd base OS as on the beta and I got the same successul results. So it seems that when the OS boots, a number of gitorious initialization steps need to be performed. Perhaps my problems relate to more steps that need to be followed?
Reid
scratch that! A reboot was needed to make it all work
I figured out that the problem with git operations was that ~git/script/poller wasn't running. When I ran it manually like this, everything worked:
% cd ~git
% export RAILS_ENV=production
% script/poller run
There is a script at /etc/init.d/git-poller that runs this as a daemon on startup, so after a reboot, it ran automatically and now everything is running correctly. So yay!
Reid
Tried to build patch a VM today
Hi, I tried applying this patch to a running turnkey-core-11.1-lucid-x86-vmdk, here are the logs:
http://pastebin.com/kGp7WDrL
I have highlighted what seems like the first occurance of errors, though some are obviously benin.
In the end, the /var/www/gitorious/config/gitorious.yml was absent, and hoe refused to install, I needed to specify the version with gem install hoe --version 2.6.0.
It looks that everything works after rebuilding the config, and using Reid Ellis's various posts. (Thanks!)
Nice it worked
This patch is getting old, probably need some updating. But the official appliance is comming soon, is just a matter of waiting.
nice , can't wait! thx
nice , can't wait! thx turnkey :)
Can not login in after build
First thank you for this gitorious install, I really would like to get this running as I am sure alot of folks would. Things went well with the patch, but I had to install the gem bundler before I could run the any db stuff could take place. After that a few odd little tweaks and looky looky I have the home page of the gitorious site..... although you can login, comes back with an internal server error.
Any ideas on this or better yet is there a VM applicance already configured ;-)
BTW build with 11.1 turnkey core.
Thanks for any guidance in advance.
Bill
Still no official appliance released yet
So still have to wait for the TKL core devs (Alon & Liraz). As Adrian (the dev of this patch) says just above, this patch is getting quite old now so may require some tweaking to get it to work properly.
Good luck.
Updated Patch for turnkey-core-11.1-lucid-x86.iso
Hello,
I modified the gitorious patch so that it will build a functioning gitorious server from the turnkey-core-11.1-lucid-x86.iso image.
I modified it to use:
I also modified it to use bundle exec to perform the rake commands.
I modified git-poller to use "bundle exec" because a gem thats required was the incorrect version and the poller would just shut down. This was causing the ssh_keys to never be activated and the repository to never be created.
However, I did notice that my call to "/root/create_admin_turnkey" is out of order and it needs to be run manually. Tomorrow I will try to fix and test that problem.
I wasn't sure how to post files in the comments, feel free to download the file from http://dl.dropbox.com/u/193989/gitorious-11.1-lucid-x86.tar.gz and give it a shot.
If you have any problems let me know.
Thx for the update!
Thx for the update!
Keep us posted pls
Nice work Scott
You can only attach files to the first post of a new thread so just start a new thread and attach your updated patch to it. Then just post a link at the bottom of this thread pointing to your new thread.
Good to know
Thanks Jeremy,
I will create a new thread when I think I have all the bugs worked out of the patch.
I am currently reading the webmin module development documentation. I am considering creating a webmin module to include in the patch so that we can look in webmin to see the poller, sphinx, git-daemon and activemq service status. I think it would also be nice to be able to change the hostname, rotate the logs, etc. all from the webmin module.
A Webmin module sounds awesome!
That'd be great if you developed a custom Webmin module. The only thing that springs to mind would be when you develop this perhaps consider rather than making just one module; create 2 modules: One with Gitorious specific settings and another a more generic module - that could potentially be included with Core (and thus all TKL appliances).
New Patch from Scott
Just tried the patch and during patching you get an error, it creates a tmp directory and then it creates a gitorious-lucid file instead of a gitorious-11.1-lucid.tar.gz where can I change this?
Thanks
Bill
Oops
I renamed the tar file before uploading it, I guess you cant do that
I have renamed the patch folder. I will bundle, test and push it out to my dropbox shortly.
Update: I guess my employer blocks the port that apt-get uses.. so I can't test the change until this evening.
In the meantime try:
or
The directory in the gz file
The directory in the gz file is incorrect for what is being looked for after uncompressed in the /tmp directory
Re tared the gitorious-lucid directory to gitorious-11.1-lucid.x86 and the patching starts to work
create admin error
Scott
Here is the error on create admin
/var/www/gitorious/config/environment.rb:26: Your config/gitorious.yml does not have an entry for your current Rails environment. Please consult config/gitorious.sample.yml for instructions. (Runtim
eError)
from /var/www/gitorious/config/../vendor/rails/railties/lib/initializer.rb:111:in `run'
from /var/www/gitorious/config/environment.rb:24
from ./create_admin_turnkey:2:in `require'
from ./create_admin_turnkey:2
Any sugguestions, sorry not too familiar with ruby and rails.
Specify the RAILS_ENV
You should be able to run it by specifying the rails environment
RAILS_ENV=production /root/create_admin_turnkey
Also, I just pushed an updated version to dropbox at the same url.
http://dl.dropbox.com/u/193989/gitorious-11.1-lucid-x86.tar.gz
It should fix the folder structure problem and take care of creating the admin for you as well. (although I haven't had a chance to test it)
I take it out for a spin....
I take it out for a spin.... will let you know what happens
Thanks
New Try
Scott,
Just so you know, the dreaded 500 server error upon login. This seems to be the only function gitorious has LOL. Never can get past this.
Just letting you know. BTW running inside a VM .
Error details?
Can you give me the details of what its complaining about? I am not an expert, but I will see what I can figure out.
Well logging in it won't take
Well logging in it won't take you anywhere but the Internal server error. Looking at the running processes the ultrasphinx process is not starting. You also have to uninstall and downgrade two gems.
ultrasphinx is just for searching so I don't think that is a problem, since you can not log in. You apparently have this running I wish I knew the secret.
Thanks
Give this one a shot
So there were a couple small problems with that last one. I have pushed a new version up to my dropbox at the same url: http://dl.dropbox.com/u/193989/gitorious-11.1-lucid-x86.tar.gz
Just to make sure that I am not doing anything out of the ordinary.. here are the steps I used to patch and build my system.
Here are the steps I use to patch the iso:
All of those steps worked fine for me, however I did notice that the sphinx service was not running. Its because /var/www/gitorious/db/sphinx/* is owned by root.
To fix this problem:
After I verified everything is working then I change my hostname to match my fully qualified domain name as Adrian outlined above.
I figure some people will not want this to be an internet facing appliance. If that is the case, then I recommend changing the hostname to whatever you want and then setup ZeroConf to enable browsing by name without the need to hardcode the hostfile.
ZeroConf is as easy as apt-get install avahi-daemon. Then you can access your appliance by browsing to https://hostname.local without the need to modify your hosts file or adding a dns server.
Hope all of this works for you
Scott
******************* You shouldn't need any of this *********************
Here are some common problems that I have ran into while messing around with gitorious the last 2 days.
Scot Success.....!!!!!
Success .... Nice Job ... now just need to recreate this and face the internet.
Thanks for all your work,responses and help.
Cheers,
Bill
Any chance of an official appliance?
After all this time, all these patches, and the promises of an official appliance .. is there any news on it? Maybe an upload to the SourceForge Turnkey project? If not yet, is there anything we can do to help?
P.S. Is there a way to add more tags to this post? Like maybe "git" and/or "gitorious"?
How can I help?
I would be more than happy to push my iso to the Turnkey Community sf project. Who do I contact to give them the iso?
Thanks Scott! Some suggestions
First of all thanks for taking time to upgrade the patch and contribute back. That's what makes opensource great! You seem to know very well about the app, contrary to my initial patch, which was a bit of a try-and-fail patch. I'll point you places where you can contribute with the project:
- Webmin modules would be a great addition, if they are simple to implement. But for this appliance, I think it would be more important to code an inithook script to configure the admin password. You can use the create_admin_turnkey as a base for the implementation, and maybe add the hostname setting. For information on this, you can check the inithooks documentation in the site.
- Feel free to update the dev's wiki page for this patch. Update it to point to the new thread were you'll attach your final version of the patch. Also, you gave a lot of useful documentation on how to setup the appliance once is running. That would make a great addition to the documentation wiki. Add it now to the dev's page until the official appliance is out. With this update, surely the core team will have less to worry when implementing the official appliance. BTW, they are currently working hard to make the project a bit more open and accept more outside collaboration, so the lucid releases second batch will probably take some time.
- When you feel your patch is complete, you can contact Jed or myself, and we'll happily create the iso and upload it to the community appliances. I'll suggest Jed right now as my laptop died and I've been a bit with my hands tied up, but I'll hope to get a new one by the end of july.
Again, thanks for your contribution, it's nice to see someone jump in and give a hand. If you have further questions the general forum is the right place to ask.
Newbie question
Now that I have this running and such, I am having trouble getting a local git repository into the gitorious repository... I should know this but....if it was svn I would be in but its not and I need help.
Thanks for any info or point me in the right direction, trying to make this KISS princple like LOL for us git virgins.
Thanks again Scott for the working instructions and patch.
Reply to myself
Figured it out ... I am lil dense sometimes... Thanks.. btw had the wrong ssh key installed for my laptop that is why the errors not pushing repository.
Any news on the turnkey gitorious offical release?
Any news on the turnkey gitorious offical release?
TKL devs 'release when done'
So no word on when, although considering part 2 of v11.x release is well overdue hopefully we'll see it sometime soonish.
This will be epic when it is done
I have spent weeks of my like trying to have gitorious installed. I was depressed and defeated, but this gives me hope.
Keep up the good work!
I am buildding another Gitorious server using the latest patch..
I'm trying to create another Gitorious server (the first is running nicely under VirtualBox on an iMac).
First try didn't go well. I'm using Scott Beardsley's latest patch at http://dl.dropbox.com/u/193989/gitorious-11.1-lucid-x86.tar.gz. There seems to be no /var/www/gitorious directory -- I think the "git clone" failed.
Is there any means for these scripts to handle errors like this gracefully? i.e. tell the user "Can't connect to git server, patch failed"?
"create_admin_turnkey" doesn't work
The "create_admin_turnkey" script in root's home directory doesn't work any more. Gitorious now stores encrypted passwords with salt. So now I have to figure out how to make the password encrypted so that the admin user can be created.
Migrating an old repo to a new machine
Took me a while to figure out I needed to run this to upgrade my gitorious MySQL database.
That's specific to the version of gitorious I was at (20110223130355). egad.
How to migrate your gitorious server to a new instance
I've finished a successful test run of moving our Gitorious instance to a new install of Gitorious (on machine "newgitorious") using this patch. Here are the key steps I used:
As root on the old VM/machine:
On the newgitorious machine:
As root:
avahi-daemon allowed me to use ZeroConf so I could use the hostname newgitorious.local without any mucking about with /etc/hosts or similar. I think this should be included with all TKL distributions. Using "applianceName.localhost" is sub-optimal to say the least.
As user "git":
As mentioned before, this is related to the specific version of Gitorious I was running on the old machine. I really wish the Gitorious dev team would update their real revision number more often. By now it should really be at version 2.7 or something.
Hope this all helps someone else.
Upgrade
Reid,
Kinda confused, where did you find the version number for your old gitorious?
Also, when you moved you database from your old version to the version 2.0 that is out there now what exactly where your steps and you also moved the repositories I assume. I am trying to update so I can be at this new versioning level from gitorious.org to make the updating easier I hope.
Thanks
Bill
Version numbers
Hi Bill,
The version number was the least obvious part of all this. It involves the directory db/migrate. On the old machine, the last file in there was 20100316124803_create_archived_events.rb. Looking on the new machine, the very next file was 20110223130355_add_suspended_at_to_projects.rb - I ended up using the "20110223130355" string as my argument to "bundle exec rake db:migrate:up" since otherwise, Rails didn't seem able to migrate. I would have expected to simply do a "bundle exec rake db:migrate" and have everything taken care of, but alas, such was not the case.
I found the "bundle exec rake db:migrate:up" command through a series of google searches frankly.
Hope this helps,
Reid
Versioning
Hey Ried,
Thanks for the info.
Curious how you made the move of repositories (copy the data I am sure) but the update to the MySql, its not just a restore or is it really that simple.
Bill
mysql dump
The rsync command took care of copying over the git repositories, and the only other thing I used was mysqldump. I added flags so that I could do it over and over until I got it right though.
One thing that may not be finished are the ssh keys. I think there is a rails command needed to push them into ~git/.ssh ...
Reid
5 months later and no Gitorious appliance
Looks like TKL isn't interested in appliances any more. Maybe we need a tkl-users.org or something to host user-generated appliances?
Jed created a site to host community appliances
I wouldn't say TKL don't have an interest in appliances. What I would say is that they became overwhelmed with the work. I asked the main team to be more open for the project's health, after all, it's an opensource project. What I understood is that the main infraestructure for creating appliances isn't community friendly, and they decided to make an effort to enhance the whole thing so they can be more open to contributions in the future.
The fact is that as the second batch of appliances (a lot of them almost a year old since they got published), there is no info on timeline/features being developed. I'm sure they are working, and I expect to see the results by the Precise Pangolin release. Meanwhile, we can upload them to the community site.
I also suggested a site dedicated to tklpatches with a voting system, as I understand it's going to be impossible to review/officialize all patches into appliances, but at least as you suggest, give a space to community to post appliances and be reviewed by other community members. I'm sure we can push as community to see some of these things done, or start something ourselves and later we could get the official support.
Definitely interested in expanding TurnKey...
We are definitely interested in expanding the TurnKey library, but as Adrian mentioned we decided to focus our efforts on moving our build infrastructure to the cloud, which will make it much easier to open up the development process to others and ease our current bottle neck.
We also have plans/ideas of creating a more "developers" orientated section for developing and sharing appliances (and later implement reviews/voting/etc).
BTW, I just came across GitLab (open source clone of GitHub). Would be interesting to see how it compares to Gitorious.
Great idea
both on the GitLab appliance, more more importantly on the idea of 'Community TKLs'.
Gitlab?
After all that work, you went with Gitlab instead of Gitorious? Can you tell us why? Does Gitlab have more/better features? On the surface, it seems to use the same components as Gitorious..
We followed Adrian Moya's recommendation
Adrian Moya, that nice chap who developed the Gitorious TKLPatch recommended we do Gitlab after using both. He said he found Gitlab more powerful. I think it's based on Gitorious BTW...
Add new comment