Drush (drupal shell) on TurnKey Drupal appliance

I'll have to study the tklpatch creation process. In the meantime, I've attached the steps I used to install Drush on a freshly installed Drupal appliance to the original post.

1. # apt-get update
2. # apt-get install wget php5-cli
3. Install Drush
    # cd /tmp
    # wget http://ftp.drupal.org/files/projects/drush-All-Versions-2.1.tar.gz
    # tar -zxvf drush-All-Versions-2.1.tar.gz
    # chown -R root.root drush
    # mv drush /usr/share
        # ln -s /usr/share/drush/drush /usr/bin/drush
    # cp /usr/share/drush/examples/example.drushrc.php ~/.drushrc.php
   Edit ~/.drushrc.php replacing
    # $options['r'] = '/home/USER/workspace/drupal-6';
   with
    $options['r'] = '/usr/share/drupal6/';
4. Test
    # drush help
    # drush status
    # drush statusmodules
5. Cleanup
    # rm /tmp/drush-All-Versions-2.1.tar.gz

Note: Edited on 1/23/10 to change php-cli to php5-cli, use /tmp instead of /opt, and cleanup after.

Note: Edited on 4/25/10 to correct path error reported by Johnny Canuck.

Note: Edited on 6/6/10 to correct symlink error reported by Dan.

You can contribute a patch to the appliance that enables drush.

See this page: /docs/tklpatch

The drush sql sync command does need to drop the target database before syncing the new data, otherwise the result would be a merge of the old and new databases.  I hope the decision was to drop all tables and not just those in the source.  My problem is just a bit different.  I have a website that was originally started in Drupal 4.7 and is now at 5.21.  Over the course of several years experimenting with different modules and upgrading versions, a fair amount of 'cruft' has crept into my database.  As I prepare to upgrade to 6.xx, I'm trying to figure out a strategy to identify what obsolete tables have been left behind by modules that didn't clean up when they were uninstalled or crept in when data was copied from one database to another without clearing (dropping all tables) the target database.

My thought at this point is create a sandbox with an empty database, then using drush, make a list of installed_modules and a list of enabled_modules from the production site and use those lists to install a fresh copy of Drupal 5.21 with the identical modules installed and enabled.  Then I can use phpMyAdmin to compare the two databases and remove any obsolete tables i.e. tables that exist in production but not in the 'clean' copy.  If I could figure out how to automate this last step, I could create a 'drush sql clean' command.

Do you think this would be a useful effort?  I would probably only use it once, so it wouldn't be worth the effort unless others could benefit as well.

 Yes, it does choose some tables to update structure-only, to leave cache and stuff out.

No, AFAIK it doesn't yet contain remote selective deletes of missing stuff, that would be scary.

But I think there is a 'drop-database' (or rebuild or something) in the works (I have one in my own custom additions) that could become part of the process. Check the issue queue I guess.

Edit: It's in already. Sql sync - drop target database and recreate 

dman,

Does the drush sql-sync command have an option for syncing just the table structure between the two databases while leaving the records unchanged?  I'm thinking it would be handy to remove any tables in the dest that don't exit in the source.  Normally I rely on drush updatedb to add and initialize any new tables, but sometimes old tables from removed modules get left behind.

  I tried out the appliance as a ready-made home for an existing site - I want to bundle a fully working snapshot as an archive.

drush is THE tool for folk that will be doing many Drupal deployments, and the setup of it is a hassle each time, so it would be lovely if it were in this distro.

NOTE that it requires php-cli - which turnkey-drupal6-2009.10-hardy-x86-virtualbox did not have available.

Anyway, with drush, the deployment was a breeze (save for one annoyance).

I could 

1. define a site-alias that described the 'turnkey' appliance (its paths and IP)

2. push the db from my dev site to the appliance

drush sql-sync devsite turnkey

3. push the files and modules across

drush rsync devsite\:\!site turnkey\:\!site

drush rsync devsite\:sites/all/ turnkey\:sites/all/

And the site almost worked.

(... I spent a long while trying to find why it kept wanting to 'install' even though all the bits were configured. Eventually I found the custom code to delete the /tmp/install_redirect.php thing)

Apart from that, it was three steps to a working, cloned drupal site on a stand-alone VM,

... anyway. drush is a key part of drupal management now.  Would be cool to use it,

Well, yes tar preserves symlinks by default, but it doesn't follow them either.  If you backup using tar from the drupal root, then the contents of sites, profiles, and files won't be included because they are symlinks.  If you use tar's -h option to dereference symlinks, then all paths are included, but when you restore, the symlinks are overwritten which would break the site upon the next debian update.  I've come to the conclusion that I need to backup core, sites, profiles, and files as separate tar files then dump the database(s) and finally tar the whole collection with compression.  Special handling will be needed for .htaccess, robots.txt, and favicon.ico if it exists.  Those are the files in drupal core that are likely to modified by webmasters.

I hope to develop a system robust enough to backup my current system with a single file tree, and restore it into a TKL Drupal appliance and have everything end up where it should.  I'll post the results here when I'm done.

BTW, a tklpatch for adding drush to TurnKey Drupal sounds great!

I've been working on testing a tklpatch for adding drush-All-Versions-2.1, to TurnKey Drupal.  I wasn't aware of the debian package for drush that dman mentioned.  I'll have to check it out to see if it makes sense to use that instead.

My testing so far hasn't revealed any problems with drush that are unique to TurnKey Drupal.  Some of the drush commands don't work because they require certain modules like simpletest to be installed, but that is a drush issue and not related to the TurnKey install.

I've delayed posting the patch because I wanted to include some scripts that I've written to supplement drush including updating Drupal core.  Before committing to using TKL Drupal in a virtual machine, I wanted to make sure that I have all the functionality of my old site.  What I've found is the use of symlinks for sites, profiles, files, etc broke my original scripts which expected everything to be stored under the drupal root.  The last remaining issue I have to solve is to figure out how to use tar to backup and restore all the relevant parts while preserving the symlinks.  If anyone has experience with some of the more esoteric tar commands, I could use some advice.

[ -d $patch_dir/overlay ] && tklpatch-apply-overlay $rootfs $patch_dir/overlay
[ -x $patch_dir/conf ] && tklpatch-apply-conf $rootfs $patch_dir/conf
[ -d $patch_dir/debs ] && tklpatch-apply-debs $rootfs $patch_dir/debs

Sorry guys, it was late last night and I wasn't thinking too clearly.  I didn't mean to imply that the order of applying debs, overlays, and the conf script should be changed.  As Alon points out, its important to be able to make changes after the debs are installed, and changing the order would break tklpatches that others have prepared.

I must have been channeling previous experience in building rpms.  I just took another look at the Debian Binary Package Building HOWTO and noted that Debian only provides the 'postinst' and 'prerm' scripts.  The rpm spec file, on the other hand, is much more fine grained, providing macros that are executed in order.  They include %prep, %setup, %patch, %build, %install, %clean, %files, among others.  The %setup, %patch, and %build roughly correspond to the installation of the /debs.  %install and %clean correspond to the tklpatch conf script, and %files to the /overlay.  Neither Debian nor tklpatch have the equivalent of %prep.  What I was thinking last night was the potential for adding an additional script, 'preconf', that is optional but if it exists, is executed before installing the debs and applying the overlay.  This would add some additional flexibility without breaking previous patches.  Just my 2 cents.

Update:  I edited tklpatch-apply and copied the apply-conf line to before apply-debs, then changed the line to install 'preconf' if it exists.  I then moved the install of the prerequisite, php5-cli to a 'preconf' file in the patch_dir.  Worked like a champ.  Both 'conf' and 'preconf' were applied in the expected sequence.

[ -x $patch_dir/preconf ] && tklpatch-apply-conf $rootfs $patch_dir/preconf
[ -d $patch_dir/debs ] && tklpatch-apply-debs $rootfs $patch_dir/debs
[ -d $patch_dir/overlay ] && tklpatch-apply-overlay $rootfs $patch_dir/overlay
[ -x $patch_dir/conf ] && tklpatch-apply-conf $rootfs $patch_dir/conf

tklpatch-apply-debs could be modified to identify and install any prerequisites, but that would make it more complicated and might lead to some nasty surprises.  Better IMHO to let the patch developer identify any required packages and install them via 'preconf'.

My attempt to build a tklpatch using drush_2.0-6_all.deb failed as follows:

# tklpatch turnkey-drupal6-2009.10-hardy-x86.iso drush_2.0-6_all_tkl_drupal6.patch
# extracting root filesystem and isolinux from ISO

created 28730 files
created 2956 directories
created 1665 symlinks
created 41 devices
created 0 fifos
TKLPATCH_ISOLABEL: drush_2.0-6_all_tkl_drupal6.patch
# applying patch drush_2.0-6_all_tkl_drupal6.patch
# installing 1 debs from drush_2.0-6_all_tkl_drupal6.patch/debs
# chroot execute: dpkg -i /*.deb
Selecting previously deselected package drush.
(Reading database ... 23373 files and directories currently installed.)
Unpacking drush (from /drush_2.0-6_all.deb) ...
dpkg: dependency problems prevent configuration of drush:
 drush depends on php5-cli (>= 5.2); however:
  Package php5-cli is not installed.
dpkg: error processing drush (--install):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 drush

Apparently tklpatch processes the /debs before executing the conf file.  My conf file looks like this:

#!/bin/bash -ex
install()
{
    apt-get update
    DEBIAN_FRONTEND=noninteractive apt-get -y \
        -o DPkg::Options::=--force-confdef \
        -o DPkg::Options::=--force-confold \
        install $@
}

# Install drush (Drupal shell) - a highly useful package for managing Drupal
# drush_2.0-6

# Install required packages
install wget php5-cli;

# Let udhcpc set hostname
sed -i -e 's/#SET_HOSTNAME/SET_HOSTNAME/' /etc/udhcpc/*;

# Tell the world what we've done!
echo 'Installed drush_2.0-6_all (Drupal shell) on TurnKey Drupal!' >> /etc/issue

Is there a possibility for a pre-install and post-install script ala Debian?  I suppose I can download the php5-cli deb and add it to the /debs folder.  Will try again tomorrow.

Thanks Alon,  I couldn't have asked for any more flexibility.  I'll use the new version to create a new patch for Drush and then submit it here when I'm done testing.

That's what I get for relying on memory and the command line history.  The lines should have read:

# mv drush /usr/share
# cp /usr/share/drush/example.drushrc.php ~/.drushrc.php

Instead of adding /usr/share/drush to the PATH, I've been creating a link as follows:

# ln -s /usr/share/drush/drush.php /usr/bin/drush

That is consistent with the installation of the Debian version of Drush.  http://packages.debian.org/sid/drush

Hi,

Following the instructions at:

/files/attachments/InstallDrush_1.txt

there may be a couple of issues:

1) I think the lines:

# mv drush /usr/share
# cp example.drushrc.php ~/.drushrc.php

should be:

# cp drush/example.drushrc.php ~/.drushrc.php
# mv drush /usr/share

2) After completing the instructions, drush is not executing, because of a path issue. In my case I edited ~/.bashrc and added the following line:

PATH=$PATH:/usr/share/drush

Am a noob, so these may not be the best practices. At least it now works for me.