TurnKey Linux Virtual Appliance Library

TKLPatch for ClipBucket

[update 2011-10-30] (Jeremy) - Thanks for your great work on this patch Adrian! Please see here for an updated version.

Hi! Do you want your own private youtube? you got it! This TKLPatch will convert Turnkey Core Lucid into a video streamming server running the latests version of clipbucket, all preconfigured and working for you! 
 
Someone pointed this software in the forums and I found that it can be a nice TKL so I decided to take a look. Developing this patch was harder than I imagined, with lots of retries trying to make a good compilation of ffmpeg that the program liked and listed "all green checks". But finally, as I'm not installing from svn trunk, the program won't detect the version of ffmpeg. It expects a "SVN-rNNNNN" version, but the compilation ends up with "0.6". It doesn't matter as it works perfectly. I could fool the program setting a version "by hand", but I don't see the point. If I could get the exact version from the tarball I'd do this, but they use a version.sh script that gets info from .svn folders which don't exist in the tarball sources. 
 
Other challenge was to simulate the install, because it didn't include an automatic installer script. I also detected a bug in the ffmpeg modules detection which was reported in the forums. Finally, after some frustration and hard work, here it is! It needs testing a lot, as I just made the basics tests. 
 
If anyone with more experience in this software consider I should include other codecs in the compilation of ffmpeg, comments are welcome. I focused on including the ones that the software marked as required. 
 
I decided to user the tarball over the SVN-trunk to make this patch more "stable". The version for the package generated and installed is "0.6+src20100615", composed by the version, the source and the date that the tarball was published. But comments are welcome. As I'm using checkinstall to create a deb, you can easily remove it with apt-get remove ffmpeg and compile your own from SVN trunk if you want. 
 

Features:

- Clipbucket 2.0.91
- http and https access. 
- ffmpeg compiled with required codecs. 
 

What it does:

 
1. Set Hostname
HOSTNAME=clipbucket
echo "$HOSTNAME" > /etc/hostname
sed -i "s|127.0.1.1 \(.*\)|127.0.1.1 $HOSTNAME|" /etc/hosts
hostname clipbucket
2. Add multiverse repository, needed for libfaac-dev. 
sed -i "s|# deb http://archive.ubuntu.com/ubuntu lucid-updates multiverse|deb http://archive.ubuntu.com/ubuntu lucid-updates multiverse|" /etc/apt/sources.list.d/sources.list
sed -i "s|# deb http://archive.ubuntu.com/ubuntu lucid multiverse|deb http://archive.ubuntu.com/ubuntu lucid multiverse|" /etc/apt/sources.list.d/sources.list
3. Update package information
apt-get update
4. Install required packages
install apache2 php5 mysql-server php5-mysql flvtool2 gpac php5-cli php5-gd ssl-cert libx264-dev libtheora-dev libfaac-dev libmp3lame-dev libvorbis-dev libxvidcore-dev build-essential yasm checkinstall
5. Download and compile ffmpeg from tarball release
cd /usr/local/src
wget http://ffmpeg.org/releases/ffmpeg-0.6.tar.gz
tar xzf ffmpeg-0.6.tar.gz
cd ffmpeg-0.6
./configure --enable-libx264 --enable-libxvid --enable-libfaac --enable-libvorbis --enable-libmp3lame --enable-libtheora --enable-postproc --enable-pthreads --enable-gpl --enable-nonfree
make
checkinstall --pkgname=ffmpeg --pkgversion "0.6+src20100615" --backup=no --default
 
6. Download clipbucket
cd /usr/local/src
wget -O clipbucket_2.0.91.zip http://sourceforge.net/projects/clipbucket/files/ClipBucket%20v2/clipbucket_2.0.91.zip/download
 
7. Unzip and move clipbucket
unzip -q clipbucket_2.0.91.zip 
mv upload/ /var/www/clipbucket
 
8. Create database and database user
mysqladmin -u root create clipbucket
mysql -u root <<-END_PRIVS
GRANT ALL PRIVILEGES ON clipbucket.* TO clipbucket@localhost IDENTIFIED BY "clipbucket";
flush privileges;
END_PRIVS
 
9. Enable apache modules
a2enmod rewrite ssl
 
10. Enable site
a2dissite default
a2ensite clipbucket
a2ensite clipbucket-ssl
 
11. Simulate install. Here I'm using an extra sql file overlayed in /tmp
cd /var/www/clipbucket
chmod 777 files files/conversion_queue files/logs files/original files/temp files/thumbs files/videos files/mass_uploads files/temp/install.me images images/avatars images/backgrounds images/category_thumbs images/groups_thumbs cache install 
chmod 755 includes
cp install/dbconnect.php includes/dbconnect.php
sed -i "s/_DB_HOST_/localhost/" includes/dbconnect.php
sed -i "s/_DB_NAME_/clipbucket/" includes/dbconnect.php
sed -i "s/_DB_USER_/clipbucket/" includes/dbconnect.php
sed -i "s/_DB_PASS_/clipbucket/" includes/dbconnect.php
sed -i "s/_TABLE_PREFIX_/cb_/" includes/dbconnect.php
sed "s/{tbl_prefix}/cb_/" install/cb_v2.sql > install/cb_v2_turnkey.sql
sed -i "s/(154, 'use_subs', '0');//" install/cb_v2_turnkey.sql
mysql -u root clipbucket < install/cb_v2_turnkey.sql
mysql -u root clipbucket < /tmp/cbturnkey.sql
cp install/install.lock files/install.lock
cp install/clipbucket.php includes/clipbucket.php
rm files/temp/install.me
12. Stop services
service apache2 stop
service mysql stop
 
13. Clean
apt-get clean
 
Enjoy!
Attached: 
Basil Kurian's picture

Nice application

Wow , it's a too nice application


Jeremy's picture

This looks very cool!

I will definitely have a use for this at work for a range of training videos we have. I need to see how I can get the videos into a usable format first though (damn DRM). 

Script kicks back an error

When I run the script, it stops with the following error:

E: Couldn't find package libfaac-dev

I'm a bit of a noob with Linux, but not afraid to poke around and play. I'm running TKL Core in a Virtualbox VM.

Thanks!

Are you using tklpatch to apply the patch?

In case you're not, and you are running these steps manually, be sure that you have the multiverse repo enabled and that you did an apt-get update. (Steps 2 and 3)

You can check manually with 

cat /etc/apt/sources.list.d/sources.list

And check that lucid multiverse and lucid-updates multiverse don't have a # in front. 

Then apt-get update and the package should be found. 

Also, be sure that the VM has internet. Use bridge network mode for virtualbox. 

Okay, I finally got a chance to dig into this again

If I do not run apt-get upgrade after installation and manually uncomment the multiverse lines, the installation moves forward. I'm running the script that is linked on this page (not manually).

But I'm still having problems.

The script runs all the way until the "a2ensite clipbucket" command errors out with "ERROR: Site clipbucket does not exist!"

Sorry for all the dumb questions, I really appreciate the help as I try to learn!

Jeremy's picture

But are you applying the script with TKLPatch?

I'm assuming not - in which case you will also need to copy the files out of the /overlay folder. These files need to be copied over / (root).

That particular error (ERROR: Site clipbucket does not exist) occurs because the clipbucket file (from /overlay) doesn't exist in /etc/apache2/sites-available as the command expects.

If you can, it would be much better to just install TKLPatch and let it take care of it all automatically.

You, Sir, are a scholar and a gentleman.

Thank you for the nudge in the right direction. I wasn't running the tklpatch that you so kindly pointed me to and it didn't occur to me that I needed to copy the overlay directory to the filesystem. (duh)

Now, I'm researching why Apache (I think) is rewriting all the paths to "domain.tld". I can change the host file of my client to fool the browser into checking the correct host, but I'm still missing something (obvious, I'm sure).

Again, thanks for the help!

How's the clipbucket appliance coming?

Wondering if the appliance is going to be released soon. Thanks for all the great appliances. Keep it up.

Jeremy's picture

No definate timeline...

Due to Murphy's Law and the TKL core Dev team's desire not to dissapoint, TKL releases come as soon as they are ready and with little or no warning. At this point the roadmap includes a stable release of the current RCs (which I would imagine will be very soon) followed by a release of the new appliances (I assume that they'll be released as RCs first too, or perhaps even betas). You can be asured that Alon and Liraz are working their butts off to get them out the door!

2 problems

Hi all,

We have installed clipbucket according to the installation procedure in this page, However we have encountered two problems:

First of all, we modify the maximum file upload size through admin_area, and then upload  a file which is less than the threshold (Our file size to upload is 21 MB, with the file extension of FLV or MP4), but unfortunately, we got the error : "Server (IO) Error"

secondly, the installed clipbucket is able to just play the files with MPG extention, and it fails to play other uploaded files with other extensions: FLV, MP4, XVID, DIVIX, ....

In fact, we got the "File does not exist: /var/www/clipbucket/files/videos/no_video.flv" error through apache error.log

By the way, all the system checks are green except the FFMPG r19000.

 

thanks alot for your helps;

ClipBucket Install

Hiya,

I followed your tutorial as shown on your site...  and all went ok.  The only problem I am having now is that when I go to upload videos, it wont let me. 

If it helps, I have installed it on a VMWare based system.  The main system is Windows 7, and I use VMWare to test out other installations.

Hope you can help, as this has been a teriffic tutorial, and taught me a lot!

 

Many thanks,

Jordan

Ask about domain.tld

Thanks Adrian for the useful script. I have successfully using that script using tklpatch-apply to install clipbucket. But I still have a problem here. When I access the site using browser, the link always pointing to domain.tld/XXX and the CSS does not work.

Thanks for your help.

Regards

Jeremy's picture

Updated patch released

I have updated Adrian's patch. Please see here.

@Habib - I'm not sure what's happening there. It doens't do that for me, although I didn't actually test this patch at all (just my update). But unless there has been a change in the ClipBucket software itself (and that was how it used to behave but has changed) Adrian's patch shouldn't do that IMO. Perhaps try my updated patch?

How to install script

I no pretty much next to nothing in regards to linux.. With that being said, how do I install this script on Turnkey Core?  I have tried numerous tutorials to no avail.. Any Help?

Jeremy's picture

It's well documented... but I'm feeling generous :)

I do a fair bit of patching and so I have a dedicated TKL Core VM for doing this. I would recommend that you do likewise and patch an ISO (inside the Core VM) and then install from the ISO.

So my workflow is like this:

  1. Install Core to VM (VBox is a good free desktop one)
  2. Inside Core - install TKLPatch
    apt-get update && apt-get install tklpatch
  3. Download/upload relevant ISO
    download direct into appliance using wget - OR
    download in your web browser and upload into Core via SFTP (I prefer Filezilla)
  4. Download the patch - I always do this inside Core - I have on occasion had patches corrupted by web browser. get the web address of patch by right-clicking it >> Copy Link Location
    wget http://web-address...
  5. Apply patch to ISO:
    tklpatch patch.tar.gz iso-filename.iso
  6. Copy out the ISO using SFTP and test in VirtualBox

Also I have created an updated version of this patch - see here

ClipBucket in OpenVZ

I dont find it... is not available in the OpenVZ templates?

Thanks.

Jeremy's picture

Unfortunatley this never made it to official release

So you won't find it as an official appliance in any format. But you may be able to apply the patch to an exisiting TKL OVZ template. I had a crack at creating an update of this but never got it finished and got sidetracked elsewhere....

I have also documented (and tested) applying a patch to an OVZ template (rather than an ISO) which may be worth a shot. Have a read here.

Post new comment

The content of this field is kept private and will not be shown publicly. If you have a Gravatar account, used to display your avatar.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <p> <span> <div> <h1> <h2> <h3> <h4> <h5> <h6> <img> <map> <area> <hr> <br> <br /> <ul> <ol> <li> <dl> <dt> <dd> <table> <tr> <td> <em> <b> <u> <i> <strong> <font> <del> <ins> <sub> <sup> <quote> <blockquote> <pre> <address> <code> <cite> <strike> <caption>

More information about formatting options

Leave this field empty. It's part of a security mechanism.
(Dear spammers: moderators are notified of all new posts. Spam is deleted immediately)