szczym's picture

hello

First of all i would like to thank you, developers of turnkey - its smooth and flexible software ! I use it everyday not only for web development.

I'm building an appliance based on turnkey lamp for automatic video publication via podcast and bittorrent from local network to drupal6 site. Its intended for sharing source material (usually licensed on CC) among artists and journalists. Of course there might be other scenarios of that system (say piracy) but i intend it as a box installed in "free-culture" multimedia studios.

So the plan is:

  1. file is being copied into a local folder over samba
  2. drupal picks up the file (via media mover)
    • extract metadata
    • encode it (ffmpeg)
    • make torrent out of the original (basic shell script)
    • copy torrent +move original file to local dir for seeding
    • send encoded file + torrent to remote drupal site
    • publish all on local site (kind of video archive)
  3. local torrent client picks up media for seeding
  4. remote site picks up the media, and publish it as both a torrent podcast and "web video"
  5. remote tracker pickup torrents from feed and starts to track it
  6. remote client (audience) download media via podcast client (miro) or bittorent (deluge with flexrss)

Right now basic functionality works, files are being encoded, copied, and published, media arrive to clients. Could you tell me how to properly install samba share so that security issues are solved in that scenario:

  1. Everyone from local net might upload media to shared folder
  2. Drupal might read and write to uploaded media and its dir
  3. Torrent client might read, write and delete the seed media
  4. Only certain users will be able to write via samba to given filder

The next step will be to add other services running on the box, say gupn, www.fireflymediaserver.org/ ect. so samba is critical

Thanks for your help!

Forum: 
Liraz Siri's picture

Sounds like a very interesting project, and we'd love to support you with it any way we can, including adding it to our roster of appliances (more about that later).

As to your question, Samba can be a bit of a pain to setup and configure if you're unfamiliar with it. I recommend you start with the following:

apt-get install samba
apt-get install webmin-samba
In my experience, the Webmin samba module should help you get your bearings and with a bit of experimentation you should be able to get things to work. Unfortunately, I can't give you the exact configuration details because I'm not familiar with all of the components in the setup you describe and exactly how they are supposed to work and interact with one another. I probably wouldn't get it right the first time anyhow, as non-trivial integrations like this usually require some testing and tweaking before they work.

However in principle I think I understand what you are trying to accomplish so I will give you a few helpful tips and recommendations:

  • Watch out: Samba maintains a separate user database!

    That means a Samba user it not necessarily a Unix user and vice versa. The passwords are also separate. It's important to keep that in mind so you don't get confused. You can map "Windows" users to "Unix" users though. The Webmin module contains a tool for automatically converting unix users to Windows users, though I'm not sure how useful that will be in your case.

  • I recommend you first get everything up and running under a single user: www-data. This is the user the web server/Drupal runs as.

    After you get this to work, evaluate your security requirements and decide if you want to do the extra work of having different processes run with different permissions. For a non-security sensitive application such as this you probably don't need to do that.

  • After you create a share through the Webmin samba interface you'll be able to configure that share in detail.

    In the "Security and access control" section you'll be able to configure what hosts are allowed access to the share, whether to allow guest access and what unix user to map to the "guest" account.

    In the "edit file permissions" section you'll be able to configure file modes, and whether to force files to be owned by a particular unix user/group and file permissions.

  • I recommend configuring an "incoming" directory as a queue so files uploaded via samba by guests on the local network are then moved elsewhere for processing.

    I'm not familiar with media mover so I don't know if it can do that, but you can also accomplish this with a cron job (e.g., runs every 5 minutes, copies files elsewhere, sets permissions and ownership). A Samba disconnect hook/script might also work, but you'll have to experiment with that.

  • I recommend you prevent access to all Samba ports from the Internet using firewall rules.

    Samba runs as root and has had security problems in the past. Note that recent versions of the LAMP appliance have a Webmin firewall module.

You may find it helpful to consult the Samba manual pages for reference, especially if you come across terms in the Webmin samba module you don't understand:
man samba
man smb.conf
The manual pages can be a bit cryptic at times though, so you may also find it helpful to consult with a good book such as Oreilly's "Using Samba". I think it's available as an ebook via various outlets. It's also available in the Safari library which offers a free trial.

Be sure to keep us updated on your progress with this project. If you walk us through the integration process in enough detail we should be able to rebuild the appliance on top of a current version of TurnKey Core and add it to our project's roster of opensource appliances, ready for Beta testing by the community.

In the best opensource tradition, you'll get credit and bragging rights of course.

The main advantage of doing that, as opposed to going the independent route is that once we add an appliance to our project we assume the burden of maintaining it at the appliance level, which frees you and other developers to focus on developing the subcomponents and integration specs for the next versions. For example, we will occasionally release new versions of TurnKey Core that includes the latest standard features, bugfixes and Ubuntu updates. When we do that we also rebuild all appliances on top of the new Core.

Hope this helps, cheers!

szczym's picture

Thanx a lot for the detailed explanation, indeed level of support from the turnkeylinux team is outstanding ! Actually i been hoping to some how stick my small project under your wings on terms, that you have outlined but first i need to make myself familiar with launchpad model of open source collaboration (on day to day basis im video producer using open source tools where its possible). Im going to deploy samba and then struggle with other parts of infrastructure, will update you on progress. Next steps will be installing the Appliance from virtualbox to real hardware and updating it to 2009.02 cheers !
Liraz Siri's picture

Part of whats great about starting small is that so far we've been able to respond to everyone personally and share our expertise. At some point the project may grow enough such that we can't do that anymore (at least not at the same level), but by then we should have better documentation so users have an easier time helping themselves and also we're really hoping the community steps up and we have users and developers helping each each other.

Anyhow, while you're migrating your appliance to 2009.02, try to make detailed technical notes of the integration process - exactly what actions you are taking to turn TurnKey Core into your appliance. The more detailed the better.

It will also be helpful if you provide us with a few tarballs (e.g., of /etc, /var/www, /usr/local), list of packages installed (dpkg -l), SQL dumps of databases and anything else you can think of that would be useful.

We'll use all of that information to reproduce your results and rebuild a new TurnKey appliance on top of Core. Once everything is working we'll put it up on the site for Beta testing and make an announcement.

BTW, joining the turnkey-discuss mailing list might make it easier to collaborate with you on some things as its better suited for long discussions, email makes it easier to attach files plus you don't have to format messages into HTML.

Cheers!

szczym's picture

I have all ready documented process of dev on our wiki pages, mainly to ease start for other multimedia folks, and to have possibility to rollback changes via virtualbox.

https://we.riseup.net/tutturu/fifu-dev
https://we.riseup.net/tutturu/fifu-dev-drupal

sorry for mess and some polish worlds there, is work in progress. any way it will have different torrent component (im planing rtorrent + wtorrent) so those docs are outdated. as fast as i get to some alfa state, im going to publish it here and to my colleges for review.

(The riseup wiki is more like groupware so might seem a bit wired.)
Liraz Siri's picture

I went over the wiki notes and its an excellent start. The main challenge with these kind of instructions is getting the level of detail sufficient to allow someone other than yourself to reproduce your results into a working appliance.

You'll know you've nailed it when your fellow colleagues report that they managed to do that. At that point we'll try to build an alpha/beta appliance and add it to the site to receive wider testing.

Anyhow I think as a supplement to the instructions having tarballs of various relevant directories (/etc /var/www) could be a big help. If you can put a prototype appliance online (e.g., on a file hosting service) in any format (e.g., VirtualBox image) that will be even more helpful as we can study a live appliance interactively to discover additional details as required.

szczym's picture

Helo

I would like to post it all ready so that we cane work on it together, excuse me that im so sluggish but nova days the developer of tracker module for drupal have got active again and seems like port to drupal 6 have moved forward so im testing it how to add it to the web infrastructure via rss (the current tracker is very bad software).

Yes, the instructions cover only basic steps, setup of media mover i hope to provide as import/export code snippets but just nova days i get to the working prototype state so documentation is outdated. In few days i will provide you with working copy.

cheers !

Add new comment