Bin13's picture

Hi,

Firstly, Happy New Year! and thanks for the great work from which I have benefitted so much to date.

I would like to run Run Torrent Server through OpenVPN co-existing on the same core.

Although I realise that there must be previous attempts to do this, I can find no searchable reference to this on your site.

Being still very much a Newbie to Linux, I would appreciate some guidance about how to start, especially the right batting order for a fresh co-installation and anything NOT to do, please?

Would it be possible to create a combined installation ISO, or would that be an unnecessary step?

Thank you in anticipation.

Bin.

Forum: 
Jeremy Davis's picture

Whilst it would be a cool combo, it's not something that I have ever done, so this will be a bit of a case of the blind leading the blind. Having said that, I've been using Linux exclusively for years now and I have deep insights into how TurnKey works, so hopefully I might provide some value... :)

Currently TurnKey servers are monolithic. I.e. each server comes as a single ISO (etc) which includes a complete Linux OS with specific software installed to achieve a specific ends (e.g. torrent server, OpenVPN, etc). We do have plans to switch our model and allow the easy combining of applications into a single server, but we're probably still quite a while away from that (sorry).

In the meantime, you have a few options. Which would be best will depend on where you are planning to run these services, the resources available, how much patience you have and how willing you are to do a deep dive into Linux. TBH, I learned most of what I know by playing with TurnKey years ago. Once I'd developed a bit of Linux experience, I started running it locally (as my desktop) and other than a few initial teething issues, I haven't looked back. Personally I now find Windows so painful to work with! But I digress...

Reading your message, I assume that you wish to run these services locally so I will continue with that assumption. If that's wrong, please correct me. Broadly speaking, you have 2 options:

  1. Install 2 separate TurnKey servers and configure the torrent server to connect to the internet via the VPN provided by the OpenVPN server.
  2. Start with one of the existing appliances and manually install and configure the additional software you wish to use.

Option #1 in some respects is not ideal, as you'll then have 2 separate servers to maintain and configure. IMO it's really only suitable if you have a host where you can run the servers as multiple VMs.

Despite the redundancy and additional overhead, personally I actually prefer that method. Although I have a home server which is a hypervisor (VM host; FWIW I'm running Proxmox) so I like the flexibility that having a raft of separate VMs gives me. Also seeing as Proxmox also provides LXC (Linux Containers - minimalist Linux VMs) I run a lot of stuff under LXC which is really light weight.

A slight variation of that theme would be to use the TurnKey LXC appliance. As per my mention of LXC above, it provides a way to host multiple TurnKey apps as LXC guests. Although LXC does have some limitations and I'm not 100% sure whether OpenVPN would run nicely on it?! (Although I haven't tested and the torrent server should be fine).

So depending on your circumstance, option #2 may be preferable? If I were to go that path, I'd probably start with the OpenVPN appliance because I don't know much about it and Transmission (the torrent client we provide in our current torrent server appliance) is relatively easy to install. But OTOH, I don't use Windows filesharing so wouldn't bother installing anything other than the torrent client, so YMMV.

For reference, the build code of these 2 appliances can be found on GitHub (torrent server & OpenVPN).

Installing Transmission on your OpenVPN server, should be as easy as:

apt-get update
apt-get install transmission-daemon 

You should find the config file in /etc/transmission. Please note that to change any config, please stop the service first, then restart after the changes have been made. Stop a service like this:

service trasmission-daemon stop

To start it again, replace "stop" with "start".

I know that I have really glossed over lots of details, but please feel free to ask specific questions and provide some more context and I'll try to help out as best I can.

Bin13's picture

Hi Jeremy,

Thank you for your kind investment of time and thought.

The Turnkey Torrent Server seems to have led a merry dance lately, changing torrent clients twice in a very short space of time. Sadly, much of the documentation has not caught up, but during periods of significant change, the opportunity to incorporate new but popular features such as VPN client integration does not seem to be such a big ask as it would with a long-term stable product.

Sadly, I still rely heavily of TurnKey howtos and I admit that although I did not feel confident enough to attempt it directly, the thought of strapping OpenVPN or a VPN client onto the package did not seem to register the usual "list of things not to do to something that works".

I really don't have (yet) the sort of metals that support VM and would be venturing into the unknown with that approach, however ideal it would be.

I had not considered two TK machines, but would feel a l;ittle defeated by this method as my objective is to reduce power consumption on a box that runs 24/7.

One of the issues regardsing adding Transmission to the VPN would be the potential loss of Samba?

I have this fear of destabilising everything by messing with anything that runs as efficiently and in such small footprints as TurnKey Appliances do.

However, I do believe that adding a VPN, possibly bi-directional to TurnKey Torrent Server would be, to quote my son AND grandson "the coolest thing", so very slowly and carefully, I intend to try.

My confidence dissolves, when I realise on every occasion that my Linux knowledge would make most newbies feel confident! However .. .. .. .. .

 

Jeremy Davis's picture

Ideally, I would have preferred to keep the rTorrent/ruTorrent appliance and have the Transmission one alongside. But it wasn't to be.

There were a number of users who were really upset that we dropped rTorrent/ruTorrent, but we had way to many complaints about it being unstable. FWIW I couldn't ever reproduce the issues, but we had way to many reports for it to be in people's imagination.

What are the specs of your "always on" box? Assuming that it was built in the last Proxmox. The minimum hardware requirements are x86_64 (i.e. 64 bits) and VT-x/AMD-V cpu extensions. Unless it's one of the early old Atom CPUs, it should support 64 bit.

CPU virtualisation support is a slightly different (and messier) situation. AFAIK AMD have been providing AMD-V on all their chips since the Athlon 64 chips (circa early 2006). So if you have an AMD chip, you should be good.

Intel CPUs have VT-x instead, and their philosophy was a bit more scattered. They first provided it in the later high end Pentium 4 chips (circa late 2005). Many of the Core2Duo chips had it too, but not all of them. I used to have a high end Intel Core2Quad that didn't have it. If you have an Intel chip, I suggest you find out what it is and consult the internet to see if it supports it. Intel still provide the full specs on all their old CPUs last time I checked.

I mention Proxmox, because then you can run each server as it's own VM. So you get the redundancy of having multiple servers (which personally I like) but only need to leave one machine running. I ran a Proxmox server on an old desktop system for years and had up to 20 VMs running no problems (and one was even Windows!) The only reason why I ended upgrading was I was running out of RAM (I had upgrade it as far as possible; 8GB). My new Proxmox rig is a low power octocore Atom with 32 GB RAM. Each VM doesn't perform quite as well as it used to on the old server (slower CPU speed) but with 8 cores and 32GB RAM I have tons of headroom.

Personally, installing Proxmox was probably the single best thing I ever did when I first started exploring Linux (ok, finding TurnKey was probably the best, but it was close second). Being able to spin up a server, clone it, then trash it all in a matter of moments makes playing with Linux so much easier. No need to be scared of breaking things (so long as you have a backup first). And new things can be tried in a disposable VM (worst case, trash it, rinse and repeat).

Even if you'd rather not go that route, there are other ways to skin the cat. Samba can be installed too and whilst it isn't the easiest beast to configure, it's possible. As a hint, I suggest that you install the webmin-samba package as well to make it a little easier to configure.

Also, back when I was using Windows, I used to have WinSSHFS installed. That allows you to mount remote Linux directories over SSH (native and preinstalled on all TurnKey servers). Development had been abandoned and it was a little buggy (on Win7) when the computer woke up after being asleep, but otherwise worked fantastic. Last I checked, someone had adopted it and it was under development again. I assume that the bugs have been fixed.

Bin13's picture

Hi Jeremy, Again, Thank you for a comprehensive reply. Sadly, I am going to have to take a time-out to digest some of it as I really am very new to Linux and the delights that it offers. Equally sadly, in the eyes of my son I am a metal man and VMs are a science that I have avoided to date. The major reason has been not having a machine that that had enough metal not to be seen as a digital joke whilst attempting to intall a VM. I have never looked at Proxmox and after your comments, fully intend to read up on it and try to play with it. I too am in love with TurnKey. Pretty much everything I have tried out has worked, but so much documentation seems missing or out of date to assist with configuration. I know that if I was more familiar with basic Linux, it would be less of a scare. I have eagerly anticipated the trend that seems to be under consideration to make Turnkey Appliances modular. Most times I use a TurnKey Appliance, it would have been perfect if I could have 'bolted on' features available on other TurnKey releases. My original objective was to use TurnKey Torrent Server with a VPN client for all data to and from the internet. Ideally, I wouild wish to add to the Samba server (already in existence on TurnKey Torrent Server) the extra features of TurnKey Media Server (Again not a huge leap.) As a final touch, the features of Turnkey VPN Server for personal external access would be the final touch of pure indulgence. At this point, panic sets in as I am still trying to learn the basics of Linux. By way of feedback, I have had the latest version of TurnKey Torrent Server with Transmission running 24/7 for around 10 days rock solidly with around 4MB / 0.5MB transfer without a murmur. I am using a mobile GUI, "Transmission Remote" and allowing for my rather slow connection (Don't ask.), it has run like a train, albeit a somewhat colourless one. Time to do some more reading and playing with TurnKey Linux. Thanks again and have a great weekend.  
Jeremy Davis's picture

Sorry this is going to be another longish and somewhat rambling one... (I tried, but it blew out...)

Yes I hear you re the docs... It's a serious shortcoming of TurnKey. I've made some efforts to improve that, but it's been spotty at best. The problem is that those that need it can't do it, and those that could do it, often get tied up with other "more important" (and/or more interesting) things!

Something worth keeping in mind though, is that under the hood, TurnKey is Debian (v14.x = Debian Jessie aka 8.x). There are some initial configuration differences and we provide some custom software, but essentially anything that works for Debian should work for TurnKey too. Out of interest, Ubuntu is also based on Debian so instructions for Ubuntu will often be useful too. Although be a bit careful, as Ubuntu and Debian aren't binary compatible (TurnKey IS binary compatible with Debian).

I get your apprehension, but I couldn't recommend Proxmox more highly. For me it helped alleviate many of my concerns. Like I say, the beauty of it is that you can risk trashing a test server, without risking any other servers you already have configured and running.

If you want a hand seeing if your hardware is up to the task, please feel free to share what you have. If you already have Linux running on it, please just share the output of:

cat /proc/cpuinfo | grep -m 1 "model name"

If it's running Windows, then apparently according to google you can do that from the commandline with this:

wmic CPU get NAME
Although YMMV (and I don't have Windows so can't check). Via the GUI you're looking for "System Information". That should give the CPU model number.

Personally I use TKLBAM for backups mostly these days, but Proxmox also provides a built in backup mechanism, so you don't even need to risk the particular server you plan to work on (stuff doesn't work, roll back; rinse, repeat).

Obviously you have plans and goals which you'd like to achieve, but while you're learning, I would encourage you to not get too caught up in "getting it right". If you approach it as "play" with more of a mindset of "experimenting" and "I wonder what happens if I ...?", then you can't lose! Remember; if you're not breaking things, you aren't playing hard enough! :)

Also FWIW, the MediaServer is actually built on top of the fileserver (same as the TorrentServer).

chrizzle23's picture

Broadly speaking, you have 2 options:
  1. Install 2 separate TurnKey servers and configure the torrent server to connect to the internet via the VPN provided by the OpenVPN server.
  I just set up the OpenVPN server portion in Proxmox and I thought I would share my experience.   First off, I am neither an expert in Proxmox, OpenVPN, IPTables, or even Linux in general. I mostly just read docs and hack things together. A major problem for this set up was terminology. There really isn't a commonly understood name for using OpenVPN to connect as a client and then route traffic through that connection. Googling for documents mostly led me to super advanced setups that didn't apply to my situation.   My home network is pretty simple, so I don't need multiple interfaces, a dhcp server, or anything like that. My router is 192.168.1.1 and I wanted to add this VPN router with the IP of 192.168.1.254. The hosts that need access to the VPN tunnel can just use static IP's with the default gateway of 192.168.1.254. Nothing else to set up on the client side. Thankfully I found a document by another pirate with similar interests: https://support.hidemyass.com/hc/en-us/articles/202721486-Using-Linux-Virtual-Machine-instead-of-a-router-for-VPN   I have been using Proxmox for only a short while, and the product is great, but the documentation is woefully out of date. I am also a fan of using Turnkey Linux ISO's to build VM's on other virtualization platforms like VirtualBox and Hyper-V. Proxmox has the ability to run Linux containers, which are very resource efficient, especially with RAM. I decided to use containers instead of full VMs to save resources, but containers can be tricky.  
Although LXC does have some limitations and I'm not 100% sure whether OpenVPN would run nicely on it?!
  The primary issue is that the Turnkey OpenVPN container template has trouble with /dev/net/tun. I think this is a problem with container platforms in general, because I saw a lot of Docker pages about the same problem.   The issue, and the script to fix it, is documented here: https://forum.proxmox.com/threads/turnkey-linux-openvpn-template-issues.31668/   It's just a matter of adding the script to /etc/init.d and you're off to the races.   I set up my Turnkey Linux OpenVPN container template and chose the "Client" configuration. I used SCP to copy the .ovpn file from my provider (AirVPN in my case) to /etc/openvn/airvpn.conf. Then I tested my connection:
openvpn --config /etc/openvpn/airvpn.conf &
Keep in mind that you have to fix the /dev/tun problem or openvpn will not start up.   If your connection is working, you will be able to SSH to your VPN box and use wget to grab a file from a webserver and the server will show your VPN provider's IP.   Once your VPN client connection is working, you want your box to "dial in" when it boots. this is done by editing /etc/default/openvpn and changing one of the AUTOSTART options:
nano /etc/default/openvpn
AUTOSTART="airvpn"
Now that your client connection happens at boot, you need to configure the routing of traffic.   I set my server to have only one interface, eth0. The routing happens between eth0 and tun0, the VPN connection. IThis can be acheived by adding these rules to iptables:
iptables -A FORWARD -o tun0 -i eth0 -s 192.168.1.0/24 -m conntrack --ctstate NEW -j ACCEPT
iptables -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A POSTROUTING -t nat -j MASQUERADE
iptables-save | tee /etc/iptables.up.rules
  That last line is important, the rules file already exists and is called when networking starts in /etc/network/interfaces, so don't change the name.   Once you are sure it's all working, reboot your server to make sure that your modifications were properly saved.   Now that the router part is done, it's time to start building the Torrent Server :-)
Jeremy Davis's picture

That's brilliant!

Thanks so much for posting back with what you discovered!

I've opened a new issue on our tracker to include the init script to create the /dev/tun device!

I wonder if it's also worth considering how we could add your additional IPTable rules too?! TBH, I just haven't played with OpenVPN enough to be sure... If you have any thoughts on it, I'd love to hear.

chrizzle23's picture

Thank you for the compliment, but the real work was done by Wolfgang on the Proxmox forums who wrote the TUN script and Adam Keily with HideMyAss who found the IPTables rules. All I did was rub a few documents together.   As far as including the IPTables rules somewhere, it's not a bad idea. I'm just not confident in my knowledge of OpenVPN to say that is the best way to set up a client. Maybe put them in a document somewhere?   There are docs on Github that discuss the "Gateway" config (https://github.com/turnkeylinux-apps/openvpn/blob/master/docs/gateway.rst) and the "Server" config (https://github.com/turnkeylinux-apps/openvpn/blob/master/docs/site-to-site.rst) but there is no such doc for a "Client" config. My assumption is that the "Server" config and the "Client" config actually mean "Site-to-Site Server" and "Site-to-Site Client" but again, I am not sure how else the experts might use Turnkey and the OpenVPN Appliance to say for certain.   Documentation is tough, that's why it's such a problem for so many open source projects.
Jeremy Davis's picture

I actually thought there were docs missing at some point in the past. That is until I realised that the site-to-site docs covered both OpenVPN server and OpenVPN client (which are both TurnKey OpenVPN servers just to confuse things...)

What I really need to do is sit down with it and set it up myself from scratch and test it out. Then I can be a bit more confident on updating the docs...

But I still think that the tun device creation would be good to add to the LXC containers (and probably docker too) so at least then it will work.

Regarding you 'just' "rub[bing] a few documents together", yeah ok that is somewhat true. But if you hadn't posted and bought that info to my attention, then we wouldn't be able to make the improvement for the benefit of future users (once we update it). So from one perspective, it was a small thing you did (i.e. collecting and sharing information); but from another, your actions will likely have the biggest impact down the line! So you deserve all the recognition I gave you! :)

Docs are an ongoing issue for any small open source project. For us behind the scenes, there are so many jobs that need doing and so few resources. We almost always have "more important" (i.e. higher priority) tasks to take care of, so docs often get neglected. It's also hard to get developers motivated to write up good docs (most developers find them boring and tedious).

As for community volunteers; as a general rule, those that need the docs don't have the knowledge to create them. Those that have the knowledge, don't have the interest, time or energy. So it's a bit of a vicious cycle really.

Anyway, thanks again for you input. And if you did want to document anything, then our docs section (of the website) is actually a wiki. The dedicated OpenVPN appliance docs page is here. You'll need to be logged in, but you should be able to edit that if you wish. You can also make changes to the docs on GitHub, although you'll need a (free) GitHub account to do that. Our "gitflow" is noted here but please feel free to ask if you need more guidance.

Good luck with it all! :)

Chrizzle23's picture

I updated the wiki page for the OpenVPN appliance with the tun0 script and setting up the client for routing.

I should also mention that someone who knows more about routing than me could add a section to the document about "split tunnelling" which is basically routing some of your networks' traffic thru the VPN connection and letting some of it go through the Internet in the clear. This is useful for Site-to-Site VPNs where you just want your internal traffic going through the tunnel and the rest going through the ISP.

I also put a little blurb in the Torrent Server wiki page about using the OpenVPN Appliance in client mode and included a link to the OpenVPN app page.

I just got my torrent server working late last night, and so far it's looking good. I had to do more fiddling with IP tables to get port forwarding to work through the VPN connection.

Using containers for both machines is great. I configured both of them with a single core and 512mb of RAM, but they use about a tenth of that. Being able to pull TKL templates is a huge feature for Proxmox.

Now to get the Media Server going :-)

Jeremy Davis's picture

Great work. Sounds like things are going pretty well for you. Thanks tons for updating those docs, that's brilliant!

I'm also really glad to hear that Proxmox is working out for you. I agree that it's pretty awesome! :)

Bin13's picture

Hi Jeremy and Chrizzle23,

Just checking in to say a big "Thank you!" for contributing to this thread whilst I plod on with the mission of learning about Linux to a more useful extent.

Whereas health, winter and family commitments have soaked up huge amounts of time, I have been spending precious minutes at a time trying.

I (carefully and overlapping) "switched" broadband providers before Christmas and have been awaiting "almost immediate reconnection" throughout the period of 10 painful weeks, with initially mobile (fringe area) and then a temporary connection until the end of February! Needless to say, regardless of almost infinite effort, the whole thing disintegrated.

I am now suffering an ISP provided modem that I cannot change; It cannot work properly (ever!) and I have even had to completely reconfigure my network for it.- Crazy!

Sadly, the machine that I was working on was working so well when I did reconnect, filled the hard disk with test files and then crashed, corrupting the system. I am now working to force Transmission to STOP when it fills it's space less 5GB. Again, the more I discover about Linux, the more I realise that I do not know.

As I said, the big enemy, having finally got my family reconnected to their job, college and school websites and VPNs, has been time. I did learn that one ISP/modem does not equal another when it comes to VPNs!!

Now, I hope that I might have a little time and re-surfacing here seemed a good starting point, if only to apologise for seeming to abandon this thread.

 

Add new comment