DevOps and TurnKey Linux

I'm super excited as the guys have released the 'buildtasks' publicly. Its the code that builds all the different build formats... TBH I have no idea how it works (or how you apply it) but I'll be having a look really soon. If you beat me to it, please feel free to post back anything you glean (up to you but if you'd rather a new thread than taking this one further away from it's intention please do so...)

Oops almost forgot the link: https://github.com/turnkeylinux/buildtasks

Thanks Jeremy, I had forgotten I had upload rights there.  I've uploaded a copy of the latest ISO file into a new folder, turnkey-ansible-test.  http://sourceforge.net/projects/turnkeylinuxcom/files/turnkey-ansible-test/turnkey-ansible-13.1-rc4-wheezy-amd64.iso/download

Hopefully, other interested parties will give it a try, kick the tires, and provide some feedback.  I'll post a quick getting started guide below.

But it looks like you are still there! :) As is the TurnKey community downloads page...

You already had upload rights, but I've just added you to the admins as well.

And FWIW the SF login page is now: https://sourceforge.net/auth/

As for the TKLDev output; currently ISO is all that TKLDev will output. I have been pushing Liraz to get the other build code public ASAP. It's currently all tangled up with some sensitive stuff that can't be public. The build code is part of the automated program that turns the ISOs that came out of TKLDev into all the other build formats and uploads direct to the TurnKey mirror network. Once the code that builds the other formats is public then people will be able to produce any format they desire. It will make it easier for new formats to be supported (or current formats to be improved) too. I have also suggested that the (TKLDev) build process stop at an earlier generic point (rather than building an ISO by default) as IMO making an ISO then converting that to other formats seems a bit redundant (especially OVZ and LXC which are really just archived root directories...). Anyway we'll need to wait for all that...

I'm not sure when the change occurred.  It's been over a year since I last visited.  If I go to http://sourceforge.net/account/login.php, I get page not found.  

I just figured out that I need to create a new project, in order to get the login screen.  I could go ahead and create a project and upload the iso there, but I like your idea of a "Community" shared upload facility better. My TKLdev setup can only build iso's and I would really like to test other formats, i.e. LXC.  Would have to figure out how to upload a build package rather than an iso if your suggestion is to work.  Maybe a tar-zip of the project directory excluding the build directory?  Or maybe the files could be pulled from the developer's github account.  Submit your project's url, wait a few minutes, and then download the result in whatever format you want.  Just a thought.

Late last year (or perhaps even early this year) I logged in again (it's been quite a long time...) and all seemed to be as per usual. I enable another user upload rights to upload an ISO there which he did and all seemed to go ok. So unless something has happening in the last month or so I'd still think that's the best option.

If SourceForge is definitely no longer any good, then perhaps we need to look at a TurnKey "Community" upload/download (or better still build) facility. That would be ideal really... If appliances could be designed and tested by community members then just built by a cloud hosted TKLDev which spits out all build types...

Thanks Jeremy.  Since most people won't want to take the trouble to setup TKLdev just to build an ISO for testing, I'm looking for a place to upload and store the image files.  SourceForge has been revised and I'm not sure my account there is still available.  I think GitHub still frowns on uploading ISO file there.  What is the current best practice for sharing image files for testing?

Sounds like a well thought out and tested solution. Obviously software from the repos is preferred, but sometimes it's not the best option. With your testing and rationale I see no reason why your suggested way to go wouldn't be the best option...

I'll give Alon and/or Liraz a nudge to have a look at this and see what they think. I think that the idea is sound but I will defer to them on implementation, etc.

As I pondered the pros and cons of installing packages via apt-get versus git or pip, I realized that while using either git or pip for installation provides access to the latest stable releases, it also shifts the burden of managing updates and upgrades to the TKLdev developer.  As I'm writing this, it occurred to me that there is also the matter of security, which we tend to forget since it has long been handled by the apt package distribution system (repositories and keys).

I took a stab at the problem of managing updates/upgrades by writing a wrapper script for /usr/lib/inithooks/firstboot.d/95secupdates.  To allow for appliance specific updates, I added a directory, /etc/turnkey/update.d/ where appliance specific scripts can be placed.  Note: it does not yet distinguish security updates.

/usr/bin/turnkey-update

#! /bin/bash
# ---------------------------------------------------------------------------
# turnkey-update - Download and apply package updates

# Copyright 2014, John Carver <dude4linux@gmail.com>

  # This program is free software: you can redistribute it and/or modify
  # it under the terms of the GNU General Public License as published by
  # the Free Software Foundation, either version 3 of the License, or
  # (at your option) any later version.

  # This program is distributed in the hope that it will be useful,
  # but WITHOUT ANY WARRANTY; without even the implied warranty of
  # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  # GNU General Public License at (http://www.gnu.org/licenses/) for
  # more details.

# Usage: turnkey-update [-h|--help] [-a|--all] [-s|--secure]

# Revision history:
# 2014-07-23 Created by new_script ver. 3.0
# 2015-02-14 Display help even if not root
# 2015-02-15 Generalized to run scripts in /etc/turnkey/update.d/
# ---------------------------------------------------------------------------

PROGNAME=${0##*/}
VERSION="0.3"

UPDATE_DIR="/etc/turnkey/update.d"

clean_up() { # Perform pre-exit housekeeping
  return
}

error_exit() {
  echo -e "${PROGNAME}: ${1:-"Unknown Error"}" >&2
  clean_up
  exit 1
}

graceful_exit() {
  clean_up
  exit
}

signal_exit() { # Handle trapped signals
  case $1 in
    INT)    error_exit "Program interrupted by user" ;;
    TERM)   echo -e "\n$PROGNAME: Program terminated" >&2 ; graceful_exit ;;
    *)      error_exit "$PROGNAME: Terminating on unknown signal" ;;
  esac
}

usage() {
  echo -e "Usage: $PROGNAME [-h|--help] [-a|--all] [-s|--secure]"
}

help_message() {
  cat <<- _EOF_
  $PROGNAME ver. $VERSION
  Download and apply package updates

  $(usage)

  Options:
    -h, --help          Display this help message and exit.
    -a, --all           Apply all updates
    -s, --secure        Apply only security updates (default)

  NOTE: You must be the superuser to run this script.

_EOF_
  return
}

# Trap signals
trap "signal_exit TERM" TERM HUP
trap "signal_exit INT"  INT

all="false";

# Parse command-line
while [[ -n $1 ]]; do
  case $1 in
    -h | --help)        help_message; graceful_exit ;;
    -a | --all)         echo "Apply all updates"; all="true" ;;
    -s | --secure)      echo "Apply only security updates" ;;
    -* | --*)           usage; error_exit "Unknown option $1" ;;
    *)                  echo "Argument $1 to process..." ;;
  esac
  shift
done

# Main logic

# Check for root UID
if [[ $(id -u) != 0 ]]; then
  error_exit "You must be the superuser to run this script."
fi

if [[ $all == "false" ]]; then
  export SEC_UPDATES="FORCE";
  /usr/lib/inithooks/firstboot.d/95secupdates
else
  apt-get update  -qq
  apt-get upgrade -qy
  if [[ -d $UPDATE_DIR ]]; then
    run-parts --lsbsysinit $UPDATE_DIR
  fi
fi

graceful_exit

 /etc/turnkey/update.d/ansible

#! /bin/bash
if [[ -f $(which pip) ]]; then
  pip install ansible --upgrade
fi

1. Download and install the ISO
   Download the latest release ISO from SourceForge Turnkey Linux Community (here). Install the ISO in your favorite virtual machine, VMware, VirtualBox, ProxMox, or whatever.  Note: The virtual machine must be capable of installing from an ISO, so OpenVZ, LXC, etc are not yet supported. During the install process you will be prompted to enter passphrases for root and ansible.
2. Login as user ansible
   The first time you login as ansible you will be prompted by keychain to enter the ansible password again.  This should only happen the first time you login after a reboot. Keychain is starting the ssh-agent and storing the passphrase so Ansible scripts don't keep prompting you for the passphrase. I could not figure out how to make it seemless without entering the password twice.

           /tklbam
   
   Last login: Wed Feb 18 16:09:04 2015 from 192.168.1.108
   
    * keychain 2.7.1 ~ http://www.funtoo.org
    * Starting ssh-agent...
    * Adding 1 ssh key(s): /home/ansible/.ssh/id_rsa
   Enter passphrase for /home/ansible/.ssh/id_rsa:
    * ssh-add: Identities added: /home/ansible/.ssh/id_rsa

   On subsequent logins, keychain will find the running ssh-agent.

           /tklbam
   
   Last login: Wed Feb 18 16:11:51 2015 from 192.168.1.108
   
    * keychain 2.7.1 ~ http://www.funtoo.org
    * Found existing ssh-agent: 21386
    * Known ssh key: /home/ansible/.ssh/id_rsa

3. Test ansible
   ​Check that ansible is up and running.

   ansible@ansible ~$ ansible all -m ping
   ansible | success >> {
       "changed": false,
       "ping": "pong"
   }

4. Add more hosts

   Edit ansible's inventory file, currently /etc/ansible/hosts, and add some additional hosts for testing. I recommend adding only disposable test machines until you are very comfortable with how ansible operates.  For now, you will have to manually create and install the test machines. Later I hope to automate the creation of test machines using LXC.  I'm also assuming that all test machines are TurnKey/GNU appliances.
   
   Examples for hosts and groups can be found in /etc/ansible/hosts.example.  We will create a group named 'test' and add a few test hosts.  Hosts can be identified by IP address or by name.  Host names must be registered either in /etc/hosts or in DNS.  You must be able to ssh to each of the test hosts and login as root.  I recommend you do this now for each test host so it's entered into the known_hosts file. I'm assuming that all of the root passwords are the same, otherwise you'll have to run the following step 6. once for each host. My inventory file now looks like this:

   ...
   ansible               ansible_connection=local
   
   # define the test group
   [test]
   gitlab
   jenkins
   lxc
   

5. Create init playbook
   In the ansible home directory, make a directory for ansible playbooks and copy the following yaml script into ~/playbooks/init.yml

   ---
   
   # playbooks/init.yml
   #
   # This playbook creates initial admin account and ensures that ansible can
   # access the server through sudo. You should run it just after new server has
   # been installed and connected to the network. Make sure that you can access
   # it through ssh on the root account. You might have to provide a password (in
   # that case use '-k' switch on the command line).
   #
   # Usage:
   # ansible-playbook -k playbooks/init.yml -l "<hostname>"
   #
   
   - hosts: all
     remote_user: root
     sudo: no
     gather_facts: False
     tags: init
     vars:
       ssh_user: '{{ lookup("env", "USER") }}'
       ssh_key: '{{ lookup("file", "~/.ssh/id_rsa.pub") }}'
   
     tasks:
     - name: INIT | Install python support for Ansible
       raw: apt-get -yq install python python-apt
       tags: init
   
     - action: setup
       tags: init
   
     - name: INIT | Create admin system group
       group:
         name: 'admins'
         state: 'present'
         system: True
       tags: init
   
     - name: INIT | Create admin account from current user
       user:
         name: '{{ ssh_user }}'
         state: 'present'
         shell: '/bin/bash'
         groups: 'admins'
       tags: init
   
     - name: INIT | Make sure essential software is installed
       apt:
         name: '{{ item }}'
         state: 'latest'
         install_recommends: False
       with_items:
         - python
         - python-apt
         - sudo
         - lsb-release
       tags: init
   
     - name: INIT | Install ssh public key from current account
       authorized_key:
         user: '{{ ssh_user }}'
         key: '{{ ssh_key }}'
         state: 'present'
       failed_when: (ssh_key is undefined or (ssh_key is defined and not ssh_key))
       tags: init
   
     - name: INIT | Configure admins group access in sudo
       lineinfile:
         dest: '/etc/sudoers.d/admins'
         state: 'present'
         create: True
         regexp: '^%admins'
         line: '%admins ALL=(ALL:ALL) NOPASSWD: SETENV: ALL'
         owner: 'root'
         group: 'root'
         mode: '0440'
         validate: 'visudo -cf %s'
       tags: init

6. Init the test hosts
   In order to work it's magic, ansible needs root access to your test hosts. The init playbook will first make sure that python and python-apt packages are installed. It will then create an admins group and grant it's members passwordless sudo access.  Next it creates an ansible user, makes it a member of the admins group, and adds it's own ssh key to the ~/.ssh/authorized_keys file.
   
   Warning: Before adding any production hosts to the inventory file, make sure you have taken precautions to secure the Ansible appliance and properly protected access to both the root and ansible accounts.
   
   Run ansible-playbook to execute the init.yml playbook.  The '-k' option prompts for a password.  At the prompt, enter the root password of the 'test' hosts, not the password for the ansible user.  The '-l' option limits the action to the 'test' group defined in the inventory file because the init.yml playbook specifies 'all' hosts, otherwise ansible would try to initialize itself.

   ansible@ansible ~$ ansible-playbook -k playbooks/init.yml -l "test"
   SSH password:
   
   PLAY [all] ********************************************************************
   
   TASK: [INIT | Install python support for Ansible] *****************************
   ok: [lxc]
   ok: [jenkins]
   ok: [gitlab]
   
   TASK: [setup] *****************************************************************
   ok: [lxc]
   ok: [jenkins]
   ok: [gitlab]
   
   TASK: [INIT | Create admin system group] **************************************
   ok: [lxc]
   ok: [gitlab]
   ok: [jenkins]
   
   TASK: [INIT | Create admin account from current user] *************************
   ok: [lxc]
   ok: [gitlab]
   ok: [jenkins]
   
   TASK: [INIT | Make sure essential software is installed] **********************
   ok: [lxc] => (item=python,python-apt,sudo,lsb-release)
   ok: [jenkins] => (item=python,python-apt,sudo,lsb-release)
   ok: [gitlab] => (item=python,python-apt,sudo,lsb-release)
   
   TASK: [INIT | Install ssh public key from current account] ********************
   changed: [lxc]
   changed: [jenkins]
   changed: [gitlab]
   
   TASK: [INIT | Configure admins group access in sudo] **************************
   ok: [jenkins]
   ok: [gitlab]
   ok: [lxc]
   
   PLAY RECAP ********************************************************************
   gitlab                     : ok=7    changed=1    unreachable=0    failed=0
   jenkins                    : ok=7    changed=1    unreachable=0    failed=0
   lxc                        : ok=7    changed=1    unreachable=0    failed=0   

   Note that most of the tasks report "ok:".  This was because they had been completed by an earlier run on a previous version of the ansible server.  The only thing that needed to change was the installation of the ssh public key from the ansible account of the freshly installed server. 

7. Run adhoc commands
   Now that the ssh key has been installed, we should be able to run some adhoc commands using ansible.

   ansible@ansible ~$ ansible all -a turnkey-version
   ansible | success | rc=0 >>
   turnkey-ansible-13.1-rc4-wheezy-amd64
   
   lxc | success | rc=0 >>
   turnkey-lxc-13.0-wheezy-amd64
   
   gitlab | success | rc=0 >>
   turnkey-gitlab-13.0-wheezy-amd64
   
   jenkins | success | rc=0 >>
   turnkey-jenkins-13.0-wheezy-amd64
   
   ansible@ansible ~$ ansible all -m ping
   ansible | success >> {
       "changed": false,
       "ping": "pong"
   }
   
   jenkins | success >> {
       "changed": false,
       "ping": "pong"
   }
   
   lxc | success >> {
       "changed": false,
       "ping": "pong"
   }
   
   gitlab | success >> {
       "changed": false,
       "ping": "pong"
   }

8. Experiment with Ansible Galaxy

   Now you may want to experiment with ansible-galaxy to download a role or create a new one. Browse Ansible Galaxy to find an interesting role to download e.g. lxc

   ansible@ansible ~$ mkdir roles
   ansible@ansible ~$ ansible-galaxy install deimosfr.lxc -p roles
   - downloading role 'lxc', owned by deimosfr
   - downloading role from https://github.com/deimosfr/ansible-lxc/archive/v1.4.tar.gz
   - extracting deimosfr.lxc to roles/deimosfr.lxc
   - deimosfr.lxc was installed successfully
   ansible@ansible ~$ tree roles
   roles
   `-- deimosfr.lxc
       |-- LICENSE
       |-- README.md
       |-- defaults
       |   `-- main.yml
       |-- files
       |   |-- kernel.list
       |   |-- lxc-convert
       |   |-- lxc-debian-0.9.0-upstream-squeeze
       |   |-- lxc-debian-0.9.0-upstream-wheezy
       |   `-- lxc-resources-stats
       |-- handlers
       |   `-- main.yml
       |-- meta
       |   `-- main.yml
       |-- tasks
       |   |-- lxc_install.yml
       |   |-- main.yml
       |   `-- prequesite.yml
       `-- templates
           |-- dnsmasq.conf.j2
           |-- interfaces.j2
           |-- interfaces_bridge.j2
           |-- interfaces_nat.j2
           `-- lxc-template.conf.j2
   
   7 directories, 18 files

   Make your own role.

   ansible@ansible ~$ ansible-galaxy init my-first-role
   - my-first-role was created successfully
   ansible@ansible ~$ tree my-first-role/
   my-first-role/
   |-- README.md
   |-- defaults
   |   `-- main.yml
   |-- files
   |-- handlers
   |   `-- main.yml
   |-- meta
   |   `-- main.yml
   |-- tasks
   |   `-- main.yml
   |-- templates
   `-- vars
       `-- main.yml
   
   7 directories, 6 files

9. Enjoy

   I hope this QuickStart Guide has been helpful.

    

I've just pushed the rc4 version of the Ansible appliance.  The first versions of the appliance used apt-get to install some of the python packages required by Ansible, but used pip to install Ansible itself.  This led to a situation with a mix of older (from apt) and newer (from pip) packages.  The result was a lot of warning messages when trying to upgrade the python packages.  Version rc3 was an attempt to avoid this by installing everything using apt, but that led to other problems.  The latest version, rc4, bootstraps the installation of pip, and then uses pip to install Ansible and all of it's dependencies.  I'm hoping this will be the easiest to keep updated.

I've just pushed the rc3 version of the Ansible appliance to https://github.com/Dude4Linux/ansible.  It is running the current stable release of Ansible, version 1.8.2.

There is still some more development work to be done before final release.  The ToDo list includes:

1. Install the experimental ansible-lxc module and configure it to manage containers on a TurnKey LXC appliance.
2. Develop a turnkey module for managing TurnKey appliances.
3. Combine GitLab, Ansible, LXC, and Jenkins appliances configured for DevOps workflow.

Ansible is an IT automation tool. It can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments or zero downtime rolling updates.

The Ansible TurnKey GNU/Linux appliance has been released as part of the 14.0 stable suite of appliances.