TurnKey Linux Virtual Appliance Library

Making TurnKey more turnkey - the end to default passwords

In our quest to make the upcoming TurnKey 11.0 release more "turnkey", I set out to extend the firstboot inithooks to include application specific configuration hooks such as setting of the admin password, email and domain to serve (where applicable).

I'm glad to announce that the quest is now over, and that puts the end to default passwords.

But what about hosting/cloud deployment where the user doesn't have boot interaction? Well, all the configurations can be pre-seeded. We'll be adding support for preseeding to the Hub soon after the 11.0 release. Until then, the instances will use default passwords, but they can be easily changed by executing the inithooks directly.

While on my quest, it was interesting to get a birds eye view of how the different applications store their passwords, so I thought I'd share:

Comparison table (22 applications)

  clear crypt md4 md5 sha1 salt
tomcat x          
trac   x        
otrs   x        
twiki   x        
mldonkey     x      
joomla       x    
mantis       x    
wordpress       x    
dokuwiki       x    
phpbb       x    
extplorer       x    
gallery       x   x
deki       x   x
mediawiki       x   x
moodle       x   x
prestashop       x   x
magento       x   x
vtiger       x   x
bugzilla       x   x
roundup         x  
redmine         x  
django         x x
 

Foot notes

  • clear: Passwords are stored in clear text.
  • crypt: Passwords are hashed with crypt or htpasswd.
  • md4/md5/sha1: Passwords are put through a cryptographic hash function, which is a deterministic one-way procedure that takes a block of data and returns a fixed-size bit string.
  • salt: A salt (random bits) are added to the password before passing it through the hash function. Some of the applications use a randomly generated salt stored in a configuration file, others calculate it on the fly and add it the hash itself, while others use the user id as the salt. Using a salt is meant to add to the complexity and time it would take an attacker (who obtained the hashed passwords) to determine the original clear-text password.

Now for some code snippets

Just in case you're interested, here are some code snippets as well as the password storage locations. The full code will of course be included in the final 11.0 release.
 

tomcat

doc = xml.dom.minidom.parse(TOMCAT_USERS).documentElement
users = doc.getElementsByTagName('user')
for user in users:
    if not user.getAttribute('username') == 'admin':
        continue

    user.setAttribute('password', password)
FILE: /etc/tomcat6/tomcat-users.xml
 

trac

system("htpasswd -cb %s admin %s" % (authfile, password))
FILE: /etc/trac/htpasswd
 

otrs

hashpass = crypt.crypt(password, 'ro')    # 2 chars of username/email
MYSQL: otrs2.users pw
 

twiki

output = getoutput("htpasswd -bn foo %s" % password)
hashpass = output.split(":")[1].strip()
FILE: /var/www/twiki/data/.htpasswd
 

mldonkey

MD4_HASH=$(echo -n $PASSWORD | openssl dgst -md4 | tr [a-z] [A-Z])
FILE: /var/lib/mldonkey/users.ini
 

joomla

hashpass = hashlib.md5(password).hexdigest()
MYSQL: jos_db.jos_users password
 

mantis

hashpass = hashlib.md5(password).hexdigest()
MYSQL: mantis.mantis_user_table password
 

wordpress

hashpass = hashlib.md5(password).hexdigest()
MYSQL: wordpress.wp_users user_pass
 

dokuwiki

hashpass = hashlib.md5(password).hexdigest()
FILE: /var/lib/dokuwiki/acl/users.auth.php
 

phpbb

hashpass = hashlib.md5(password).hexdigest()
MYSQL: phpbb3.phpbb_users user_password
 

extplorer

MD5_HASH=$(echo -n $PASSWORD | md5sum | cut -d " " -f 1)
hashpass = hashlib.md5(password).hexdigest()
FILE: /var/www/extplorer/config/.htusers.php
 

gallery

salt = "".join(random.choice(string.letters) for line in range(4))
hashpass = salt + hashlib.md5(salt + password).hexdigest()
MYSQL: gallery2.g2_User g_hashedPassword
 

deki

hashpass = hashlib.md5(password).hexdigest()
hashpass = hashlib.md5("1-" + hashpass).hexdigest()     # userid 1
MYSQL: wikidb.users user_password
 

mediawiki

hashpass = hashlib.md5(password).hexdigest()
hashpass = hashlib.md5("1-" + hashpass).hexdigest()     # userid 1
MYSQL: wiki_db.user user_password
 

moodle

for line in file(conffile).readlines():
    m = re.match("\$CFG->passwordsaltmain = '(.*)';", line)
    if m:
        salt = m.group(1)

hashpass = hashlib.md5(password + salt).hexdigest()
MYSQL: moodle.user SET password
CONFFILE: /var/www/moodle/config.php
 

prestashop

for line in file(conffile).readlines():
    m = re.match("define\('_COOKIE_KEY_', '(.*)'", line)
    if m:
        cookie_key = m.group(1)

hashpass = hashlib.md5(cookie_key + password).hexdigest()
MYSQL: prestashop.employee passwd
CONFFILE: /var/www/prestashop/config/settings.inc.php
 

magento

salt = "".join(random.choice(string.letters) for line in range(2))
hashpass = hashlib.md5(salt + password).hexdigest() + ":" + salt
MYSQL: magento.admin_user SET password
 

vtiger

hashpass = hashlib.md5(password).hexdigest()

$salt = substr($username, 0, 2);
$salt = '$1$' . str_pad($salt, 9, '0');
$encrypted_password = crypt($password, $salt);
MYSQL: vtigercrm.vtiger_users user_hash, user_password  
 

bugzilla

$salt = '';
for ( my $i=0 ; $i < 8 ; ++$i ) {
    $salt .= $saltchars[rand(64)];
}

$cryptedpassword = crypt($password, $salt);
MYSQL: bugzilla3.profiles cryptpassword
 

roundup

hashpass = "{SHA}" + hashlib.sha1(password).hexdigest()
MYSQL: roundup._user _password
 

redmine

hashpass = hashlib.sha1(password).hexdigest()
MYSQL: railsapp_production.users hashed_password
 

django

salt = hashlib.sha1(str(random.random())).hexdigest()[:5]
hash = hashlib.sha1(salt + password).hexdigest()
hashpass = 'sha1$%s$%s' % (salt, hash)
MYSQL: django_db.auth_user password
You can get future posts delivered by email or good old-fashioned RSS.
TurnKey also has a presence on Google+, Twitter and Facebook.

Comments

Nice!

Excellent work guys, looking good, I like also that domains are being added where applicable. I'm anxiously waiting for the beta of the new appliances! 

Alon Swartz's picture

Thanks Adrian

It was a lot of work but I think it adds a lot to the user-experience and was worth the effort.

We'll be including Magento, PrestaShop and VTiger with the 11.0 release (coming shortly), as they are ready and no point in holding them back. I've also taken notes for feedback (for you and Basil) which I need to summarize - I'll try get that done soon. 

Looking forward for your feedback

I'm sure I'll learn a couple of lessons from that feedback. I'v already learned a lot helping this project. BTW, I'm missing your feedback on the tklpatch commands of the tkldevenv. I'll upload the entire patch to github so that when you are finally free of the 11pt1 release, you can easily pull/diff those. As you are the author of the original scripts, your feedback is invaluable to the future of that appliance. 

Alon Swartz's picture

Good idea regarding github...

Good idea regarding github, that will make it a lot easier to pull the code. It should also make your life simpler publishing updates, and tracking revisions (which I hope you're already doing).

I'll push out the feedback as soon as I can.

Brilliant

This is brilliant. And knowing how to do that is magical and genious from my point of view. I'm trying to make sense of the code snippets and not getting very far. I understand when values are being passed to MySQL - but take Redmine for example: is that line a part of a configuration file, or is that the value of the field in the MySQL table? Fascinated.


Alon Swartz's picture

Thanks Rik! The code snippets...

Thanks Rik!

The code snippets only show how we take the clear text password and convert it into the desired hash for storage. The code displayed in the post are (mainly) taken from the /usr/lib/inithooks/bin/$APP.py, which is called from the related script in firstboot.d (which may preseed the answers).

Why the separation of code? Unix philosophy - do one thing, and do it well. Take a look (if you haven't already) at the inithook documentation.

To wet your appetite (as you asked so nicely), I've attached the full bin/redmine.py

#!/usr/bin/python
"""Set Redmine admin password and email

Option:
    --pass=     unless provided, will ask interactively
    --email=    unless provided, will ask interactively
"""

import sys
import getopt
import hashlib

from dialog_wrapper import Dialog
from mysqlconf import MySQL

def usage(s=None):
    if s:
        print >> sys.stderr, "Error:", s
    print >> sys.stderr, "Syntax: %s [options]" % sys.argv[0]
    print >> sys.stderr, __doc__
    sys.exit(1)

def main():
    try:
        opts, args = getopt.gnu_getopt(sys.argv[1:], "h",
                                       ['help', 'pass=', 'email='])
    except getopt.GetoptError, e:
        usage(e)

    password = ""
    email = ""
    for opt, val in opts:
        if opt in ('-h', '--help'):
            usage()
        elif opt == '--pass':
            password = val
        elif opt == '--email':
            email = val

    if not password:
        d = Dialog('TurnKey Linux - First boot configuration')
        password = d.get_password(
            "Redmine Password",
            "Enter new password for the Redmine 'admin' account.")

    if not email:
        if 'd' not in locals():
            d = Dialog('TurnKey Linux - First boot configuration')

        email = d.get_email(
            "Redmine Email",
            "Enter email address for the Redmine 'admin' account.",
            "admin@example.com")

    hashpass = hashlib.sha1(password).hexdigest()

    m = MySQL()
    m.execute('UPDATE railsapp_production.users SET mail=\"%s\" WHERE login=\"admin\";' % email)
    m.execute('UPDATE railsapp_production.users SET hashed_password=\"%s\" WHERE login=\"admin\";' % hashpass)

if __name__ == "__main__":
    main()

Alon, Thanks for taking the time

I know how busy you are and really fully appreciate that you took the time to lay some of this out for me. I'm working through Python in a class, and am gonna enjoy digging in see mechanism. Thanks again,

Rik


Are all passwords tested?

This is interesting to no end, that you are able to do this and this efficiently.

The only problem I see is that, I swear, from my research, Wordpress does not store its passwords using md5. They did years ago, if you see ticket 2394. Two changes were accepted changeset was accepted md5 was upgraded to a wp_hash_password() function. The function uses phpass which uses crypt_blowfish and salts the passwords.

So, my only concern is that either I misunderstand how WP hashes their passwords or your passwords might not work!

Keep up the great work though!

Sorry, I didn't proofread.

Sorry, I didn't proofread. That one confusing sentence should say: "Two changes were accepted and the md5() hash was upgraded to a custom wp_hash_password() function."

Alon Swartz's picture

Yes, all are tested and work

Regarding Wordpress, you're right. I got a little lazy with this one as it was towards the end. While going over the code I found that wordpress will update the md5 hash to the internally used hash on login, which is included for backwards compatibility when the user logs on. I should of included a note about that in the summary. Thanks for pointing it out.

I decided to go this route, not only because of laziness, but for the sake of simplicity (not having to re-implement the hashing mechanism in python).

Makes sense... why would you

Makes sense... why would you do it the hard way when they update md5 hashes automatically. Only problem will be if they take that out, but if it's there for backwards compatability, then it should be there for some time to come.

Great work.

Jeremy's picture

Briliant!

Very tidy feature!

Alon Swartz's picture

TurnKey Hub updated to support 11.0 and preseeding

I've been asked via email, on twitter as well on the forums when the 11.0 release is coming out, and I promised before the end of the year. We partly lived up to my promise. All 11.0 ISO's have been released, TKLBAM profiles updated, and Amazon Images (AMI's) uploaded.

And now, before the new year comes in, I pushed out a large update to the Hub which adds:

  • Support for all 11.0 appliances. 2009-10.2 appliances are still available under the legacy option.
  • Support for preseeding 11.0 appliances (launch form updates via AJAX). TKLBAM will also be initialized via preseeding so you don't need to copy/paste the Hub API Key.
  • Support for Asia Pacific - Singapore region (ap-southeast-1).
  • New help/FAQ section.
  • Minor bugfixes

We'll be making the official 11.0 announcement once the VM builds are ready and released, and in that announcement include more details on what has changed in the Hub.

Lastly, if you come across any issues with the Hub, please post a comment or send feedback via the Hub.

DokuWiki Passwords

Just a small correction: DokuWiki supports a wide range of password hashing methods. The installer uses md5, the default for new users is salted md5, but you can also use apr1, sha1, crypt or mysql salts for example. You can even mix different options in the configuration, DokuWiki automatically determines which hash method was used.

See http://www.dokuwiki.org/config:passcrypt for more info on supported hash methods.

Alon Swartz's picture

Good to know

Thanks for posting the comment.

L. Arnold's picture

Question on importing "clear" passwords into Magento -> get hash

I am looking around the web for ways to import batches of customers into Magento and move cleartext passwords into hashed format.

I see your method for hashing the Admin password.

Is there a way, for instance in Open Office or MySQL to convert a clear password to the correct Hashed password?

The only other method I know it to tell customers they have been imported to the new system, and please follow the "lost password" link to set a new one.  This is somewhat untidy it seems.

Thanks for all the great work.  Awesome set of work here!

Way Ahead

I am new to this Virtual Appliances thing, While configuring Turnkey i am strucked at one point, I have failed to figure out the way ahead from first boot confiugration.

After resetting all passwords, I am looking for a command that shall enable me to go to the GUI or whatever is the next step.

You can access your server vai webshell or webmin

You can access your server vai webshell or webmin

See http://www.turnkeylinux.org/core for port numbers and sample screenshots.

Cheers,

Tim (Managing Director - OnePressTech)

Non-English Keyboard Configuration Password Prompt

I installed successfully the "Jenkins" appliance locally in VirtualBox, but I was denied access and banned due to many unsuccessful login attempts, same via SSH (I used the bridged network config. to access it on LAN with Putty) - then I realized : can it be caused by the prompt only accepting input from english (US) keyboard configurations ?

Jeremy's picture

There is an issue with the Jenkins appliance.

See the bug report here and in the forums here (with a workaround). However AFAIK this shouldn't affect SSH login, so perhaps the non-US keyboard is also a culpret...

My suggestion would be to either install with a US keyboard (if you have one) and then change the keyboard to whatever you are using. Or just use a very simple password initially (with keys that will work...), configure your non-US keyboard and then reset the password. I don't have instructions handy on how to set up your non-US keyboard but google should get you going, just keep in mind that TKL v12.x is based on Debian Squeeze...

Also FWIW I have run a poll to ask about setting timezone and/or non-US keyboard on first boot. It is also logged on the TKL Dev traker. There is also a bug report about non-US keyboard causing log in issues (although this seems to be mainly from the web browser).

Thanks for your reply ! I had

Thanks for your reply !

I had already reinstalled Jenkins in a new VM from scratch with a compatible password (SSH login OK), but I'm still strugglin' with the workaround though - created /var/lib/jenkins/users/admin/config.xml with contents indicated in http://www.turnkeylinux.org/forum/support/20130611/jenkins-121-login-form-does-not-worktried loggin'in with 'admin' / 'test' - still got the error... No luck after rebooting either. Tried different permissions on the dirs / file (was root/root, tried jenkins/nogroup 755) + reboot, still no luck.

Jeremy's picture

Perhaps post on that thread?

I am all out of ideas and have no experience with Jenkins. Perhaps someone on that thread has some further ideas for you?

Post new comment

The content of this field is kept private and will not be shown publicly. If you have a Gravatar account, used to display your avatar.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <p> <span> <div> <h1> <h2> <h3> <h4> <h5> <h6> <img> <map> <area> <hr> <br> <br /> <ul> <ol> <li> <dl> <dt> <dd> <table> <tr> <td> <em> <b> <u> <i> <strong> <font> <del> <ins> <sub> <sup> <quote> <blockquote> <pre> <address> <code> <cite> <strike> <caption>

More information about formatting options

Leave this field empty. It's part of a security mechanism.
(Dear spammers: moderators are notified of all new posts. Spam is deleted immediately)