TurnKey Linux Virtual Appliance Library

out of inodes

Jon Jagger's picture

Hi everyone,

This is my first post so a bit of background first - skip if you want

<background>

I've built a web-based coding dojo server which supports about 10 different languages.

http://cyber-dojo.com

It's free, no adverts, no login, instead it asks for donations to raise money to buy Raspberry Pi computers for schools. It's backed by Turnkey Rails. Turnkey is awesome. So far it's early days compared to what I'd like to do but I've more or cobbled stuff together and it seems to work and I'm starting to get some £ in and have already given a few Pi'es to local schools.

</background>

I'm pretty green at linux and admin etc. I've hit a problem that I'm running out of inodes. For example (after reclaiming some disk space on /dev/sda1 where I'm running out)

#df -i

reports

 

Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/sda1             655360  555214  100146   85% /
none                  170404    2018  168386    2% /dev
none                  188955       1  188954    1% /dev/shm
none                  188955      32  188923    1% /var/run
none                  188955       3  188952    1% /var/lock
none                  188955       1  188954    1% /lib/init/rw
/dev/sda2            19546112      90 19546022    1% /mnt
 

So I was hoping I could "move" some of the space on /dev/sda2 to /dev/sda1

I've done a bit of searching and found this

http://www.turnkeylinux.org/blog/extending-lvm

which looks promising. However, ssh'ing onto my cyber-dojo server and running 

#vgdisplay

#pvdisplay

both report nothing. They don't say command not found. They just report nothing. Is this route an option for me? If so how do I do it?

 

It turns out that cyber-dojo gobbles disk inodes at a single particular sub-folder. I was wondering whether a simple ln to point that sub-folder to /mnt and so be on dev/sda2 might be a simpler solution? Would that work?

 

Also, when I go to

https://hub.turnkeylinux.org/servers/

and I click the web-management and web-shell icons on the top right, they both say

This page is not available. How do I get them working? 

 

Thanks

Jon

Jeremy's picture

I'm not sure if AWS servers use LVM

Firstly I'd just like to say that your project looks awesome. What a great thing you are doing there! I'll definately have a closer look when time allows. (BTW you have a broken link - the 'practice' link points to http://jonjagger.blogspot.co.uk/p/cyber-dojo_8.html which reports: "Sorry, the page you were looking for in this blog does not exist.")

But back to the problem at hand; TBH I don't know enough about using AWS to provide you with any definative answers but I suspect that AWS hosted appliances don't use LVM.

Also inodes and disk space are not the same (as you've probably already aware) and for the most popular Linux FS (ext3/4) inodes can only be set at format. There is no way to increase the number of inodes whilst retaining the data that I am aware of. Generally inodes get gobbled up with lots of small files. 

IMO the best option (rather than linking to /mnt where /dev/sda2 is mounted) would be to 'mount --bind' /mnt to the location that is eating all the inodes. I have never used it but a longer term option may be to use a FS like ReiserFS which can allocate inodes dynamically.

As for your Hub issue where do the links point to? They should either point to https://<public-IP>:12321 (or 12320 respectively) or https://<FQDN>:12321 (or 12320 respectively).

Jon Jagger's picture

thank you

Ooops. Thanks for the broken link report. Fixed.

Re inodes - yes - I'm using a _lot_ of small files. Each time the test button is pressed it causes a new git commit.

Re the mount advice - thanks. I will investigate.

The links do indeed point to ports 12321 and 12320 but no joy. Do I have to setup some ssh keys first or something?

Thanks

Jon

Jeremy's picture

Again I'm not sure about AWS

But normally when you format a drive you can set the number of inodes. As you know you will be chewing them up you may be able to preset it to have tons of inodes, especially if you mount it to the specific folder where all the tons of little files will be.

As for Webmin, log in via SSH (PuTTY is a good client if you are using Windows) and try netstat -l to see if it's listening as it should be. I'd run it like this:

netstat -l |grep 12321

and this is the response I get:

tcp        0      0 *:12321                 *:*                     LISTEN     
udp        0      0 *:12321                 *:*                                

and/or see is the Webmin service is running like this:

service webmin status

Mine returns this:

webmin (pid 777) is running
Jon Jagger's picture

12321

 

I ssh'd onto cyber-dojo.com

 

netstat -l |grep 12321

returns nothing

 

service webmin status

also returns nothing... So that's some progress!  What now?

 

I did the mount --bind solution this evening. Worked great.

Found this blog post useful

http://backdrift.org/how-to-use-bind-mounts-in-linux

Many thanks

Jon

 

 

Jeremy's picture

So Webmin isn't running...

So you can try to start it:

service webmin start

And hopefully it will start... If not hopefully it will give you an error message that will help troubleshooting. Otherwise you'll need to see if you can find a logfile that helps out. I had a quick look but I'm not sure where the Webmin logs hang out. Doesn't seem to be in /var/log!?

Jon Jagger's picture

service webmin start

it works :-)  Thanks

Jeremy's picture

Yay!

Hopefully the same will apply to Webshell although I'm not sure what the service is called (might be 'webshell' or maybe 'shellinabox' or something simmilar...?) For future reference all the services should have scripts in /etc/init.d so have a look in there and hopefully you will find it...

Post new comment

The content of this field is kept private and will not be shown publicly. If you have a Gravatar account, used to display your avatar.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <p> <span> <div> <h1> <h2> <h3> <h4> <h5> <h6> <img> <map> <area> <hr> <br> <br /> <ul> <ol> <li> <dl> <dt> <dd> <table> <tr> <td> <em> <b> <u> <i> <strong> <font> <del> <ins> <sub> <sup> <quote> <blockquote> <pre> <address> <code> <cite> <strike> <caption>

More information about formatting options

Leave this field empty. It's part of a security mechanism.
(Dear spammers: moderators are notified of all new posts. Spam is deleted immediately)