out of inodes | TurnKey GNU/Linux

Hi everyone,

This is my first post so a bit of background first - skip if you want

I've built a web-based coding dojo server which supports about 10 different languages.

It's free, no adverts, no login, instead it asks for donations to raise money to buy Raspberry Pi computers for schools. It's backed by Turnkey Rails. Turnkey is awesome. So far it's early days compared to what I'd like to do but I've more or cobbled stuff together and it seems to work and I'm starting to get some £ in and have already given a few Pi'es to local schools.

</background>

I'm pretty green at linux and admin etc. I've hit a problem that I'm running out of inodes. For example (after reclaiming some disk space on /dev/sda1 where I'm running out)

#df -i

reports

Filesystem Inodes IUsed IFree IUse% Mounted on

/dev/sda1 655360 555214 100146 85% /

none 170404 2018 168386 2% /dev

none 188955 1 188954 1% /dev/shm

none 188955 32 188923 1% /var/run

none 188955 3 188952 1% /var/lock

none 188955 1 188954 1% /lib/init/rw

/dev/sda2 19546112 90 19546022 1% /mnt

So I was hoping I could "move" some of the space on /dev/sda2 to /dev/sda1

I've done a bit of searching and found this

/blog/extending-lvm

which looks promising. However, ssh'ing onto my cyber-dojo server and running

#vgdisplay

#pvdisplay

both report nothing. They don't say command not found. They just report nothing. Is this route an option for me? If so how do I do it?

It turns out that cyber-dojo gobbles disk inodes at a single particular sub-folder. I was wondering whether a simple ln to point that sub-folder to /mnt and so be on dev/sda2 might be a simpler solution? Would that work?

Also, when I go to

https://hub.turnkeylinux.org/servers/

and I click the web-management and web-shell icons on the top right, they both say

This page is not available. How do I get them working?

Thanks

Jon

Replies (7)

Yay!

Reply 1 2012-10-24

Hopefully the same will apply to Webshell although I'm not sure what the service is called (might be 'webshell' or maybe 'shellinabox' or something simmilar...?) For future reference all the services should have scripts in /etc/init.d so have a look in there and hopefully you will find it...

service webmin start

Reply 2 2012-10-23

it works :-) Thanks

So Webmin isn't running...

Reply 3 2012-10-23

So you can try to start it:

service webmin start

And hopefully it will start... If not hopefully it will give you an error message that will help troubleshooting. Otherwise you'll need to see if you can find a logfile that helps out. I had a quick look but I'm not sure where the Webmin logs hang out. Doesn't seem to be in /var/log!?

12321

Reply 4 2012-10-22

I ssh'd onto cyber-dojo.com

netstat -l |grep 12321

returns nothing

service webmin status

also returns nothing... So that's some progress! What now?

I did the mount --bind solution this evening. Worked great.

Found this blog post useful

http://backdrift.org/how-to-use-bind-mounts-in-linux∞

Many thanks

Jon

Again I'm not sure about AWS

Reply 5 2012-10-22

But normally when you format a drive you can set the number of inodes. As you know you will be chewing them up you may be able to preset it to have tons of inodes, especially if you mount it to the specific folder where all the tons of little files will be.

As for Webmin, log in via SSH (PuTTY is a good client if you are using Windows) and try netstat -l to see if it's listening as it should be. I'd run it like this:

netstat -l |grep 12321

and this is the response I get:

tcp        0      0 *:12321                 *:*                     LISTEN
udp        0      0 *:12321                 *:*

and/or see is the Webmin service is running like this:

service webmin status

Mine returns this:

webmin (pid 777) is running

thank you

Reply 6 2012-10-21

Ooops. Thanks for the broken link report. Fixed.

Re inodes - yes - I'm using a _lot_ of small files. Each time the test button is pressed it causes a new git commit.

Re the mount advice - thanks. I will investigate.

The links do indeed point to ports 12321 and 12320 but no joy. Do I have to setup some ssh keys first or something?

Thanks

Jon

I'm not sure if AWS servers use LVM

Reply 7 2012-10-21

Firstly I'd just like to say that your project looks awesome. What a great thing you are doing there! I'll definately have a closer look when time allows. (BTW you have a broken link - the 'practice' link points to http://jonjagger.blogspot.co.uk/p/cyber-dojo_8.html which reports: "Sorry, the page you were looking for in this blog does not exist.")

But back to the problem at hand; TBH I don't know enough about using AWS to provide you with any definative answers but I suspect that AWS hosted appliances don't use LVM.

Also inodes and disk space are not the same (as you've probably already aware) and for the most popular Linux FS (ext3/4) inodes can only be set at format. There is no way to increase the number of inodes whilst retaining the data that I am aware of. Generally inodes get gobbled up with lots of small files.

IMO the best option (rather than linking to /mnt where /dev/sda2 is mounted) would be to 'mount --bind' /mnt to the location that is eating all the inodes. I have never used it but a longer term option may be to use a FS like ReiserFS which can allocate inodes dynamically.

As for your Hub issue where do the links point to? They should either point to https://<public-IP>:12321 (or 12320 respectively) or https://<FQDN>:12321 (or 12320 respectively).