Blog Tags:

Extending an LVM volume: Physical volumes (partitions) -> Volume groups -> Logical volume -> Filesystem

Liraz Siri - Mon, 2011/02/21 - 15:01 - 84 comments

Logical Volume Management (AKA LVM) is a powerful, robust mechanism for managing storage space.

In TurnKey 11, instead of installing the root filesystem directly to a fixed size partition, we setup LVM by default, and install the root filesystem to a Logical Volume, which may later be expanded, even across multiple physical devices.

Unfortunately, as with anything powerful, to get the most out of LVM you first have to negotiate a learning curve. From the feedback we've been getting it seems that confusion regarding LVM is common with new users, so here's a quick "crash course"...

How LVM works

In LVM, there are several layers, each builds on top of the other:

PV[s] (Physical Volumes) -> VG[s] (Volume Groups) -> LV[s] (Logical Volumes) -> Filesystems.

Logical Volumes are allocated/extended within the boundaries of their underlying storage pool which is called a Volume Group in LVM terminology.

For example, in TurnKey the filesystem is installed by default to the /dev/turnkey/root Logical Volume, which is allocated within the turnkey Volume Group:

--- Logical volume ---
  LV Name                /dev/turnkey/root
  VG Name                turnkey
  LV UUID                OzX3fe-aRQa-81XM-0vCV-8aJo-eUL4-6J90XJ
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                17.0 GiB
  Current LE             4502
  Segments               2
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:0

Out of the box the turnkey Volume Group doesn't have too much free space:

# vgdisplay
  --- Volume group ---
  VG Name               turnkey
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               18.14 GiB
  PE Size               4.00 MiB
  Total PE              4645
  Alloc PE / Size       4480 / 17.50 GiB
  Free  PE / Size       165 / 660.00 MiB
  VG UUID               IwaFL0-QCi8-iIUE-TWjQ-R906-PYpH-gMPaH9

We can only extend a Logical Volume within the free space of the underlying Volume Group. How much free space we currently have within the Volume Group can be seen in this part of the output:

Free  PE / Size       165 / 660.00 MiB

In the above example we only have 660 MB to allocate to LVMs within the turnkey Volume Group. So if we want to extend the root LV we'll have to first extend the VG backs it up.

Volume Groups group together Physical Volumes. That's why they're called Volume Groups. This command will show us which Physical Volumes have been registered into LVM, and to which volume groups they have been assigned:

# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               turnkey
  PV Size               18.15 GiB / not usable 4.00 MiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              4645
  Free PE               165
  Allocated PE          4480
  PV UUID               H1Prpv-0VXR-7moE-zsbt-eyVt-m0fQ-GkAT6w

In this example we only have one Physical Volume (the /dev/sda2 partition) in the turnkey Volume Group.

Extending a Logical Volume

Bottom line: if the underlying Volume Group doesn't have enough free space, to extend the Logical Volume you'll first have to extend the underlying Volume Group by adding another Physical Volume to it.

In VMWare you could either create a new virtual hard disk device to add to the volume group, or extend an existing virtual hard disk device, create a new partition with cfdisk, and add the new partition to the Volume Group:

# example #1: you've added to VMWare a new virtual hard disk called /dev/sdb
pvcreate /dev/sdb
vgextend turnkey /dev/sdb

# example #2: you've expanded the existing sda hard disk
cfdisk /dev/sda  # creating /dev/sda3 (you may need to reboot before you can see this)
pvcreate /dev/sda3
vgextend turnkey /dev/sda3

After you've extended the Volume Group, you are free to extend the underlying Logical Volume:

# lvextend -L+10G /dev/turnkey/root
Extending logical volume root to 27.0 GiB
Logical volume root successfully resized

Finally, you'll have to resize the filesystem within /dev/turnkey/root so it can see that the underlying block device just got 10G bigger:

# resize2fs /dev/turnkey/root
resize2fs 1.41.11 (14-Mar-2010)
Filesystem at /dev/turnkey/root is mounted on /; on-line resizing required
old desc_blocks = 2, new_desc_blocks = 2
Performing an on-line resize of /dev/turnkey/root to  7077888 (4k) blocks.
The filesystem on /dev/turnkey/root is now 7077888 blocks long.

Have you tried using LVM? Still confused? Post a comment to discuss.

Comments

Is it better to mount an additional storage partition?

Jeremiah - Mon, 2011/02/21 - 16:10

One of the first things I did with my TKL lamp server was increase the size of the root partition but in hindsight I've wondered if it would be better to leave it's size alone and mount an additional storage partition. Of course you can still use LVM to increase the second storage partition as needed.

It seems that would be a simpler method to manage storage and that method should work as well whether I am running the server in VMware or in Amazon EC2. It also seems like the separation of large data from the TKL appliance would inherently make it simpler to swap TKL appliances if needed.

More importantly, I'm wondering if there are any issues with the way Amazon instances and storage space are handled that would make either extending root partition or mounting an additional partition a more sensible approach for adding storage space. I haven't used Amazon's EC2 yet so I don't have any hands-on experience but my impression is that it would be simpler to keep the root partition as standard as possible and just mount additional storage partitions as needed.

Anyone have any insight they'd like to share?

Advanced configurations can be better, but are also more complex

Liraz Siri - Mon, 2011/02/21 - 16:26

In general, I think it's a good practice to partition the system stuff and big data separately, but it's also more complex to setup and maintain, which is why TurnKey defaults to one big root partition.

If you install from the ISO the di-live installer allows you to setup a more advanced configuration if you want. TurnKey is Ubuntu under the hood so any partitioning scheme you would use for Ubuntu would also work with TurnKey. You may have to do some reconfiguring though (e.g., creating new logical volumes, moving data over, configuring mount points in fstab, etc.)

Regarding Amazon EC2, it lets you do exactly what you want via EBS (Elastic Block Store), which are high-speed SAN backed virtual storage devices that can be created and attached to an EC2 server instance on the fly. The Hub makes it very easy to use this functionality. You should try it.

Thanks. I agree that having

Jeremiah - Mon, 2011/02/21 - 16:46

Thanks. I agree that having one partition for TurnKey makes the most sense as a default.

I'll definitely get around to trying EC2 so I can get a feel for things. I especially need to figure out how easy it is to add space to the EC2 instance's root partition.

0% available

DRUMDUDESAN - Tue, 2015/04/21 - 00:05

Hi,

I know this is an old post but I inherited Turnkey and root is out if disk space and mysql will not load. I followed your article and it worked: but still 0% available, I think I am overlooking something simple. Any ideas? Thanks.

root@aserver~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/turnkey-root
22G 21G 0 100% /
none 120M 184K 120M 1% /dev
none 123M 0 123M 0% /dev/shm
none 123M 56K 123M 1% /var/run
none 123M 0 123M 0% /var/lock
none 123M 0 123M 0% /lib/init/rw
/dev/sda1 473M 437M 12M 98% /boot

root@aserver ~# fdisk -l

Disk /dev/sda: 53.7 GB, 53687091200 bytes
4 heads, 32 sectors/track, 819200 cylinders
Units = cylinders of 128 * 512 = 65536 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000339d8

Device Boot Start End Blocks Id System
/dev/sda1 * 1 7813 500000 83 Linux
/dev/sda2 7825 305168 19030016 8e Linux LVM
/dev/sda3 305169 573440 17169408 83 Linux
/dev/sda4 573441 819200 15728640 83 Linux

Disk /dev/sdb: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdb doesn't contain a valid partition table
root@aserver ~# ls /dev/sd*
/dev/sda /dev/sda1 /dev/sda2 /dev/sda3 /dev/sda4 /dev/sdb
root@aserver ~# pvcreate /dev/sda4
Physical volume "/dev/sda4" successfully created
root@aserver ~# vgextend turnkey /dev/sda4
Volume group "turnkey" successfully extended
root@aserver ~# lvextend -L+15 /dev/turnkey/root
Extending logical volume root to 37.02 GiB
Logical volume root successfully resized
root@aserver ~# resize
resize2fs resizecons
root@aserver ~# resize2fs /dev/turnkey/root
resize2fs 1.41.11 (14-Mar-2010)
Filesystem at /dev/turnkey/root is mounted on /; on-line resizing required
old desc_blocks = 2, new_desc_blocks = 2
Performing an on-line resize of /dev/turnkey/root to 5771264 (4k) blocks.
The filesystem on /dev/turnkey/root is now 5771264 blocks long.

root@aserver~# vgdisplay
--- Volume group ---
VG Name turnkey
System ID
Format lvm2
Metadata Areas 4
Metadata Sequence No 15
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 3
Open LV 2
Max PV 0
Cur PV 4
Act PV 4
VG Size 57.51 GiB
PE Size 4.00 MiB
Total PE 14722
Alloc PE / Size 10116 / 39.52 GiB
Free PE / Size 4606 / 17.99 GiB
VG UUID xDapZk-53wD-mIKE-5OZQ-zd3M-EggS-enn5xC

Simplest would be ...

Landis Arnold - Tue, 2015/04/21 - 00:17

I would make a new App with the space you need somewhere, then migrate the old data in with TKLBAM.

You need to assign the API key to both.

Extending the Filesystem I did do once but forgot now how.

Easiest way to make a bigger install is to install from ISO then determine your HD/RAM and other sizes at start up.

My experience anyway.

Easy Fixed

DRUMDUDESAN - Tue, 2015/04/21 - 18:17

Noticed there was another sda3 that was never extended. I extended it and resized. Wiki is back up and running.

#>lvextend -L +17.99G /dev/turnkey/root

Rounding up size to full physical extent 17.99 GiB
Extending logical volume root to 40.01 GiB
Logical volume root successfully resizedresize2fs /dev/turnkey/root
resize2fs 1.41.11 (14-Mar-2010)
Filesystem at /dev/turnkey/root is mounted on /; on-line resizing required
old desc_blocks = 2, new_desc_blocks = 3
Performing an on-line resize of /dev/turnkey/root to 10487808 (4k) blocks.
The filesystem on /dev/turnkey/root is now 10487808 blocks long.
root@aserver ~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/turnkey-root
40G 21G 17G 56% /
none 120M 184K 120M 1% /dev
none 123M 0 123M 0% /dev/shm
none 123M 56K 123M 1% /var/run
none 123M 0 123M 0% /var/lock
none 123M 0 123M 0% /lib/init/rw
/dev/sda1 473M 437M 12M 98% /boot

Excellent blog post Liraz

Jeremy Davis - Tue, 2011/02/22 - 00:30

I don't have immediate need for this info and have only just given it a quick scan (rather than a proper read) but appreciate it never the less. I will get back to this and have a good read as LVM is a bit of a mystery to me (I get the basic idea but it still seems a bit esoteric). I'm sure that many others will really appreciate it too.

Nice clarification

flexbean - Tue, 2011/02/22 - 02:22

Thank you for the clarification. Much clearer now. Any potential of there being a webmin mod for this?

There already is...

Alon Swartz - Tue, 2011/02/22 - 15:30

There already is, and it's pre-installed in all TurnKey appliances v11+.

IIRC, log into webmin -> Hardware ->LVM

Fantastic!

Gizmo - Tue, 2011/02/22 - 11:01

Thanks so much for this Liraz! It worked a treat! Very easy to figure out due to your very simple and clear instructions =D

Lets elaborate on the IIRC Webmin Mod...

Landis Arnold - Wed, 2011/02/23 - 02:22

Last week I extended my Physical Volume in VMWare but ran into the block of not being able to expand the LVM. I missed the "middle part" about Volume Groups. I will study and Test. The Webmin module sounds good (for newbs like me).

Expanding the physical volume itself was pretty easy in VMware (use the Thin format), but unlike in windows, VMWare does not have an easy way to work with Linux Partitions after that - at least that I could find. The methodologies above will be more than helpful.

Partitioning in linux

Guest - Sat, 2011/03/05 - 18:41

oh, Though i am a newbie for linux. I am using Ubuntu from last two years but in GUI mode. Abobve partitioning theory ,i found it more complex.can you help me how to partition in linux Ubuntu/Redhat using command line. Thanking you

Are you using TKL

Jeremy Davis - Sat, 2011/03/05 - 23:46

TKL uses LVM by default, so this is very relevant for TKL users. If you are looking for more general Ubuntu partitioning info then you would probably want to have a look at this and this.

reboot not necessary !

Guest - Wed, 2011/03/16 - 20:11

Do you not have partprobe on your system?

Read man page.

# example #2: you've expanded the existing sda hard disk
cfdisk /dev/sda  # creating /dev/sda3
partprobe
pvcreate /dev/sda3
vgextend turnkey /dev/sda3

You need parted istalled,

Eric Thibodeau - Wed, 2014/04/23 - 05:20

You need parted istalled, which, for some reason, is not included in the core turnkey setup.

Just what I needed...

Jason Lehman - Wed, 2011/05/04 - 21:23

This is great, thanks!

Loving TKL!

Jason

Just tested this myself and it worked like a charm!

Jeremy Davis - Sat, 2011/10/01 - 04:36

Yay! :)

Whose idea was it to auto-subject? <.< >.>

Clay Fig - Sun, 2011/10/02 - 14:45

I'm using tkFileServer and I used example #2.

I'm still pretty newb-ish with Linux, but, this helped me gain a better understanding of how LVM works.

I have 3 Windows 7 computers to back up: one gaming rig, one DAW, and a VMware server host; and 18~ GB is hardly enough space for one initial backup, much less three.

Thanks Liraz!

Transferring lvm volume to sda

Roberto - Mon, 2011/10/10 - 01:52

Is there any way to transfer an lvm volume (ext4-formatted, i.e.) to a physical drive partition?

AFAIK the same way you'd transfer any partition to another...

Jeremy Davis - Mon, 2011/10/10 - 05:39

So any sort of copying mechanism eg dd, rsync, etc. But probably easiest of all would be to do a TKLBAM backup of your appliance and do a clean install as you desire (no LVM if that's what you want) and restore the data in. Personally I'd test your backup before you destroy the current machine though, just in case something isn't quite right.

Out of interest why would you want to do that anyway? LVM takes a bit of getting used to but IMO it's way cool!

No luck with "dd", no need for "rsync", no need for TKLBAM, :-(

Roberto - Mon, 2011/10/10 - 10:22

Thank you, Jeremy.

I was trying "dd" with not very much luck (you know, 20G could be very much data to do a dirty test). I don't have a TKLBAM account.

TK is a great appliance, but I found some compatibility issues with Eucalyptus. First (but not only) is the partitioning of the disk. Eucalyptus is expecting /dev/sda1 while TKL is giving /dev/mapper/turnkey-root as the root filesystem. In general, KVM support for TKL would be a great feature, I think.

Thank your very much again for your time.

Regards,

Roberto.

I have no experience with Eucalyptus

Jeremy Davis - Mon, 2011/10/10 - 14:10

But I have heard some good things about it. I personally use ProxmoxVE and find it really good. TKL installs to KVM no worries (from ISO) and I have also converted lots of images to run under OVZ (PVE has both). Theoretically the TKL VM images should be able to be imported into KVM but I have only played with it once and dind't have any joy (I didn't try real hard because it's pretty quick and easy to install from ISO). But I agree a KVM image that could be easily imprted would be a plus.

dd is pretty slow but I have successfully dd'd up to 250GB images over a network (SHH) successfully on a number of occasions. That's how I migrated my physical Win2K3 server to a KVM VM. I just used a live CD and dd'd the HDD to a .raw image and KVM picked that up fine. But as I say that was with PVE not Eucalyptus so obviously YMMV.

We're going to publish images

Liraz Siri - Tue, 2011/10/11 - 01:08

We're going to publish images optimized for Eucalyptus soon. It would be great to get your feedback on that when it comes out. We'll post an update about it on the blog.

Eucalyptus issues and full support to KVM

Roberto - Mon, 2011/10/10 - 14:55

Well, I'm not very experienced with Eucalyptus or KVM, but I've seen two many issues related to the "Community" version of Eucalyptus not seen in "Enterprise", too bad.

I'm very tempted to drop Eucalyptus in favor of any other virtualization platform (OpenStack, i.e.) or oVirt, when fully available. I don't use OpenVZ, so I see a great issue in using ISO instead of full installed templates like TKL, but it seems to be the only chance by now.

When I talk about KVM support, I am thinking of virtio drivers preinstalled and an XML file, what would be absolutelly amazing to spread TKL for LIBVIRT/KVM users.

ProxmoxVE seems impressive and very straightforward! It's great, but I'm thinking of an EC2 compatible API without loosing optimizations in every VM format. Eucalyptus is not the right option, since it seems centered in GNU/Linux images under Xen, no way of having Windows if you don't have Enterprise Edition.

Thanks.

Must admit that I've never installed drivers in TKL

Jeremy Davis - Mon, 2011/10/10 - 15:24

I run TKL vanilla under KVM (on PVE) and my (anecdotal) experience has been that it seems close to bare metal performance - OVZ performance is even better (although perhaps with VirtIO drivers KVM would be equivalent?) Win is another story though! Without VirtIO drivers disk I/O can be noticably sluggish, especially under load. I haven't noticed much difference with the networking drivers though.

I understand the desire for EC2 compatability. I think OpenStack is probably the future in that regard but i guess time will tell. OTOH via the Hub using a TKLBAM backup I can migrate from KVM/OVZ/VirtualBox/bare metal to an AWS instance very easily (the Hub supports auto restore of a backup to EC2).

Anyway, good luck with your search and let us know if you find something good! :)

http://xenonhid4u.co.uk/

http://xenonhid4u.co.uk/ - Sat, 2011/10/15 - 14:37

Thanks for this great and interesting article. I really enjoyed the article. It's really useful and informative for me.

LVM

Pravin - Thu, 2011/11/17 - 16:40

Thanks lol...

Below mentioned link is very easy to understand,

http://www.redhatlinux.info/2010/11/lvm-logical-volume-manager.html

Worked like a charm!

Eric - Mon, 2012/01/30 - 17:34

Thanks for the small bites of info for a first time LVM-er.

Thank you so much!

Andy Turfer - Mon, 2012/02/13 - 16:01

Only one suggestion: put a 'donate' link on this page so that I can buy you a beer. Thank you, worked perfectly!!

LVM (LOGICAL VOLUME MANAGER)

Digitalinux - Thu, 2012/03/22 - 16:07

Looks good

Thanks for sharing....

You can get much more information from the below url,

http://www.digitalinux.com/2010/11/lvm-logical-volume-manager.html

It worked for me. Look like a

chuongnv - Wed, 2012/03/28 - 16:36

It worked for me. Look like a miracle! Thank so much.

pvcreate on unpartitioned hard disk

Juvenal - Sun, 2012/04/01 - 04:48

Hi! Thank you for the post! One question: Can I use pvcreate /dev/sdX on a unpartitioned hard disk or do I need to have some partition on it before running the command?

Thanks again!

You can use pvcreate on an unpartitioned disk

Jeremiah - Sun, 2012/04/01 - 06:50

You can use pvcreate on an unpartitioned disk without any problem. In fact, that is what example #1 shows in the post. "pvcreate /dev/sdb"

Out of inodes

Alexandre Takacs - Mon, 2012/04/02 - 22:20

My problem is a little bit different - I am out of iNodes on my turnkey LVM...

root@core /tmp# df -hi
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/mapper/turnkey-root
                        416K    416K     126  100% /
tmpfs                    63K       4     63K    1% /lib/init/rw
udev                     62K     514     62K    1% /dev
tmpfs                    63K       1     63K    1% /dev/shm
/dev/sda1               122K     207    122K    1% /boot
/dev/sda6               755K      18    755K    1% /storage

What is my best cours of action ?

Any pointer appreciated

I just had a quick google

Jeremy Davis - Tue, 2012/04/03 - 01:03

And from what I gather, it seems that the best course of action is to reformat and specify the number of inodes.

Have a read of these links for starters (although there may be betters one, they're just what came up via google):

http://serverfault.com/questions/111857/linux-help-im-running-out-of-inodes
http://www.linuxquestions.org/questions/linux-general-1/inode-lvm-and-in...
http://www.linuxquestions.org/questions/linux-newbie-8/running-out-of-in...

...fixed?

spaceyjase - Tue, 2012/05/15 - 16:40

...been doing some research and I think a it just requires navigating through the grub rescue console as the LVM binaries are in the initrd image :) Haven't tested but a variation of the below:

insmod ext2
linux /vmlinuz-2.6.32-41-generic-pae
initrd /inird.img-2.6.32-41-generic-pae
/sbin/vgchange -a y
fsck -y /dev/turnkey/root

And so on. Fingers crossed, I'll try that next time!

- spaceyjase

Thanks

Sibin - Tue, 2012/07/03 - 10:23

It was very easy to follow your steps , Thanks a Lot

thanks !

mojzis - Fri, 2012/09/28 - 23:45

this (and the fact you decided for LVM) literally saved my life when i created a VirtualBox fixed drive and it showed too small later ...

i tried adding the -r option to lvextend. it came with some interesting warning about damaging my system, so i said no, but resize2fs was ran for me anyway :)

Many thanks!!!

konig - Wed, 2012/10/24 - 18:53

Great step-by-step tutorial.

Many thanks.

regarding lvextend

raja - Fri, 2012/12/14 - 00:03

lvextend -l +2047 /dev/vgone/lv_usetest

What does +2047 stand for i know it is for 8gb how do we calculate.

lvm + encryption

user - Fri, 2012/12/21 - 15:28

Hello,

Is there any way to support lvm+encryption for truecrypt installation? This could be particularly usefull when administrator of different vms have access to the host (the hypervisor) in order to backup, restart (whatever) their own virtual appliance. If each vm is encrypted, an admin of a virtual will not be able to read and understand the content of another.

Is definitely possible

Jeremy Davis - Tue, 2013/01/22 - 03:23

But it is not supportted OOTB so you'd need to configure this manually on each guest (although perhaps you could script it to somewhat automate the process).

My suggestion would be to have a look at ProxmoxVE. It allows the configuring of users and groups which each can have certain levels of control/access on a per vm, or per host/node basis. It also allows TKL (and other Linux distros) to run under OVZ (which gives you 97-99% performance of bare metal) or KVM (which supports Windows and other OS).

Great post!

Maalini - Wed, 2013/02/06 - 19:17

Great post. Worked like a charm for me. I used to increase lvm partition on a Red Hat Virtualization VM.

Thank you.

Great Tutorial

Willie - Tue, 2013/02/12 - 07:41

This made it a lot easier on me to extend my storage.

Superb post

SteVader - Wed, 2013/04/10 - 15:31

This just saved my bacon somewhat - thanks for an excellent post.

Hero

Yeah - Thu, 2013/04/11 - 16:40

Thank you so much this was exactly the problem i had. solved!

Can't expand LVM to make a filesystem

Martin Jochimsen - Thu, 2013/04/18 - 01:26

I might have misunderstood something here, but I will try and ask my question anyway, as it seems to be a great place for knowlegde about LVM.

Long story short:

I messed up an upgrade of my ReadyNAS Duo from netgear. It runs on linux and stupid me made a normal apt-get upgrade as I often do on my Ubuntu.

After the upgrade I couldn't get in contact with my serve/disk (with all my pics, music and movies). I took the disk and put it into an usb-dockingstation to see what was wrong.

The disk (1TB) should at least have 250 GB free but I get something completly different when I test it with different commands

root@martin-ubudesktop:/home/martin# lvdisplay /dev/c/c

--- Logical volume ---

LV Path /dev/c/c

LV Name c

VG Name c

LV UUID 0h6FoC-k5xk-Ziax-d06Z-V8lW-AcOK-A0haBA

LV Write Access read/write

LV Creation host, time ,

LV Status available

# open 0

LV Size 929,09 GiB

Current LE 29731

Segments 1

Allocation inherit

Read ahead sectors auto

- currently set to 256

Block device 252:0

root@martin-ubudesktop:/home/martin# vgdisplay c

--- Volume group ---

VG Name c

System ID

Format lvm2

Metadata Areas 1

Metadata Sequence No 2

VG Access read/write

VG Status resizable

MAX LV 0

Cur LV 1

Open LV 0

Max PV 0

Cur PV 1

Act PV 1

VG Size 929,09 GiB

PE Size 32,00 MiB

Total PE 29731

Alloc PE / Size 29731 / 929,09 GiB

Free PE / Size 0 / 0

VG UUID 8mo4RP-AHcs-Hr3r-yQ1Z-uWYw-WaUo-dEEYkJ

I think the LV and LG names came form the ReadyNAS.

As I have understood this guide, I need some space to make some kind of filesystem (so I can access my files?)

If my VG doesn't have enough free space, can I then (to extend VL) use my desktop's harddisk to create a PV and then add it to the VG? (example #1???)

I know am on deep water here and I'm soon to go down for the third time.....and my wife doesn't know about it yet!

Best regards and thanks for any kind of help

Martin :-)

P.S.

Martin Jochimsen - Thu, 2013/04/18 - 01:31

Can I by the way use gparted to shrink the LVM partion?

As I said there should at least still be 250 GB free on the disk.

Possible solution

SteVader - Thu, 2013/04/18 - 01:39

You might want to take a look at this:

http://www.readynas.com/forum/viewtopic.php?f=11&t=35153

I've recovered 100% of files from a couple of borked readynas systems using that methodology.

Hope it helps!

The link you gave me looked

Martin Jochimsen - Thu, 2013/04/18 - 13:01

The link you gave me looked really promising, and while I was downloading the Ubuntu image I scrolled through the thread finding this post (page 9 third from the bottom)

Re: ReadyNAS Data Recovery - VMware recovery tool

by doyley86 » Fri Dec 28, 2012 1:19 pm

Guys I have been struggling with this for months, I have found a simple easier way of restoring all my files using some free software called DiskInternals Linux Reader you can download it free from here:http://www.diskinternals.com/linux-reader/

it finds your drive straight away and allows you to copy off all your data! Hope this helps.

Cheers

It's just a small windows program but it reads my LVM without any problems, and now I'm backing everything up nice and easy.

I will save the VMware trick for another catastrophe - for I'm pretty sure there will be more like this one to come.

Oh, and by the way. My wife has forgiven me now!!!

Martin :-)

Excuse me if I missed something...

Jeremy Davis - Thu, 2013/04/18 - 14:40

But why not just mount your LV and then copy out the files?

Something like this should work:

mount /dev/c/c /mnt

Then all your files should be somewhere in /mnt (I have no idea what subfolder - it depends on where ReadyNAS stores stuff).

You didn't miss anything

Martin Jochimsen - Thu, 2013/04/18 - 15:36

It's me who might not have been given all relavant info about my drive.

I have tried to mount the drive in any possible way.

Often I got a mesage about bad superblocks and I would try and restore the superblock with the backup superblocks, but that never worked.

Sometimes I got other messages (something about zero-lenght partitions, invalid filesystem and so on) and I could nevet get past those.

Maybe if I have had somebody who would know a lot more about this than me (and that shouldn't be hard to find), but I really tried A LOT of things.

Then I came to this page and the rest is history...

Martin :-)

Ahh ok

Jeremy Davis - Thu, 2013/04/18 - 16:04

That's a whole other issue then. My advice to start with would be to take an image of the drive before doing anything. Ideally I like to image to a same or larger size drive. dd is the go, or ddrescue if there are hardware or data corruption issues.

Otherwise I'm not much help. I had a HDD fail on me some time ago and I managed to recover some of the data (about a quarter) by writing a new super block, but I don't recall how I did it... Sorry...

Glad it got you on the right

SteVader - Thu, 2013/04/18 - 13:03

Glad it got you on the right track!

Nice article and lvextend options

Ben @ Geekswing - Mon, 2013/06/03 - 04:00

Nice article -

Two helpful things I want to add which I found out during my lvm "studies" as well

The PE's are really nice (from vgdisplay command) and give you some very fine tuned control if you want it. For example if you want to make sure to use all of the free space available, specifying a number such as "1G" or "10G" may not do it.

From one of the vgdisplay command you have

VG Size               18.14 GiB
  PE Size               4.00 MiB
  Total PE              4645
  Alloc PE / Size       4480 / 17.50 GiB
  Free  PE / Size       165 / 660.00 MiB

The PE size is 4mb, and there are 165 free PEs (660mb)

For lvextend you can use

lvextend -l +165 [lvname]

With no units by default it's the # PE

Another helpful shortcut to use all space available

lvextend -l +100%FREE

Thanks!

p.s.http://geekswing.com/geek/unix/extending-a-linux-disk-with-lvm-extending-root-partition/

Each disk for each user

Brian - Wed, 2013/07/10 - 11:05

How i CAN USe each partition for each user turnkey file server ? or better say each disk for example 1 tera byte for each user? thanks

I have no idea

Jeremy Davis - Sun, 2013/07/14 - 16:40

But I'm sure if you search on google you'll find plenty of info. Keep in mind that TKL v12.x is based on Debian Squeeze and I'm sure you'll find something. Hint: I think you are after user quotas using LVM

Each disk for each user

Brian - Mon, 2013/07/15 - 22:33

Thanks Jeremy, I will search on google about that information. My plan is use around 8 disk (usb disk) of 1 tera, and then use that 8 disk connect in RAID mode, mode mirror (2 disk mirrored=1 tera for 4 users), and I think turnkey is very powerfull tool for that aplication.

muy question is, you can do this with turnkey? (http://upload.wikimedia.org/wikipedia/commons/c/ce/RAID_100.svg), and then make quotas using LVM?

thanks a lot

Brian

AFAIK theoretically

Jeremy Davis - Tue, 2013/07/16 - 18:10

But I've never done it... I think mdam is the one to use for software RAID.

The only thing is that I'm not sure if LVM would really be relevant then... AFAIK the way that quotas work is that they use separate partitions...

LVM usage

Carl Maier - Thu, 2013/08/08 - 00:51

I tried using the webmin lvm tool, but then I read your post. Your explanation was the best I've seen yet. I followed and everything worked like a charm. I did have to reboot as you said might happen. Also note, 1. issuing resize2fs may take several minutes 2. for me webmin disk usage side bar shows the original free space now consumed - odd but no big deal.

Thanks for the post.

Thank You!!!

SamWN - Thu, 2014/02/13 - 04:48

Thank you so much, man. I've been banging my head against the wall trying to figure this our and this resolved it perfectly within 5 minutes.

LVM IT AWESOME REAL

ezekia - Tue, 2014/03/11 - 15:06

This kind of the file system is very amaizing and from now on will be using this kind of file system into my system.....

Thank you

Jesse - Tue, 2014/04/08 - 00:21

This was exactly what I needed. Thank you for the detailed post!

What I needed after 2 1/2 days of frustration

Clint Brown - Thu, 2014/04/10 - 20:26

I wasn't trying to do anything with Turnkey but rather was trying to extend a VirtualBox partition. The information on LVM was just what was needed for this Linux newbie. Thank you!

Thank you

Marcello - Tue, 2014/05/27 - 17:27

Thank you for this post!

Thank You a million times!!

Aby Sam Ross - Tue, 2014/09/16 - 17:31

I had kept aside a good chunk of my hard disk for installing other linux distros. But then was in immediate need of some space to my current home lv. I was worried that I might screw up (As I have the habit of doing so :P) trying to expand my vg. Plus I am new to lvm. I found your article bookmarked it and kept it aside for sometime till now when I really had to do something about the space requirement. But when I went through your article the flow became clear and even cleared many of the doubts I had regarding lvm.

Thanks for writting things so clearly and making my life easier. :)

Thank you!!!

Terence Oort - Fri, 2015/03/06 - 15:14

Been scratching my head with this one.

So read up on LVM. executed the steps and eureka!

Expanded my Turnkey Vmware Owncloud from 30gb to 100gb

Regards

Terence

Can't grow storage with gparted, cfdisk, pvcreate,...

Eli Taylor - Tue, 2015/07/21 - 02:32

I need to grow my storage space. I've usually started with something like VBoxManage.exe modifyhd linuxturnkeylamp.vdi --resize 30000, followed by lots of steps from various tutorials. I've tried resizing things with GParted, and have screenshot below after too many changes. I've tried cfdisk, but none of these instructions include which settings/options to select and I get errors surrounding partioning tables not setup or invalid settings. I've tried pvcreate but get a different error or overwrite warning per sda#. I've tried creating a new VHD using VirtualBox but still none of them show up afterwards for extending, using pvdisplay --maps or vgdisplay. You might realize from these tries I'm pretty notice in this area, but am starting to believe it's not just me. Anyone have some ideas or better directions on these commands? If can elaborate of any errors but not even sure which direction to go at this point.

Linux linuxturnkeylamp 3.2.0-4-amd64 #1 SMP Debian 3.2.68-1+deb7u2 x86_64
Welcome to Linuxturnkeylamp, TurnKey Linux 13.0 / Debian 7.2 Wheezy
System load: 0.08 Memory usage: 4%
Processes: 83 Swap usage: 0%
Usage of /: 99.8% of 5.89GB IP address for eth0: 10.0.2.15
IP address for eth1: 192.168.100.200
IP address for eth2: 192.168.56.100
TKLBAM (Backup and Migration): NOT INITIALIZED

root@linuxturnkeylamp ~# pvdisplay --maps
--- Physical volume ---
PV Name /dev/sda5
VG Name turnkey
PV Size 45.25 GiB / not usable 2.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 11583
Free PE 9797
Allocated PE 1786
PV UUID g0UVfV-Z30e-8hln-HUtz-F3kw-YXMt-Rm2Hre

--- Physical Segments ---
Physical extent 0 to 1530: Logical volume /dev/turnkey/root
Logical extents 0 to 1530
Physical extent 1531 to 1785: Logical volume /dev/turnkey/swap_1
Logical extents 0 to 254
Physical extent 1786 to 11582:    FREE

root@linuxturnkeylamp ~# vgdisplay

--- Volume group ---
VG Name turnkey
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 5
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 45.25 GiB
PE Size 4.00 MiB
Total PE 11583
Alloc PE / Size 1786 / 6.98 GiB
Free PE / Size 9797 / 38.27 GiB
VG UUID cnjfUb-Zp0u-Ns5Z-xchm-9J07-MCvT-IclgSf

this is a screenshot of slightly earlier states of my debugging

awesome.......i understand

nandkishor - Fri, 2016/05/20 - 19:15

awesome.......i understand how to extend lvm

Expanded with second Hard Drive, now all images and links broken

Donald Strand - Mon, 2016/10/24 - 21:52

Thanks for your awesome walkthrough. When our VM filled up the Database service wouldn't even start and we were also super sad. We weren't able to expand the partition, but your walkthrough of adding a second hard drive did work and the Database made the connection sucessfully.

However now all the links we had setup in the WP aren't functioning AND the media catalog doesn't exist anymore. Anything you can suggest to fix those things?

That sounds weird

Jeremy Davis - Tue, 2016/10/25 - 02:01

Adding an additional drive and adding it to your LVM shouldn't change anything about the data itself.

Do you have a backup you can restore from?

I wish! I have a snapshot of

Donald Strand - Wed, 2016/10/26 - 02:07

I wish! I have a snapshot of the VM before I did the expansion that I can roll back to, but during that time the drive was so full that the Database wasn't able to start. Is there a trick to getting around that? I pruned as many logs as I could...

Sorry for slow response...

Jeremy Davis - Fri, 2016/12/09 - 07:15

I missed your post here. FWIW unless it's a really recent blog post, to ensure timely response, you are probably best to start a new thread in the forums.

Anyway, as a general rule, we highly recommend using TKLBAM for backups. WOrst case you could have then restored the data to a new VM (with a bigger root volume) off the bat. Obviously a snapshot would also work. And all of that is moot now anyway...

FWIW log files are usually only pretty small so unlikely cleaning up them will give you enough room. Probably the best way to clear a bit of space is removing the apt cache:

apt-get clean

Although obviously how much room that will give you will depend on how much stuff is there (and when/if you've ever run it before).

One really handy tool to see what is taking up all your space (and give you ideas for removal candidates) is ncdu. It's not that big, but obviously it's preferable to install it before lack of space becomes an issue... Install that like this:

apt-get update && apt-get install ncdu

Then to look at your whole system, launch it like this:

ncdu /

It will show you all the files and directories in / ordered by size.

As a general rule, keeping a regular eye on your free space before it becomes a problem is highly recommended. Again probably not much use to you this time, but hopefully might help save headaches next time!? :)

VMWare KB

cdee - Fri, 2017/01/13 - 04:24

I followed these instructions and increased the root volume from 20GB to 30GB:

[I was going to paste a link here to the VMware knowledgebase, but the spam filter would not let me]. Sad.

finally got working

James - Sun, 2017/02/05 - 05:08

checking stack overflow/aks ubuntu and all they kept talking about was lvextend, this article made sense and now I understand and have a few more commands under my belt awesome thanks

vgextend can't go over 2TB?

/R - Sun, 2017/11/19 - 19:43

I'm trying to follow these instructions to extend the LVM volume by 4TB, which I already provisioned to the VM on the hypervisor level. I used gdisk to create the 4TB physical volume (/dev/sda2/ in my case) but when I add it via vgextend, I can only see 2TB of it:

root@fileserver ~# fdisk -l

Disk /dev/sda: 4 TiB, 4398046511104 bytes, 8589934592 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: A421848E-B5F0-4381-9C68-BDE298FEC445

Device        Start        End    Sectors  Size Type
/dev/sda1    123046   39062500   38939455 18.6G Linux LVM
/dev/sda2  39062504 8589934558 8550872055    4T Linux filesystem

Disk /dev/mapper/turnkey-root: 17 GiB, 18253611008 bytes, 35651584 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/turnkey-swap_1: 512 MiB, 536870912 bytes, 1048576 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
root@fileserver ~# vgextend turnkey /dev/sda2
  Physical volume "/dev/sda2" successfully created
  Volume group "turnkey" successfully extended
root@fileserver ~# vgdisplay
  --- Volume group ---
  VG Name               turnkey
  System ID
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  9
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               2.00 TiB
  PE Size               4.00 MiB
  Total PE              524272
  Alloc PE / Size       4480 / 17.50 GiB
  Free  PE / Size       519792 / 1.98 TiB
  VG UUID               IVd0Is-sPQ7-f62x-ISz5-Ol5N-eUUn-eQ2HEB

Note the "VG Size 2.00 TiB". Is there a way around this 2TB limitation? Should the limitation be present at all, when using GPT or am I missing something obvious here? Perhaps mirroring how gdisk or parted replaces fdisk when modern partition sizes are needed - there's a newer utility replacing "vgextend" that makes proper use of large GPT volumes?

What version of TurnKey are you using? Is this a VM?

Jeremy Davis - Mon, 2017/11/20 - 00:42

I'm not sure, but perhaps that's a limitation on an older version of TurnKey? I.e. if you are using v13.x or earlier, then perhaps it's a known bug/limitation? Although perhaps it still effects the current builds too? I can't be 100% sure, but AFAIK it should "just work" on v14.x, so long as the disk has a GPT partition table (2TB is the maximum volume size supported by the old school MBR).

It sounds like you are doing all the right things. A quick google, bought up a similar issue to your on SE: How to create a LVM Partition/physical volume > 2TB.

If this is a VM, then I suggest that you follow the recommendations of the top answer and add a new vHDD with the size that you want. Then, as per the answer, don't partition it at all, just add the whole disk as a PV as is.

If this is a physical hard drive, then rather than adding a new partition, why not try just increasing the size of the existing partition (rather than adding a new one)?

Either way, I urge you to make sure that you have a current working (i.e. tested) backup of your important data before you start. Worst case scenario, you can roll back to where you are now.

Thanks for the prompt reply!

/R - Mon, 2017/11/20 - 15:01

This is TurnKey FileServer version 14.1 running as a VM on ESXi 6.5. The virual drive is a single 4TB, thin-provisioned vdmk drive sitting on a 12x3TB NL-SAS RAID-10 array. I do have a stable VM snapshot I can always fall-back to, with the default 20GB turnkey volume. I've also noticed that in my paste the 4TB partition was Linux filesystem, instead of Linux LVM. I repeated the same steps, this time using LVM and got the same, 2TB result from pvcreate./n I'll try to follow you suggestion and shrink the 4TB vdmk back to the default 20GB - and then add a new, 4TB vdmk and see if I can create a single 4TB physical volume out of that, once it's formatted as a GPT partition. I'll report the results here once I have them. So far I suspect some limitation of the pvcreate tool itself and wonder if there's any alternative utilities that do the same job. All the examples of pvcreate I've found so far invovle small volumes and I've yet to see an example of it working for >2TB GPT partitions.

Hmm ok...

Jeremy Davis - Mon, 2017/11/20 - 22:46

When you add the new disk, don't partition it at all. You don't even want a partition table. I know that sounds weird, but that's what you'll need to do.

I've just had a closer look at my home server because I was sure that I had a VG larger than 2TB. It's not TurnKey, but it is Debian Jessie based (same as TurnKey) so I was hoping it should be the same. Unfortunately, it seems that the version of LVM I have is a little newer, so YMMV.

root@tkl-14-2 ~# lvm version
  LVM version:     2.02.111(2) (2014-09-01)
  Library version: 1.02.90 (2014-09-01)
  Driver version:  4.27.0

root@pve ~# lvm version
  LVM version:     2.02.116(2) (2015-01-30)
  Library version: 1.02.93 (2015-01-30)
  Driver version:  4.34.0

Unfortunately, the different lvm version means I can't be sure this will work for you (but hopefully it will). It turns out that I do have a VG over 2TB, but only just. FWIW, here's how it looks:

root@pve ~# vgdisplay pve3
  --- Volume group ---
  VG Name               pve3
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               2.73 TiB
  PE Size               4.00 MiB
  Total PE              715396
  Alloc PE / Size       707789 / 2.70 TiB
  Free  PE / Size       7607 / 29.71 GiB
  VG UUID               eMT1kU-T5ZC-9HzC-QgpG-5Mmb-tqOj-ZK7ESO

And it's just a single disk (3TB on the cover; ~2.7TB in the real world):

root@pve ~# pvdisplay /dev/sdb
  --- Physical volume ---
  PV Name               /dev/sdb
  VG Name               pve3
  PV Size               2.73 TiB / not usable 4.46 MiB
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              715396
  Free PE               7607
  Allocated PE          707789
  PV UUID               hAv4dj-yTXL-Cikh-lgpr-EcUs-DDbb-ZaU20r

And it's not partitioned at all:

root@pve ~# fdisk -l /dev/sdb

Disk /dev/sdb: 2.7 TiB, 3000592982016 bytes, 5860533168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

So give that a go, hopefully that will work for you.

If that still doesn't work, perhaps one more thing worth trying is to create individual partitions smaller than 2TB and add them all to the same VG. TBH, I'm not sure it will work, but hopefully...

Works fine with a single drive

/R - Sat, 2017/12/02 - 15:37

I can confirm that presenting just one disk to the system works as expected:

root@fileserver ~# fdisk -l

Disk /dev/sda: 4 TiB, 4398046511104 bytes, 8589934592 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 652E58D0-8FC3-40D4-A759-ED5B1EAD2415

Device     Start        End    Sectors Size Type
/dev/sda1   2048       4095       2048   1M BIOS boot
/dev/sda2   4096 8589932543 8589928448   4T Linux LVM

Disk /dev/mapper/turnkey-root: 4 TiB, 4393751543808 bytes, 8581545984 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/turnkey-swap_1: 4 GiB, 4290772992 bytes, 8380416 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Probably the easiest way to do it is to use the TurnKey Fileserver appliance in .iso format - rather than the .ova and sort out the storage right in the installer.

I don't have LVM partition and the volume group

chaosbanane - Thu, 2017/12/28 - 16:51

Strange. I have no idea how this happened but I don't seem to have the LVM setup with the turnkey volume group. I added 20 GB to my partition using

cfdisk /dev/sda
pvcreate /dev/sda3

--> result is /dev/sda3

root@core ~# fdisk -l

Disk /dev/sda: 30 GiB, 32212254720 bytes, 62914560 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x99481a0f

Device     Boot    Start      End  Sectors  Size Id Type
/dev/sda1  *        2048 18874367 18872320    9G 83 Linux
/dev/sda2       18876414 20969471  2093058 1022M  5 Extended
/dev/sda3       20969472 62914559 41945088   20G 83 Linux
/dev/sda5       18876416 20969471  2093056 1022M 82 Linux swap / Solaris

Now i have the problem that I don't have a volume group:

root@core ~# vgdisplay
No volume groups found

Can anyone help me with the reason and a hint how I can solve it otherwise? Thank you!

Doesn't look like you are using LVM

Jeremy Davis - Mon, 2018/02/05 - 01:52

Apologies on such a slow response.

If you installed TurnKey from ISO, then LVM is optional. IIRC the default option in v14.0 was no LVM. I don't recall if it was v14.1 or v14.2 when we changed it to be the default.

Thank you

chaosbanane - Mon, 2018/02/05 - 13:02

Thank you for you anwer!

Search form

You are here

Extending an LVM volume: Physical volumes (partitions) -> Volume groups -> Logical volume -> Filesystem

How LVM works

Extending a Logical Volume

Comments

Re: ReadyNAS Data Recovery - VMware recovery tool

Pages

Add new comment

Anonymous HTML