RAW raw是默认的格式,格式简单,容易转换为其他的格式。需要文件系统的支持才能支持sparse file 创建image # qemu-img create -f raw flat.img 10GFormatting 'flat.img', fmt=raw size=10737418240 如果我们ls则看到 ls -lh flat.img -rw-r--r-- 1 root root 10G Jun 30 22:27 flat.img 但是并不真正占用10G # du -h flat.img 0 flat.img # qemu-img info flat.img image: flat.imgfile format: rawvirtual size: 10G (10737418240 bytes)disk size: 0 我们可以用dd来制作sparse file 下面的命令制作的是非sparse的文件 # dd if=/dev/zero of=flat1.img bs=1024k count=10001000+0 records in1000+0 records out1048576000 bytes (1.0 GB) copied, 0.66135 s, 1.6 GB/s block size是1024k,共1000个block # qemu-img info flat1.img image: flat1.imgfile format: rawvirtual size: 1.0G (1048576000 bytes)disk size: 1.0G 下面的命令制作的是sparse的文件 # dd if=/dev/zero of=flat2.img bs=1024k count=0 seek=20480+0 records in0+0 records out0 bytes (0 B) copied, 0.000141061 s, 0.0 kB/s # qemu-img info flat2.img image: flat2.imgfile format: rawvirtual size: 2.0G (2147483648 bytes)disk size: 0 seek的意思是将文件的结尾设在那个地方。 如何copy一个sparse文件呢 # dd if=/dev/zero of=flat3.img bs=1024k count=1 seek=2048 1+0 records in1+0 records out1048576 bytes (1.0 MB) copied, 0.0010721 s, 978 MB/s # qemu-img info flat3.img image: flat3.imgfile format: rawvirtual size: 2.0G (2148532224 bytes)disk size: 1.0M # cp --sparse=never flat3.img flat3-never.img 这样拷贝会占用整个2G空间 # qemu-img info flat3-never.img image: flat3-never.imgfile format: rawvirtual size: 2.0G (2148532224 bytes)disk size: 2.0G # cp --sparse=always flat3.img flat3-always.img 这样拷贝原来写入的1M的0也会被去掉 # qemu-img info flat3-always.img image: flat3-always.imgfile format: rawvirtual size: 2.0G (2148532224 bytes)disk size: 0 # cp --sparse=always flat3-never.img flat3-never-always.img 这样拷贝原来的2G都会被清理掉 # qemu-img info flat3-never-always.img image: flat3-never-always.imgfile format: rawvirtual size: 2.0G (2148532224 bytes)disk size: 0 qcow2 qcow2是动态的,相对于raw来说,有下列的好处: 即便文件系统不支持sparse file,文件大小也很小 Copy on write Snapshot 压缩 加密 qcow2的格式如下: 2-Level loopup qcow2的数据是存储在data clusters里面的,每个cluster是512 byte sector 为了能够管理这些cluster,qcow2保存了两层的Table,L1 table指向L2 Table,L2 Table管理data cluster. 所以在image里面的offset会被解析成三部分,L1 Table Pointer先找L1,L1 Table Pointer+ offset[0]是L1中的一个entry,读出来便是L2 Table Pointer, L2 Table Pointer + offset[1]是L2中的一个entry,读出来便是data cluster pointer, data cluster pointer +offset[3]便是数据所在的位置。 Reference Count 每一个data cluster都有RefCount,因为可能有多个snapshot都引用每一个数据块 Copy-on-write 这是backing file的用处,一个qcow2的image可以保存另一个disk image的改变,而不影响另一个image root:/var/lib/nova/instances/5d6cb926-5237-4b73-a7b4-e9af851023e7# qemu-img info diskimage: diskfile format: qcow2virtual size: 20G (21474836480 bytes)disk size: 450Mcluster_size: 65536backing file: /var/lib/nova/instances/_base/45a5886083c924b1fbe412b2ad92f6ce47381610Format specific information: compat: 1.1 lazy refcounts: false root:/var/lib/nova/instances/5d6cb926-5237-4b73-a7b4-e9af851023e7# qemu-img info /var/lib/nova/instances/_base/45a5886083c924b1fbe412b2ad92f6ce47381610image: /var/lib/nova/instances/_base/45a5886083c924b1fbe412b2ad92f6ce47381610file format: rawvirtual size: 5.0G (5368709120 bytes)disk size: 1.4G 上面的输出表示有一个raw的image是/var/lib/nova/instances/_base/45a5886083c924b1fbe412b2ad92f6ce47381610,我们基于他生成一个虚拟机,但是不想改变这个image的内容,所以我们基于它生成了qcow2的image :/var/lib/nova/instances/5d6cb926-5237-4b73-a7b4-e9af851023e7/disk。 一开始新的image是空的,读取的内容都从老的image里面读取。 当一个data cluster被写入,发生改变的时候,在新的image里面创建一个新的data cluster,这就是copy on write的意义。 Snapshot snapshot其实也是copy on write的一种应用,但是两者有微妙的不同。 广义来讲,有两种snapshot,一中时internal snapshot,一种是external snapshot internal snapshot是qcow2中的snapshot table所维护的snapshot,所有的snapshot都是在同一个文件中维护。 在monitor中,savevm, loadvm都是针对internal snapshot来操作的 我们创建一个虚拟机 # qemu-system-x86_64 -enable-kvm -name ubuntutest -m 2048 -hda ubuntutest.img -hdb ubuntutest1.img -boot menu=on -vnc :19 -net nic,model=virtio -net tap,ifname=tap0,script=no,downscript=no 当我们在monitor中执行savevm后,ubuntutest.img和ubuntutest1.img都会在image内部打一个snapshot $ qemu-img info ubuntutest.imgimage: ubuntutest.imgfile format: qcow2virtual size: 5.0G (5368709120 bytes)disk size: 2.1Gcluster_size: 65536Snapshot list:ID TAG VM SIZE DATE VM CLOCK1 snapshot1 390M 2014-06-28 04:04:42 01:00:59.2682 vm-20140630230902 239M 2014-06-30 23:09:02 00:04:07.783Format specific information: compat: 1.1 lazy refcounts: false $ qemu-img info ubuntutest1.img image: ubuntutest1.imgfile format: qcow2virtual size: 4.0G (4294967296 bytes)disk size: 134Mcluster_size: 65536Snapshot list:ID TAG VM SIZE DATE VM CLOCK2 vm-20140630230902 0 2014-06-30 23:09:02 00:04:07.783Format specific information: compat: 1.1 lazy refcounts: false 然而在monitor中info block的时候,还是指向的这两个文件。 其中的原理是,当打一个snapshot后,会在snapshot table中建立一项,但是起初是空的,包含L1 table的一个复制,当L2 table或者data cluster改变的时候,则会将原来的数据复制一份,由snapshot的L1 table来维护,而原来的data cluster已经改变,在原地。 external snapshot则往往采用上面的copy on write的方法,当打snapshot的时候,将当前的image不再改变,创建一个新的image,以原来的image作为backing file,然后虚拟机使用新的image。 在monitor中,snapshot_blkdev ide0-hd1 ubuntutest-snapshot.img qcow2,就是采用的这种方式,当这个命令执行完毕后,在monitor中info block 而且我们发现,在HOST机器上多了一个文件 $ qemu-img info ubuntutest-snapshot.img image: ubuntutest-snapshot.imgfile format: qcow2virtual size: 4.0G (4294967296 bytes)disk size: 964Kcluster_size: 65536backing file: ubuntutest1.imgbacking file format: qcow2Format specific information: compat: 1.1 lazy refcounts: false backing file可以是raw,也可以是qcow2,但是一旦打了snapshot,新的格式就是qcow2了。 两者很相似,稍微的不同是: 对于internal snapshot, 刚打完snapshot的时候,原image集合是不变的,snapshot的集合是空的,接下来的操作,写入在原image,将不变的加入snapshot集合 对于external snapshot,刚打完snapshot的时候,原image变成snapshot,snapshot集合是全集,新image是空的,接下来的操作,写入在新image,将改变的加入新image的集合。 it uses ‘qemu-img snapshot -c‘ if the domain is offline and –disk-only was not specified it uses qemu’s ‘savevm‘ if the domain is online and –disk-only was not specified it uses qemu’s ‘snapshot_blkdev‘ if the domain is online and –disk-only is specified virsh有命令支持snapshot 对已internal snapshot root:/etc/neutron# virsh snapshot-list instance-0000000aName Creation Time State------------------------------------------------------------ root:/etc/neutron# virsh snapshot-create instance-0000000aDomain snapshot 1404158715 created root:/etc/neutron# virsh snapshot-list instance-0000000a Name Creation Time State------------------------------------------------------------1404158715 2014-07-01 04:05:15 +0800 running root# ps aux | grep instance-0000000alibvirt+ 9057 0.0 1.2 6846900 821876 ? Sl Jun26 5:34 qemu-system-x86_64 -enable-kvm -name instance-0000000a -S -machine pc-i440fx-trusty,accel=kvm,usb=off -cpu SandyBridge,+erms,+smep,+fsgsbase,+pdpe1gb,+rdrand,+f16c,+osxsave,+dca,+pcid,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 2048 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 5d6cb926-5237-4b73-a7b4-e9af851023e7 -smbios type=1,manufacturer=OpenStack Foundation,product=OpenStack Nova,version=2014.1,serial=80590690-87d2-e311-b1b0-a0481cabdfb4,uuid=5d6cb926-5237-4b73-a7b4-e9af851023e7 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-0000000a.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/instances/5d6cb926-5237-4b73-a7b4-e9af851023e7/disk,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=31,id=hostnet0,vhost=on,vhostfd=33 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:fe:2a:41,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/5d6cb926-5237-4b73-a7b4-e9af851023e7/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 0.0.0.0:8 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5root 17709 0.0 0.0 10468 924 pts/1 S+ 04:06 0:00 grep --color=auto instance-0000000a root# qemu-img info /var/lib/nova/instances/5d6cb926-5237-4b73-a7b4-e9af851023e7/diskimage: /var/lib/nova/instances/5d6cb926-5237-4b73-a7b4-e9af851023e7/diskfile format: qcow2virtual size: 20G (21474836480 bytes)disk size: 1.1Gcluster_size: 65536backing file: /var/lib/nova/instances/_base/45a5886083c924b1fbe412b2ad92f6ce47381610Snapshot list:ID TAG VM SIZE DATE VM CLOCK1 1404158715 725M 2014-07-01 04:05:15 117:35:52.160Format specific information: compat: 1.1 lazy refcounts: false 对于external snapshot # virsh snapshot-create-as Instance01 snap1-Instance01 "Instance01 snapshot description" --diskspec vda,file=/home/cliu8/images/instance01-snapshot.img --disk-only --atomicerror: internal error: unable to execute QEMU command 'transaction': Could not open '/home/cliu8/images/instance01.img': Could not open '/home/cliu8/images/instance01.img': Permission denied: Permission denied 从网上查,是因为下面的问题: ------------------------------------------------------------------------------------- https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1004606 Bug Description When you attempt to create a virtual machine snapshot onto a separate disk, this fails with an error message. The error message is different depending on whether you use snapshot-create or snapshot-create-as but both approaches fail. The intention is to be able to take a backup of a live VM by freezing the running image file and applying snapshot deltas to a separate file so the backup can save the filesystem image without it being modified during the backup. (i.e. standard practice in VM world) Format being used is QCOW2 for the disk file which supports snapshots. Manually using qemu-img to create a snapshot on the file does work without any problems. If you use snapshot-create-as : snapshot-create-as winxppro3 snapname --disk-only --diskspec hda,snapshot=external,file=/srv/virtual-machines/tester/snap01.qcow2error: internal error unable to execute QEMU command 'blockdev-snapshot-sync': An undefined error has ocurred Or if you use snapshot-create and an XML file specification : virsh # snapshot-create winxppro3 snapspec.xmlerror: argument unsupported: unable to handle disk requests in snapshot snapspec.xml is :
Disable the AppArmor profiles for libvirt $ ln -s /etc/apparmor.d/usr.sbin.libvirtd /etc/apparmor.d/disable/$ ln -s /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper /etc/apparmor.d/disable/$ apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd$ apparmor_parser -R /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1096125
virsh create-snapshot fails
Bug Description
Hello,
i have a plain installation of Ubuntu 12.10 server,i installed package ubuntu-virt-server and added the user to the kvm and libvirtd group. (qemu-kvm 1.2 libvirt 0.9.13)After that i can run kvm virtual machines.
Now i am trying to get a snapshot of the disk (first step to achieve live backup of virtual machines)virsh snapshot-create-as [vmname] snapshot1 "snapshot" --disk-only --atomic
but i am getting the following error:error: internal error unable to execute QEMU command 'transaction': Could not open '/var/lib/libvirt/images/ubu1204.snapshot1'
(i even tryed with permission 777 )
Hi Serge,
i made a snapshot after disabling apparmor and it works!
Here are the commands used:
/etc/init.d/apparmor stopupdate-rc.d -f apparmor removereboot nowvirsh snapshot-create-as ubu-server-64-10.04.3 snap1 --disk-only
So i changed the target package of the bug to apparmor
Thank you again
affects: kvm (Ubuntu) → apparmor (Ubuntu)
----------------------------------------------------------------------------
我们可以在/etc/libvirt/qemu.conf将apparmor去掉
security_driver = "none"
重启service libvirt-bin restart
重启虚拟机
virsh destroy Instance02
virsh start Instance02
# virsh snapshot-create-as Instance02 snap1-Instance02 "Instance02 snapshot description" --diskspec vda,file=/home/cliu8/images/instance02-snapshot.img --disk-only --atomic Domain snapshot snap1-Instance02 created
# qemu-img info instance02-snapshot.imgimage: instance02-snapshot.imgfile format: qcow2virtual size: 5.0G (5368709120 bytes)disk size: 2.2Mcluster_size: 65536backing file: /home/cliu8/images/instance02.imgbacking file format: qcow2Format specific information: compat: 1.1 lazy refcounts: false
在创建一个
# virsh snapshot-create-as Instance02 snap2-Instance02 "Instance02 snapshot description" --diskspec vda,file=/home/cliu8/images/instance02-snapshot2.img --disk-only --atomic Domain snapshot snap2-Instance02 created
# virsh snapshot-list Instance02 --treesnap1-Instance02 | +- snap2-Instance02
root:/home/cliu8/images# qemu-img info instance02-snapshot.imgimage: instance02-snapshot.imgfile format: qcow2virtual size: 5.0G (5368709120 bytes)disk size: 2.2Mcluster_size: 65536backing file: /home/cliu8/images/instance02.imgbacking file format: qcow2Format specific information: compat: 1.1 lazy refcounts: falseroot:/home/cliu8/images# qemu-img info instance02-snapshot2.imgimage: instance02-snapshot2.imgfile format: qcow2virtual size: 5.0G (5368709120 bytes)disk size: 196Kcluster_size: 65536backing file: /home/cliu8/images/instance02-snapshot.imgbacking file format: qcow2Format specific information: compat: 1.1 lazy refcounts: false
看了我们需要单独的一节来阐述snapshot和AppArmor security deriver了
Convert, compress, encrypt, enlarge
image格式之间可以转换
raw可以转换为qcow2
root:/home/cliu8/images# qemu-img info flat1.imgimage: flat1.imgfile format: rawvirtual size: 1.0G (1048576000 bytes)disk size: 1.0Groot:/home/cliu8/images# qemu-img convert -f raw -O qcow2 flat1.img flat1.qcow2root:/home/cliu8/images# qemu-img info flat1.qcow2image: flat1.qcow2file format: qcow2virtual size: 1.0G (1048576000 bytes)disk size: 196Kcluster_size: 65536Format specific information: compat: 1.1 lazy refcounts: false
有时候qcow2也可以转换为qcow2,转换的过程中,没用的data cluster就被去掉了
root:/home/cliu8/images# qemu-img info ubuntutest.img image: ubuntutest.imgfile format: qcow2virtual size: 5.0G (5368709120 bytes)disk size: 2.1Gcluster_size: 65536Snapshot list:ID TAG VM SIZE DATE VM CLOCK1 snapshot1 390M 2014-06-28 04:04:42 01:00:59.2682 vm-20140630230902 239M 2014-06-30 23:09:02 00:04:07.783Format specific information: compat: 1.1 lazy refcounts: falseroot:/home/cliu8/images# qemu-img convert -f qcow2 -O qcow2 ubuntutest.img ubuntutest.qcow2 root:/home/cliu8/images# qemu-img info ubuntutest.qcow2image: ubuntutest.qcow2file format: qcow2virtual size: 5.0G (5368709120 bytes)disk size: 1.4Gcluster_size: 65536Format specific information: compat: 1.1 lazy refcounts: false
还可以压缩
root:/home/cliu8/images# qemu-img convert -c -f qcow2 -O qcow2 ubuntutest.img ubuntutest-compress.qcow2 root:/home/cliu8/images# qemu-img info ubuntutest-compress.qcow2image: ubuntutest-compress.qcow2file format: qcow2virtual size: 5.0G (5368709120 bytes)disk size: 465Mcluster_size: 65536Format specific information: compat: 1.1 lazy refcounts: false
还可以加密
root:/home/cliu8/images# qemu-img convert -o encryption -f qcow2 -O qcow2 ubuntutest.img ubuntutest-encrypt.qcow2 Disk image 'ubuntutest-encrypt.qcow2' is encrypted.password: root:/home/cliu8/images# qemu-img info ubuntutest-encrypt.qcow2image: ubuntutest-encrypt.qcow2file format: qcow2virtual size: 5.0G (5368709120 bytes)disk size: 1.4Gencrypted: yescluster_size: 65536Format specific information: compat: 1.1 lazy refcounts: false
我们运行一个虚拟机
# qemu-system-x86_64 -enable-kvm -name ubuntutest -m 2048 -hda ubuntutest-encrypt.qcow2 -boot menu=on -vnc :19 -net nic,model=virtio -net tap,ifname=tap0,script=no,downscript=n
发现一开始虚拟机并不启动
进入monitor输入用户名密码,方才启动
也可以在monitor中修改密码
block_passwd ide0-hd0 newpassword
image可以扩大,首先必须instance在shutdown的情况下,另外是必须raw format才能enlarge,最后扩大的空间既不会被partition,也不会被format
# qemu-img resize ubuntutest.img +10G qemu-img: Can't resize an image which has snapshotsqemu-img: This image does not support resize
root:/home/cliu8/images# qemu-img resize ubuntu-14.04.qcow2 +10GImage resized.root:/home/cliu8/images# qemu-img info ubuntu-14.04.qcow2image: ubuntu-14.04.qcow2file format: qcow2virtual size: 15G (16106127360 bytes)disk size: 461Mcluster_size: 65536Format specific information: compat: 1.1 lazy refcounts: false
看了qcow2如果没有snapshot也能enlarge
发表评论