FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Fri May 14, 2021 15:11



Post new topic Reply to topic  [ 4 posts ] 
Bad performance when using ext4 inside VirtualBox 
Author Message

Joined: Thu Mar 04, 2010 20:34
Posts: 7
Post Bad performance when using ext4 inside VirtualBox
Hi!

I observed some strange behaviour when using VirtualBox in conjunction with ntfs-3g. I don't know if it is a problem with VirtualBox oder ntfs-3g but maybe someone can help me tracking this down.

When I place a VDI (virtualbox disk image) on a ntfs volume mounted with ntfs-3g, attach it to a virtual machine and format it to ext4 inside of this machine, I get very high disk usage on the host system and very bad performance on the guest system.

Steps to reproduce:
1. Create a virtual machine in VirtualBox
2. During this process create a VDI on a ntfs partition which is mounted with ntfs-3g
3. Install a Debian Squeeze inside this virtual machine
4. During setup change the partition layout so that the partitions are formatted to ext4

Result:
- Setup will be very slow (at least 5 times slower than normal)
- The hard disk of your host system will make much noise and will be active all the time. Sounds like the head is jumping rapidly between fixed positions.

Reproducable: always

Some things I checked:
- Using ext3 inside of the guest system works without problems and very fast
- Using ext4 inside of the guest system and place the VDI on a ext4 partition instead of a ntfs parition works without problems
- The problem occurs when using fixed-size-VDIs and dynamically growing VDIs, so this has no impact on the problem

When the problem occurs gnome-system-monitor shows 'blkdev_issue_flash' as waiting channel for process mount.ntfs-3g.

I did a strace on the VirtualBox process and it shows:
Code:
[...]
poll([{fd=14, events=POLLIN}, {fd=25, events=POLLIN|POLLPRI}, {fd=27, events=POLLIN|POLLPRI}, {fd=28, events=POLLIN|POLLPRI}, {fd=29, events=POLLIN|POLLPRI}, {fd=30, events=POLLIN}, {fd=31, events=POLLIN}, {fd=13, events=POLLIN}, {fd=32, events=POLLIN}, {fd=37, events=POLLIN}], 10, 0) = 0 (Timeout)
read(13, 0x22bfbf4, 4096)               = -1 EAGAIN (Resource temporarily unavailable)
read(13, 0x22bfbf4, 4096)               = -1 EAGAIN (Resource temporarily unavailable)
read(30, 0x234ecb4, 4096)               = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=14, events=POLLIN}, {fd=25, events=POLLIN|POLLPRI}, {fd=27, events=POLLIN|POLLPRI}, {fd=28, events=POLLIN|POLLPRI}, {fd=29, events=POLLIN|POLLPRI}, {fd=30, events=POLLIN}, {fd=31, events=POLLIN}, {fd=13, events=POLLIN}, {fd=32, events=POLLIN}, {fd=37, events=POLLIN}], 10, 0) = 0 (Timeout)
read(13, 0x22bfbf4, 4096)               = -1 EAGAIN (Resource temporarily unavailable)
read(13, 0x22bfbf4, 4096)               = -1 EAGAIN (Resource temporarily unavailable)
read(30, 0x234ecb4, 4096)               = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=14, events=POLLIN}, {fd=25, events=POLLIN|POLLPRI}, {fd=27, events=POLLIN|POLLPRI}, {fd=28, events=POLLIN|POLLPRI}, {fd=29, events=POLLIN|POLLPRI}, {fd=30, events=POLLIN}, {fd=31, events=POLLIN}, {fd=13, events=POLLIN}, {fd=32, events=POLLIN}, {fd=37, events=POLLIN}], 10, 49) = 0 (Timeout)
read(13, 0x22bfbf4, 4096)               = -1 EAGAIN (Resource temporarily unavailable)
read(13, 0x22bfbf4, 4096)               = -1 EAGAIN (Resource temporarily unavailable)
read(30, 0x234ecb4, 4096)               = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=14, events=POLLIN}, {fd=25, events=POLLIN|POLLPRI}, {fd=27, events=POLLIN|POLLPRI}, {fd=28, events=POLLIN|POLLPRI}, {fd=29, events=POLLIN|POLLPRI}, {fd=30, events=POLLIN}, {fd=31, events=POLLIN}, {fd=13, events=POLLIN}, {fd=32, events=POLLIN}, {fd=37, events=POLLIN}], 10, 0) = 0 (Timeout)
[...]


A strace on mount.ntfs-3g shows:
Code:
[...]
fsync(3)                                = 0
writev(4, [{"\20\0\0\0\0\0\0\0\251_\231\0\0\0\0\0", 16}], 1) = 16
read(4, "D\0\0\0\26\0\0\0\252_\231\0\0\0\0\0\23\242\3\0\0\0\0\0\350\3\0\0d\0\0\0"..., 135168) = 68
writev(4, [{"\20\0\0\0\303\377\377\377\252_\231\0\0\0\0\0", 16}], 1) = 16
read(4, "\340\1\0\0\20\0\0\0\253_\231\0\0\0\0\0\23\242\3\0\0\0\0\0\350\3\0\0d\0\0\0"..., 135168) = 480
pwrite(3, "\220\1\0\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 400, 283668353096) = 400
pread(3, "INDX(\0\t\0\333\335O\20\0\0\0\0\3\0\0\0\0\0\0\0(\0\0\0\300\1\0\0"..., 4096, 10588512256) = 4096
pread(3, "INDX(\0\t\0/}E \0\0\0\0\4\0\0\0\0\0\0\0(\0\0\0h\5\0\0"..., 4096, 10588516352) = 4096
pwrite(3, "INDX(\0\t\0/}E \0\0\0\0\4\0\0\0\0\0\0\0(\0\0\0h\5\0\0"..., 4096, 10588516352) = 4096
pwrite(3, "FILE0\0\3\0\0\0\0\0\0\0\0\0]\0\1\0008\0\1\0\360\1\0\0\0\4\0\0"..., 1024, 166046616576) = 1024
writev(4, [{"\30\0\0\0\0\0\0\0\253_\231\0\0\0\0\0", 16}, {"\220\1\0\0\0\0\0\0", 8}], 2) = 24
read(4, "8\0\0\0\24\0\0\0\254_\231\0\0\0\0\0\23\242\3\0\0\0\0\0\350\3\0\0d\0\0\0"..., 135168) = 56
fsync(3)                                = 0
writev(4, [{"\20\0\0\0\0\0\0\0\254_\231\0\0\0\0\0", 16}], 1) = 16
read(4, "D\0\0\0\26\0\0\0\255_\231\0\0\0\0\0\23\242\3\0\0\0\0\0\350\3\0\0d\0\0\0"..., 135168) = 68
writev(4, [{"\20\0\0\0\303\377\377\377\255_\231\0\0\0\0\0", 16}], 1) = 16
read(4, "\340\1\0\0\20\0\0\0\256_\231\0\0\0\0\0\23\242\3\0\0\0\0\0\350\3\0\0d\0\0\0"..., 135168) = 480
pwrite(3, "\220\1\0\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 400, 283668353096) = 400
pread(3, "INDX(\0\t\0\333\335O\20\0\0\0\0\3\0\0\0\0\0\0\0(\0\0\0\300\1\0\0"..., 4096, 10588512256) = 4096
pread(3, "INDX(\0\t\0/}E \0\0\0\0\4\0\0\0\0\0\0\0(\0\0\0h\5\0\0"..., 4096, 10588516352) = 4096
pwrite(3, "INDX(\0\t\0/}E \0\0\0\0\4\0\0\0\0\0\0\0(\0\0\0h\5\0\0"..., 4096, 10588516352) = 4096
pwrite(3, "FILE0\0\3\0\0\0\0\0\0\0\0\0]\0\1\0008\0\1\0\360\1\0\0\0\4\0\0"..., 1024, 166046616576) = 1024
writev(4, [{"\30\0\0\0\0\0\0\0\256_\231\0\0\0\0\0", 16}, {"\220\1\0\0\0\0\0\0", 8}], 2) = 24
read(4, "8\0\0\0\24\0\0\0\257_\231\0\0\0\0\0\23\242\3\0\0\0\0\0\350\3\0\0d\0\0\0"..., 135168) = 56
fsync(3)                                = 0
writev(4, [{"\20\0\0\0\0\0\0\0\257_\231\0\0\0\0\0", 16}], 1) = 16
read(4, "D\0\0\0\26\0\0\0\260_\231\0\0\0\0\0\23\242\3\0\0\0\0\0\350\3\0\0d\0\0\0"..., 135168) = 68
writev(4, [{"\20\0\0\0\303\377\377\377\260_\231\0\0\0\0\0", 16}], 1) = 16
read(4, "\340\1\0\0\20\0\0\0\261_\231\0\0\0\0\0\23\242\3\0\0\0\0\0\350\3\0\0d\0\0\0"..., 135168) = 480
pwrite(3, "\220\1\0\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 400, 283668353096) = 400
pread(3, "INDX(\0\t\0\333\335O\20\0\0\0\0\3\0\0\0\0\0\0\0(\0\0\0\300\1\0\0"..., 4096, 10588512256) = 4096
pread(3, "INDX(\0\t\0/}E \0\0\0\0\4\0\0\0\0\0\0\0(\0\0\0h\5\0\0"..., 4096, 10588516352) = 4096
pwrite(3, "INDX(\0\t\0/}E \0\0\0\0\4\0\0\0\0\0\0\0(\0\0\0h\5\0\0"..., 4096, 10588516352) = 4096
pwrite(3, "FILE0\0\3\0\0\0\0\0\0\0\0\0]\0\1\0008\0\1\0\360\1\0\0\0\4\0\0"..., 1024, 166046616576) = 1024
writev(4, [{"\30\0\0\0\0\0\0\0\261_\231\0\0\0\0\0", 16}, {"\220\1\0\0\0\0\0\0", 8}], 2) = 24
read(4, "8\0\0\0\24\0\0\0\262_\231\0\0\0\0\0\23\242\3\0\0\0\0\0\350\3\0\0d\0\0\0"..., 135168) = 56
fsync(3)                                = 0
[...]


Software versions:
- Linux: Arch Linux with kernel 2.6.37-ARCH amd64 running
- ntfs-3g: 2011.1.15 external FUSE 28
- FUSE: 2.8.5
- VirtualBox: 4.0.4 OSE
- Guest system: Debian Squeeze amd64 Installer (debian-6.0.0-amd64-netinst.iso)

Can someone imagine what is going wrong here? It may also be a problem with VirtualBox but as I said everything is fine when placing the VDI on a ext4-partition.

Greetings,
Michael


Sat Feb 19, 2011 13:17
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Bad performance when using ext4 inside VirtualBox
Hi Michael,

Quote:
I observed some strange behaviour when using VirtualBox in conjunction with ntfs-3g. I don't know if it is a problem with VirtualBox oder ntfs-3g but maybe someone can help me tracking this down.

I am not familiar with this configuration, so take my comments with care.
Quote:
pwrite(3, "INDX(\0\t\0/}E \0\0\0\0\4\0\0\0\0\0\0\0(\0\0\0h\5\0\0"..., 4096, 10588516352) = 4096
pwrite(3, "FILE0\0\3\0\0\0\0\0\0\0\0\0]\0\1\0008\0\1\0\360\1\0\0\0\4\0\0"..., 1024, 166046616576) = 1024
writev(4, [{"\30\0\0\0\0\0\0\0\261_\231\0\0\0\0\0", 16}, {"\220\1\0\0\0\0\0\0", 8}], 2) = 24
read(4, "8\0\0\0\24\0\0\0\262_\231\0\0\0\0\0\23\242\3\0\0\0\0\0\350\3\0\0d\0\0\0"..., 135168) = 56
fsync(3)

I see a significant number of fsync(3) in your strace, and according to data passed to pwrite(3) the file descriptor 3 is likely to be on ntfs.

Now, fsync() flushes data to disk, thus disabling the write cache, getting lower throughput and creating stress on the output device. fsync() has been newly implemented, so you get a new behavior. If the fsync() is important in your situation (typically for a database), it should be kept this way, if it is not you should avoid it.

Do you have any control on the use of fsync() ? Did you mount the ntfs device with the sync option ? I do not see fsync() on descriptor 4, maybe the fsync() is buried in a specific application.

Regards

Jean-Pierre


Sat Feb 19, 2011 14:08
Profile

Joined: Thu Mar 04, 2010 20:34
Posts: 7
Post Re: Bad performance when using ext4 inside VirtualBox
jpa wrote:
I see a significant number of fsync(3) in your strace, and according to data passed to pwrite(3) the file descriptor 3 is likely to be on ntfs.

Indeed this seems to be the problem. I tried again using ext3 inside of the virtual machine and a strace on mount.ntfs-3g showed a lot of reads and writes but almost no fsyncs.

jpa wrote:
Do you have any control on the use of fsync() ? Did you mount the ntfs device with the sync option ? I do not see fsync() on descriptor 4, maybe the fsync() is buried in a specific application.

The ntfs device is mounted with default options, mount shows:
Code:
/dev/sda2 on /windows/Daten type fuseblk (rw,nosuid,nodev,allow_other,blksize=4096)

Inside of the virtual machine, mount shows:
Code:
/dev/sda1 on /target type ext3 (rw,relatime,errors=remount-ro,data=ordered) [when using ext3]
/dev/sda1 on /target type ext4 (rw,relatime,errors=remount-ro,barrier=1,data=ordered) [when using ext4]


I think here we found the problem: If I understand it correctly barrier=1 causes ext4 to flush after a set of changes has been written to the journal. Remounting the ext4 target device with barrier=0 before file copying starts solves the problem completely.

The question now is: why does it perform well when the VDI is lying on a ext4 partition but not when it is lying on ntfs? Because ext4 is handled by the kernel I don't think I can do a strace on that, do I?


Sat Feb 19, 2011 15:32
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Bad performance when using ext4 inside VirtualBox
Hi Michael,

Quote:
I think here we found the problem: If I understand it correctly barrier=1 causes ext4 to flush after a set of changes has been written to the journal. Remounting the ext4 target device with barrier=0 before file copying starts solves the problem completely.

Mitigated news which confirm the issue. You have transactions activated in ext4, but when run over ntfs which is not transactional, you do not get real transactions. After a power disruption you may have to check both file systems if you disable barriers (it would apparently be the same with ext3 over ntfs).
Quote:
The question now is: why does it perform well when the VDI is lying on a ext4 partition but not when it is lying on ntfs?

The barriers used by ext4 mean syncing the data at commit points. If running ext4 over ntfs, they have to be implemented as fsyncs, as there is no other way of forcing a sync in ntfs.

Do you have the same cluster size in host ntfs and guest ext4 ? Did you try defragmenting the host ntfs ?
Quote:
Because ext4 is handled by the kernel I don't think I can do a strace on that, do I?

You should be able to, if you strace some Virtualbox process.

Regards

Jean-Pierre


Tue Feb 22, 2011 16:51
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.