FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Thu May 13, 2021 20:48



Post new topic Reply to topic  [ 4 posts ] 
IO Errors on Directory with many files 
Author Message

Joined: Thu Jun 07, 2012 18:50
Posts: 2
Post IO Errors on Directory with many files
I'm experiencing a problem I hope someone can provide some insight into.

I have a Windows 2003 server that I maintain a test instance of as a VM using VirtualBox. As VirtualBox apparently has issues with ignoring flush requests from the guest and the SATA controller is known to be unstable in theis version of Virtualbox, I have configured it to use the LSI SCSI controller and am setting the ignoreflush flag to 0. On a regular basis I update the files on the VM from the physical server by booting the vm with a debian boot cd, mounting the volumes using ntfs-3g 2011.4.12AR.6 and using rsync to update the image to match the physical machine. On the server I have a data directory (D:/Data/AppData/) from an application that contains approximately 128,000 files. Rsync successfully transfers the files until it reaches this directory at which point I see messages like the following in the daemon.log

Quote:
Jun 6 00:15:21 localhost vm0 ntfs-3g[1505]: Version 2011.4.12AR.6 external FUSE 28
Jun 6 00:15:21 localhost vm0 ntfs-3g[1505]: Mounted /dev/sda3 (Read-Write, label "New Volume", NTFS 3.1)
Jun 6 00:15:21 localhost vm0 ntfs-3g[1505]: Cmdline options: rw,sync,blkdev,windows_names
Jun 6 00:15:21 localhost vm0 ntfs-3g[1505]: Mount options: rw,sync,blkdev,allow_other,nonempty,relatime,fsname=/dev/sda3,blkdev,blksize=4096
Jun 6 00:15:21 localhost vm0 ntfs-3g[1505]: Ownership and permissions disabled, configuration type 7
Jun 6 00:15:46 localhost vm0 ntfs-3g[1505]: ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument
Jun 6 00:21:05 localhost vm0 ntfs-3g[1505]: ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument
Jun 6 00:26:08 localhost vm0 ntfs-3g[1505]: ntfs_rl_sparse: bad runlist: Invalid argument
Jun 6 00:26:08 localhost vm0 ntfs-3g[1505]: Could not check whether sparse
Jun 6 00:26:08 localhost vm0 ntfs-3g[1505]: Mapping pairs update failed: Input/output error
Jun 6 00:26:08 localhost vm0 ntfs-3g[1505]: ntfs_rl_sparse: bad runlist: Invalid argument
Jun 6 00:26:08 localhost vm0 ntfs-3g[1505]: Could not check whether sparse
Jun 6 00:26:08 localhost vm0 ntfs-3g[1505]: Failed to restore old mapping pairs: Input/output error
Jun 6 00:26:08 localhost vm0 ntfs-3g[1505]: Failed to enlarge attribute: Input/output error
Jun 6 00:26:08 localhost vm0 ntfs-3g[1505]: ntfs_attr_mst_pwrite: written=-1: Input/output error
Jun 6 00:26:08 localhost vm0 ntfs-3g[1505]: Failed to write index block 6382, inode 42657: Input/output error
Jun 6 00:26:08 localhost vm0 ntfs-3g[1505]: Failed to add entry to the index: Input/output error


The image I am updating was originally created using VMWare converter and does not seem to be defective in any way. When manually adding, modifying or removing files in this directory I experience no errors. Examining the error messages above and attempting to match this to the source code, the problem appears to be in the runlist. The only scenario that I have found that would result in the 'bad runlist' message is having a negative value in the runlist that is not the LCN_HOLE value. When running
Code:
ntfsinfo -i
with the reported inode number it reports it as being the directory discussed above (D:/Data/AppData/). As, to the best of my knowledge, this directory has no problems before the attempt to rsync data to it, the corruption appears to occur during the synchronization process.

At this time I am in the process of rebuilding the bootcd with a more recent stable release of NTFS-3g (2012.1.15AR.5) in the hopes that this is a bug that has been resolved.

1) Is this a known problem with ntfs-3g 20011.4.12AR.6? If so has it been corrected in more recent releases?
2) How can I prove (or disprove) whether this error is pre-existing in the image I'm starting from?
3) If this error is actually being caused by/during the synchronization process, how can I determine when, why, and how?
4) What additional data would be helpful in diagnosing and debugging this error


Thu Jun 07, 2012 20:01
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: IO Errors on Directory with many files
Hi,

Quote:
1) Is this a known problem with ntfs-3g 20011.4.12AR.6? If so has it been corrected in more recent releases?

I think a very rare error related to runlists has been fixed, but I cannot tell whether it is related to your issue.
Quote:
2) How can I prove (or disprove) whether this error is pre-existing in the image I'm starting from?

For checking and fixing an ntfs file system, I only trust chkdsk on Windows.

For checking only, you can try to build a metadata image, this forces a check on the runlists, and the resulting image can be checked on another computer, including with chkdsk.

Note : disregard the errors like the one below :
Code:
0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument

These errors occur on inodes which were not yet initialized, they are meaningless.
Quote:
3) If this error is actually being caused by/during the synchronization process, how can I determine when, why, and how?

No idea.
Quote:
4) What additional data would be helpful in diagnosing and debugging this error

If it is a runlist error, it should be visible in the ntfsinfo output : the "ntfs_rl_sparse: bad runlist" means there is an invalid lcn (a negative value, which you may spot in the output). Now, this invalid lcn may have been issued during an update and never written to the partition.

Regards

Jean-Pierre


Thu Jun 07, 2012 21:50
Profile

Joined: Thu Jun 07, 2012 18:50
Posts: 2
Post Re: IO Errors on Directory with many files
Jean-Pierre

I appreciate the prompt response,I do have several questions resluting from it tho.
Quote:
For checking and fixing an ntfs file system, I only trust chkdsk on Windows.

I saw a comment on the forum somewhere that the commercially supported package from Tuxera is capable of detecting and repairing issues on NTFS volumes. Would this also be capable of detecting and correcting this type of error?
Quote:
If it is a runlist error, it should be visible in the ntfsinfo output

Would it appear in the plain output of 'ntfsinfo -i' or would I need more verbosity on it to see the LCN numbers?
Quote:
Now, this invalid lcn may have been issued during an update and never written to the partition.

Am I understanding you correctly thinking that the presence of the error may prevent the error from being written to disk?


Fri Jun 08, 2012 00:10
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: IO Errors on Directory with many files
Hi,

Quote:
I saw a comment on the forum somewhere that the commercially supported package from Tuxera is capable of detecting and repairing issues on NTFS volumes. Would this also be capable of detecting and correcting this type of error?

I do not know. I am not a Tuxera employee, and I do not have access to their proprietary products. You may contact their commercial support.
Quote:
Would it appear in the plain output of 'ntfsinfo -i' or would I need more verbosity on it to see the LCN numbers?

Use "ntfsinfo -vi". The valid negative values are shown as plain text.
Quote:
Am I understanding you correctly thinking that the presence of the error may prevent the error from being written to disk?

Most likely yes, because when ntfs-3g discovers an abnormal condition, there is a try to restore the original condition. Such a try was mentioned in your log, however it failed and there could be something visible on disk.
Code:
Failed to restore old mapping pairs: Input/output error


Regards

Jean-Pierre


Fri Jun 08, 2012 08:26
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.