FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Fri May 14, 2021 15:39



Post new topic Reply to topic  [ 16 posts ] 
ntfsclone ignoring --force switch 
Author Message

Joined: Sat Oct 22, 2011 08:36
Posts: 2
Post ntfsclone ignoring --force switch
I have a Xen VM that I'm trying to create a backup of with ntfsclone

I've taken an LVM snapshot and then mounted the partition using kpartx. The file system is not clean according to ntfsclone, I assume this is because I've taken a snapshot of an active disk. I want to ignore any problems with the disk consistency check so I use the --force option, but this is the output:

[user@host ~]# ntfsclone_bin --force -s -O ntfs-backup.img /dev/mapper/kpartx-mapped-device

ntfsclone v2.0.0 (libntfs 10:0:0)
NTFS volume version: 3.1
Cluster size : 4096 bytes
Current volume size: 26841444352 bytes (26842 MB)
Current device size: 26841448448 bytes (26842 MB)
Scanning volume ...
100.00 percent completed
Accounting clusters ...
Cluster accounting failed at 1819582 (0x1bc3be): extra cluster in $Bitmap
Totally 1 cluster accounting mismatches.
ERROR: Filesystem check failed! Windows wasn't shutdown properly or inconsistent
filesystem. Please run chkdsk /f on Windows then reboot it TWICE.
[user@host ~]#

Is this expected behaviour? I thought with the "--force" option it would ignore the results of the filesystem check?

Thanks :)


Sat Oct 22, 2011 08:43
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: ntfsclone ignoring --force switch
Hi,


Quote:
I've taken an LVM snapshot and then mounted the partition using kpartx. The file system is not clean according to ntfsclone, I assume this is because I've taken a snapshot of an active disk.

Doing so, you got an intermediate state, which is inconsistent.

Quote:
I want to ignore any problems with the disk consistency check so I use the --force option, but this is the output:

Code:
ntfsclone v2.0.0 (libntfs 10:0:0)
NTFS volume version: 3.1
[...]
Cluster accounting failed at 1819582 (0x1bc3be): extra cluster in $Bitmap
Totally 1 cluster accounting mismatches.
ERROR: Filesystem check failed! Windows wasn't shutdown properly or inconsistent


Is this expected behaviour?

Yes. There is a reserved cluster which is not mentioned in any file. Ntfsclone could omit cloning useful information, and refuses to make an invalid clone. You should fix the snapshot before cloning. Maybe you only have to umount your original LV and redo the snapshot.

Quote:
I thought with the "--force" option it would ignore the results of the filesystem check?

No. The "--force" option is to ignore a request for checking the partition, which might be marked on the partition (you probably do not have this mark set). The recommended way of fixing a partition is to do a couple of chkdsk on Windows as indicated in the error message.

Regards

Jean-Pierre


Sat Oct 22, 2011 21:06
Profile

Joined: Sat Oct 22, 2011 08:36
Posts: 2
Post Re: ntfsclone ignoring --force switch
Hi Jean-Pierre, thanks very much for the comprehensive reply, much appreciated. Great work on the tools, keep it up :)


Sat Oct 22, 2011 21:32
Profile

Joined: Sun Nov 04, 2012 00:56
Posts: 6
Post please allow cloning of inconsistent state fs
I have an imminent failing disk drive.
I am attempting to read all possible information from it.
It has a large ntfs partition with small but important amount of information.
Using dd_rhelp in combination with dd_rescue takes way too long, as it attempts to read the entire partition which is not what I need.

Using ntfsclone with "--rescue" option looks like is what I need, however since the fs wasn't closed properly ntfsclone refuses to continue.

So the combination of badsectors and unclean fs shutdown is not permitted to ntfsclone.
Please allow "--force" or "--ignore-fs-check" do what it's supposed to do: include problematic clusters in the image, too, and let me chkdsk on the result AFTER i manage to read the bits from the damaged drive.

Thanks in advance.


Sun Nov 04, 2012 01:08
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: ntfsclone ignoring --force switch
Hi,
Quote:
So the combination of badsectors and unclean fs shutdown is not permitted to ntfsclone.

To be precise, saving an inconsistent file system is not allowed, because it could be unusable after restoring.
Quote:
So the combination of badsectors and unclean fs shutdown is not permitted to ntfsclone.
Please allow "--force" or "--ignore-fs-check" do what it's supposed to do: include problematic clusters in the image, too, and let me chkdsk on the result AFTER i manage to read the bits from the damaged drive.

I agree that one need something to save as much data as can be from a failing drive.

Please try the patch below. This is intended to allow "--ignore-fs-check" when "--rescue" is also requested. You may also need "--force" if the file system is marked as dirty.

Check whether the image can be restored before disposing of the old drive...

Also do a "ntfsfix --clear-bad-sectors" on the new drive to delete the bad sector marks which were cloned.

*** edit

Attached is a patch set with four patches. Depending on the version you are using, you may need all of them or only the latest ones. Only the two latest ones are needed over ntfs-3g 2012.1.15AR.8. Please test the result, as making a clone based on inconsistent data can really lead to a unusable backup.

Regards

Jean-Pierre


Attachments:
ntfsclone.fixes.gz [4.47 KiB]
Downloaded 1209 times
Sun Nov 04, 2012 10:07
Profile

Joined: Sun Nov 04, 2012 00:56
Posts: 6
Post Re: ntfsclone ignoring --force switch
thank you very much, I will post feedback.


Sun Nov 04, 2012 14:20
Profile

Joined: Sun Nov 04, 2012 00:56
Posts: 6
Post Re: ntfsclone ignoring --force switch
So far it compiled and it's working, although only 3 percent done because of the poor disk state. Anyway much faster than commercial tools like symantec ghost, which rescued only a fifth of what ntfsclone did in the same timeframe.

In the meantime I did manage to mount read-only and copy the critical needed files though.
Thanks again. May I reference your patches to Debian package or shall I expect them to be part of the next release?
Many thanks, again!


Tue Nov 06, 2012 20:07
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: ntfsclone ignoring --force switch
Hi,

Quote:
only 3 percent done because of the poor disk state

Probably not enough to be useful...
Quote:
May I reference your patches to Debian package or shall I expect them to be part of the next release?

Some of them have already been released, the other ones should be in the next release, ... which probably means next year.

Regards

Jean-Pierre


Tue Nov 06, 2012 22:19
Profile

Joined: Sun Nov 04, 2012 00:56
Posts: 6
Post Re: ntfsclone ignoring --force switch
jpa wrote:
Hi,

Quote:
only 3 percent done because of the poor disk state

Probably not enough to be useful...


8.73 percent completed by now. still going slower than regular cp over ntfs-3g read-only mount.
looking with ibm's nmon i can see only 4k (=volume cluster size) per each i/o read. strace shows one llseek, one read. maybe lots of fragments?

DiskName Busy Read Write Xfers Size Peak% Peak-RW InFlight │
│sdd 100% 12.0 0.0KB/s 3.0 4.0KB 101% 2182.2KB/s 1 │


jpa wrote:
Quote:
May I reference your patches to Debian package or shall I expect them to be part of the next release?

Some of them have already been released, the other ones should be in the next release, ... which probably means next year.

Regards

Jean-Pierre

thanks


Thu Nov 08, 2012 02:04
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: ntfsclone ignoring --force switch
Hi,

Quote:
8.73 percent completed by now. still going slower than regular cp over ntfs-3g read-only mount.
looking with ibm's nmon i can see only 4k (=volume cluster size) per each i/o read. strace shows one llseek, one read. maybe lots of fragments?

Yes, currently ntfsclone can only copy a single cluster at a time, and changing that would be complex, the more in a rescue situation where you have to locate a bad cluster within a set of clusters read globally.

This should not be a problem if you are cloning to another device. If the target is a file you should use the -s option, otherwise you will create a big fragmented sparse file with dramatic performances.

Regards

Jean-Pierre


Thu Nov 08, 2012 11:34
Profile

Joined: Sun Nov 04, 2012 00:56
Posts: 6
Post Re: ntfsclone ignoring --force switch
jpa wrote:
Quote:
8.73 percent completed by now. still going slower than regular cp over ntfs-3g read-only mount.
looking with ibm's nmon i can see only 4k (=volume cluster size) per each i/o read. strace shows one llseek, one read. maybe lots of fragments?


Yes, currently ntfsclone can only copy a single cluster at a time, and changing that would be complex, the more in a rescue situation where you have to locate a bad cluster within a set of clusters read globally.

This should not be a problem if you are cloning to another device. If the target is a file you should use the -s option, otherwise you will create a big fragmented sparse file with dramatic performances.


Thanks for raising attention about sparse, I was already saving in special image format ;)
There is no need for changing ntfsclone, you were kind enough to allow ignore-fs-check (and extremely fast!). This is more than enough.
Maybe write a separate tool dedicated to ntfs data rescue that would operate similarly to dd_rescue+dd_rhelp. Feeding dd_rescue with a list of blocks to read is possible, so all that is needed for this to work would a list of all sectors that are used. Might save some time when reading X gb of data from a 10X gb partition.
Do you think is this feasible (for example ntfsinfo dumping the list of relative sectors)?


Fri Nov 09, 2012 00:19
Profile

Joined: Fri Nov 09, 2012 16:04
Posts: 3
Post Re: ntfsclone ignoring --force switch
Hello,

I am attempting to install ntfs-3g from Git so that I can have the --ignore-fs-check capability, but ./configure is failing:

./configure: line 12990: syntax error near unexpected token `1.2.2,'
./configure: line 12990: ` AM_PATH_LIBGCRYPT(1.2.2, have_libgcrypt=true ,'


Thoughts?


Fri Nov 09, 2012 16:15
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: ntfsclone ignoring --force switch
Hi,

Quote:
I am attempting to install ntfs-3g from Git so that I can have the --ignore-fs-check capability, but ./configure is failing:

This is not on git yet, it has to follow the quality assurance procedure.

And there will not be a --ignore-fs-check configure option, but an option to ntfsclone.

If you want the option, you can apply the patches posted earlier in this thread.

Regards

Jean-Pierre


Fri Nov 09, 2012 22:07
Profile

Joined: Fri Nov 09, 2012 16:04
Posts: 3
Post Re: ntfsclone ignoring --force switch
Thank you so much for the quick reply. I'm sorry my post wasn't very clear. I actually had two issues:

Issue 1: I was hoping to get the updated --ignore-fs-check option for ntfsclone (I will use the patch for now, as you suggested)

Issue 2: I get the bad token error when I run ./configure (I'm not sure what's causing this one)


Fri Nov 09, 2012 22:42
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: ntfsclone ignoring --force switch
Hi,

Quote:
Issue 2: I get the bad token error when I run ./configure (I'm not sure what's causing this one)


Which ntfs-3g version? with which OS ? autoconf ? which configure options ?

I do not have "AM_PATH_LIBGCRYPT(1.2.2, " in my configure script, though this appears in the configure.ac script used to generate configure.

An autoconf error ?

If you get the git version, you have to generate the configure script by "./autogen.sh" (which relies on autoconf), but if you get a tarball (ntfs-3g-2012.1.15AR.8 from http://www.tuxera.com/community/ntfs-3g-advanced/ is roughly the same as git), the configure script is ready for use (currently I am relying on autoconf 2.68).

If you have an incompatible autoconf, try restarting from a tarball.

Regards

Jean-Pierre


Fri Nov 09, 2012 23:14
Profile

Joined: Fri Nov 09, 2012 16:04
Posts: 3
Post Re: ntfsclone ignoring --force switch
Got it. Thanks again!


Fri Nov 09, 2012 23:20
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 16 posts ] 


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.