FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Sun May 16, 2021 01:59



Post new topic Reply to topic  [ 15 posts ] 
NTFS clone error on restore. ERROR: Invalid command code -1 
Author Message

Joined: Tue Jun 16, 2015 16:22
Posts: 4
Post NTFS clone error on restore. ERROR: Invalid command code -1
Hello,

We are trying to restore our server backup made by ntfsclone . The problem is that we cannot convert it to image and receiving error:
Ntfsclone image version: 10.1
Cluster size : 4096 bytes
Image volume size : 1073738674176 bytes (1073739 MB)
Image device size : 1073738678272 bytes
Space in use : 914928 MB (85.2%)
Offset to image data : 56 (0x38) bytes
Restoring NTFS from image ...
ERROR: Invalid command code -1 at input offset 0x39b8df886e

Currently we are trying version ntfs-3g_ntfsprogs-2015.3.14.
Backup has been successfully created with version v2014.2.15.

We have read some threads by Jean-Pierre André, for example, http://comments.gmane.org/gmane.comp.fi ... .devel/951 where he was helping in nfsclone image recovery. We are currently running resync utility on the image, but not quite sure what to do with its output when it finishes. Seen some threads, where bytes need to be flipped from 0 to 1.

We can attach hexdump output around the offset if that helps?

Thanks


Tue Jun 16, 2015 17:21
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: NTFS clone error on restore. ERROR: Invalid command code -1
Hi,

Quote:
ERROR: Invalid command code -1 at input offset 0x39b8df886e

Command codes are supposed to be either 0 or 1.
Quote:
Backup has been successfully created with version v2014.2.15.

This version is expected to be safe, no problem has been reported so far, though the bug you quoted was left unexplained. I have no clue about what has happened.
What were the command you used for creating the backup, and the one for restoring ?
What is the size of the backup ? Is it consistent with an 85% filled 1TB partition ?

Quote:
We can attach hexdump output around the offset if that helps?

I can try to get some clue out of it, though probably not an easy thing. Try 10K bytes ahead of the error, up to 32K beyond, such as :
dd if=your-backup bs=4096 skip=60526070 count=12 | gzip > buggy-chunk.gz

Note : this forum only accepts compressed attachments.

Regards

Jean-Pierre


Tue Jun 16, 2015 21:51
Profile

Joined: Tue Jun 16, 2015 16:22
Posts: 4
Post Re: NTFS clone error on restore. ERROR: Invalid command code -1
Hello and thanks for reply,

jpa wrote:
Hi,
Quote:
ERROR: Invalid command code -1 at input offset 0x39b8df886e

Command codes are supposed to be either 0 or 1.
Quote:
Backup has been successfully created with version v2014.2.15.

This version is expected to be safe, no problem has been reported so far, though the bug you quoted was left unexplained. I have no clue about what has happened.
What were the command you used for creating the backup, and the one for restoring ?
What is the size of the backup ? Is it consistent with an 85% filled 1TB partition ?

First we do LVM snapshot and then we run command:
ntfsclone -f -s -O /backups/r72vndrwoxy5t2 /dev/mapper/backup-r72vndrwoxy5t2X1
File size seems to be consistent with reported info by ntfsclone which is 915151535025 for this backup.
jpa wrote:
Quote:
We can attach hexdump output around the offset if that helps?

I can try to get some clue out of it, though probably not an easy thing. Try 10K bytes ahead of the error, up to 32K beyond, such as :
dd if=your-backup bs=4096 skip=60526070 count=12 | gzip > buggy-chunk.gz


We are attaching dd output around that places.

By the way, it could be just a coincidence, but the underlying LVM volume is comprised out of two segments: 240G (+/- the place where ntfsclone exits) and 760G. The initial NTFS volume was around 512G, however.


Thanks for looking at it. We are also eager to do a one-off support contract with you if that helps to get the data out.

Petras


Attachments:
buggy_chunk_60536070.gz [47.94 KiB]
Downloaded 917 times
buggy_chunk_60526070.gz [11.66 KiB]
Downloaded 946 times
buggy_chunk_60504070.gz [30.92 KiB]
Downloaded 934 times
Wed Jun 17, 2015 11:08
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: NTFS clone error on restore. ERROR: Invalid command code -1
Hi,

Quote:
We are attaching dd output around that places.

Fine. There are normal ntfsclone blocks up to location 0x39b8df886e. This location is wrong, and normal ntfsclone blocks restart from 0x39b8df8c47. So there are 985 invalid bytes. This is a strange count which either means there are 985 extra bytes or, more likely, an unknown count of bytes are missing (at least 4097-985 bytes would be missing). Note that 985 is a strange number, unrelated to sector or buffer size.

For a first try, I attach a filter which removes 985 bytes. The aim at this stage is to determine whether there are similar errors elsewhere. Please tell what happens when you start (no file to be changed) :
Code:
./fiximg < your-backup | ntfsclone -r -n -

(the source file fiximg.c is also attached in case adjustments would be needed).

Now, I need to know exactly how many blocks are missing, and this requires synchronizing on something. I can synchronize on the backup boot sector if there is a valid one in your backup file, I can also synchronize on the MFT or MFTMirr starts, but I have first to determine where they are located.

To do that, please attach the first 8K of your backup file (boot data) and the last 8K of you backup file (backup boot data).

Quote:
We are also eager to do a one-off support contract with you if that helps to get the data out.

Gratitude is always welcome.

Regards

Jean-Pierre


Attachments:
fiximg.tar.gz [2.45 KiB]
Downloaded 969 times
Wed Jun 17, 2015 14:42
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: NTFS clone error on restore. ERROR: Invalid command code -1
Hi,

Checking further the partial ntfsclone image, I found another wrong location at 0x39b8e00c4f. Surprisingly, there are 3112 strange bytes, and 3112 + 985 = 4097 which is an usual ntfsclone block size. So this looks like a fragment of about 8K bytes has been shifted by 985 bytes.

Attached is an update to fiximg to remove both chunks (985 bytes and 3112 bytes), hoping there are no more errors.

Good luck

Jean-Pierre


Attachments:
fiximg2.tar.gz [2.65 KiB]
Downloaded 906 times
Wed Jun 17, 2015 16:51
Profile

Joined: Tue Jun 16, 2015 16:22
Posts: 4
Post Re: NTFS clone error on restore. ERROR: Invalid command code -1
jpa wrote:
Hi,

Checking further the partial ntfsclone image, I found another wrong location at 0x39b8e00c4f. Surprisingly, there are 3112 strange bytes, and 3112 + 985 = 4097 which is an usual ntfsclone block size. So this looks like a fragment of about 8K bytes has been shifted by 985 bytes.

Attached is an update to fiximg to remove both chunks (985 bytes and 3112 bytes), hoping there are no more errors.

Good luck

Jean-Pierre


Hi,

We are trying second fix now. Meanwhile we have output from first fix which was unsuccessful. Also attaching first and last 8k segments.

./fiximg < rw5im1xot5mb73 | ntfsclone -r -n -
ntfsclone v2015.3.14 (libntfs-3g)
Ntfsclone image version: 10.1
Cluster size : 4096 bytes
Image volume size : 1073738674176 bytes (1073739 MB)
Image device size : 1073738678272 bytes
Space in use : 914928 MB (85.2%)
Offset to image data : 56 (0x38) bytes
Restoring NTFS from image ...
Buggy location detectedd
Writing first partial buffer, 2158 bytes
Writing second partial buffer, 953 bytes
ERROR: restore_image: corrupt image at input offset -10

Thank you,
Petras S.


Attachments:
last-8k.gz [529 Bytes]
Downloaded 943 times
first-8k.gz [2.81 KiB]
Downloaded 908 times
Wed Jun 17, 2015 17:06
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: NTFS clone error on restore. ERROR: Invalid command code -1
Hi,

Good news : your backup boot sector is a valid one. This is fortunate because the MFT and MFTMirr are located in the sane initial part of your backup, so they cannot be used for synchronization.

In your ntfsclone backup, there is a gray zone of exactly 10*4097 bytes (from 0x39b8df786d to 0x39b8e01877) polluted by some data whose limits are not exactly known, roughly from 0x39b8df7e00 to 0x39b8dffe00, about 32K. From this pollution, we can get 8*4097 bytes from 0x39b8df8c47 to 0x39b8e00c4f.

So we can bet there is exactly one block missing, and fiximg can add one extra block replacing the 985 + 3112 bytes it removed, and we can avoid a resynchronization step.

When examining the polluted zone, it is clear that its data is different. So this pollution comes from elsewhere. Nevertheless it shows an ntfsclone pattern, so the bad news is that it could come from elsewhere in the same file, and we have to expect another error at the location where this data should have been.

As the estimated pollution location (0x39b8df7e00 to 0x39b8dffe00) is sector aligned, I would blame either a hardware error, or more likely a file system error (bad sector allocation when you were doing the backup).

Regards

Jean-Pierre


Attachments:
fiximg3.tar.gz [2.95 KiB]
Downloaded 949 times
Wed Jun 17, 2015 21:38
Profile

Joined: Tue Jun 16, 2015 16:22
Posts: 4
Post Re: NTFS clone error on restore. ERROR: Invalid command code -1
Dear Jean-Pierre, thanks so far for the help.

Turns out, we've been looking at the wrong dated backup file. The one we are really looking forward to restore has an offset in a slightly different part, but essentially the same problem. We've tried to modify your fiximg3 OFFSET to the value where it fails minus 0x286e, however it looks like the garbage part might be different from the one you looked at:

./fix3 < /onapp/backups/i/q/iq6ifq2wlrvmie | ntfsclone -r -n -
ntfsclone v2015.3.14 (libntfs-3g)
Ntfsclone image version: 10.1
Cluster size : 4096 bytes
Image volume size : 1073738674176 bytes (1073739 MB)
Image device size : 1073738678272 bytes
Space in use : 954088 MB (88.9%)
Offset to image data : 56 (0x38) bytes
Restoring NTFS from image ...
0.02 percent completed
Buggy location detectedd
Writing first partial buffer, 3284 bytes
Writing second partial buffer, 3923 bytes
ERROR: Invalid command code 70 at input offset 0xfffffffffffffffe
Buggy location detected
Writing first partial buffer, 181 bytes
Inserting a gap of 1 clusters
Writing second partial buffer, 803 bytes

If we attach +/- 8kb dump around that place, would mind adjusting your buggy[] magic offsets?

Thank you


Thu Jun 18, 2015 13:38
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: NTFS clone error on restore. ERROR: Invalid command code -1
Hi,

Quote:
The one we are really looking forward to restore has an offset in a slightly different part, but essentially the same problem.

So this is a repeatable issue which has to be understood. May come from LVM, hardware, file system, ntfsclone.... To what kind of storage were you creating the backup image ?

Quote:
We've tried to modify your fiximg3 OFFSET to the value where it fails minus 0x286e
....
Restoring NTFS from image ...
0.02 percent completed

0.02% is not much. Are you sure you inserted the correct value ?
Quote:
If we attach +/- 8kb dump around that place, would mind adjusting your buggy[] magic offsets?

Probably yes, but when ntfsclone is started with option -n the buggy location is wrong, so I have no precise indication.

The attached fiximg will decode the ntfsclone commands and display more precise information. Please adjust the offset according to your findings, and see where it chokes. For now, you probably need not feed its output to ntfsclone :
Code:
./fiximg4 < /onapp/backups/i/q/iq6ifq2wlrvmie > /dev/null


Then post an 64K byte extract, starting 8K before the choking point.

Regards

Jean-Pierre


Attachments:
fiximg4.tar.gz [4.21 KiB]
Downloaded 966 times
Thu Jun 18, 2015 16:54
Profile

Joined: Wed Jan 11, 2017 13:13
Posts: 4
Post Re: NTFS clone error on restore. ERROR: Invalid command code -1
Hi, I have near the same problem. I did a backup from another computer using clonezilla and my backup is corrupted for some reason. First, I noticed that gzipped file is broken, but I managed to google solution to extract it (with zeroes at the broken data), but image have valid data up to 10.71% (guess I saw this right). Here's an output:
ntfsclone -r -n sda5.ntfs-img.aa.recovered
ntfsclone v2014.2.15AR.2 (libntfs-3g)
Ntfsclone image version: 10.1
Cluster size : 4096 bytes
Image volume size : 168034209792 bytes (168035 MB)
Image device size : 168034212864 bytes
Space in use : 102289 MB (60.9%)
Offset to image data : 56 (0x38) bytes
Restoring NTFS from image ...
ERROR: Bad offset at input location 0x290da3b54
I've cut a part near that location (dd if=sda5.ntfs-img.aa.recovered bs=4096 skip=2690460 count=38 | gzip > broken.dd.gz), hope you can give me solution how to extract the data from this file.


Attachments:
broken.dd.gz [75.28 KiB]
Downloaded 693 times
Wed Jan 11, 2017 13:28
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: NTFS clone error on restore. ERROR: Invalid command code -1
Hi,

Quote:
ERROR: Bad offset at input location 0x290da3b54
I've cut a part near that location (dd if=sda5.ntfs-img.aa.recovered bs=4096 skip=2690460 count=38 | gzip > broken.dd.gz), hope you can give me solution how to extract the data from this file.

From this excerpt, the beginning of the corrupted portion can be found, but not its end. IMHO, as there are unusually frequent repeats of the same pattern, the unzipping recovery did not work properly.

You may try extracting a longer sample starting from the same position (say count=100), but I am not confident this can lead to a recoverable image.

Regards

Jean-Pierre


Wed Jan 11, 2017 15:11
Profile

Joined: Wed Jan 11, 2017 13:13
Posts: 4
Post Re: NTFS clone error on restore. ERROR: Invalid command code -1
Okay, count=100.
Forum complaining about max file size, so http://s000.tinyupload.com/index.php?fi ... 3855475861


Wed Jan 11, 2017 15:24
Profile

Joined: Wed Jan 11, 2017 13:13
Posts: 4
Post Re: NTFS clone error on restore. ERROR: Invalid command code -1
In case that might help, I also scanned entire image with resync. Here's a log: http://rgho.st/8x8kWnzCQ.


Wed Jan 11, 2017 17:26
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: NTFS clone error on restore. ERROR: Invalid command code -1
Hi

Unfortunately, there is no valid image pattern after the initial error. The file was not un-gzipped properly.

Regards

Jean-Pierre


Wed Jan 11, 2017 22:45
Profile

Joined: Wed Jan 11, 2017 13:13
Posts: 4
Post Re: NTFS clone error on restore. ERROR: Invalid command code -1
So there is no way to extract unaffected files? That's sad.
Well, now I'll check all clonezilla backups before formatting the drive.
It doesn't complained about error while copying, but I lost my friend's data.


Thu Jan 12, 2017 09:20
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 15 posts ] 


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.