FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Wed May 12, 2021 00:29



Post new topic Reply to topic  [ 13 posts ] 
Problem using ntfsresize on GPT disk 
Author Message

Joined: Sat Mar 23, 2013 16:04
Posts: 7
Post Problem using ntfsresize on GPT disk
Using GParted an attempt to shrink a backup NTFS partition on a GPT disk failed using ntfsresize. Here is the GParted log:

""GParted 0.14.1 --enable-libparted-dmraid

Libparted 3.1
Shrink /dev/sdd16 from 223.77 GiB to 167.41 GiB 00:09:14 ( ERROR )

calibrate /dev/sdd16 00:00:01 ( SUCCESS )

path: /dev/sdd16
start: 2,199,269,376
end: 2,668,546,047
size: 469,276,672 (223.77 GiB)
check file system on /dev/sdd16 for errors and (if possible) fix them 00:00:32 ( SUCCESS )

ntfsresize -P -i -f -v /dev/sdd16

ntfsresize v2013.1.13AR.1 (libntfs-3g)
Device name : /dev/sdd16
NTFS volume version: 3.1
Cluster size : 4096 bytes
Current volume size: 240269652480 bytes (240270 MB)
Current device size: 240269656064 bytes (240270 MB)
Checking for bad sectors ...
Checking filesystem consistency ...
Accounting clusters ...
Space in use : 179756 MB (74.8%)
Collecting resizing constraints ...
Estimating smallest shrunken size supported ...
File feature Last used at By inode
Multi-Record : 236343 MB 607674
$MFTMirr : 18 MB 1
Compressed : 208775 MB 594760
Ordinary : 238117 MB 675207
You might resize at 179755118592 bytes or 179756 MB (freeing 60514 MB).
Please make a test run using both the -n and -s options before real resizing!
shrink file system 00:08:41 ( ERROR )

run simulation 00:08:41 ( ERROR )

ntfsresize -P --force --force /dev/sdd16 -s 179755286527 --no-action

ntfsresize v2013.1.13AR.1 (libntfs-3g)
Device name : /dev/sdd16
NTFS volume version: 3.1
Cluster size : 4096 bytes
Current volume size: 240269652480 bytes (240270 MB)
Current device size: 240269656064 bytes (240270 MB)
New volume size : 179755278848 bytes (179756 MB)
Checking filesystem consistency ...
Accounting clusters ...
Space in use : 179756 MB (74.8%)
Collecting resizing constraints ...
Needed relocations : 5303034 (21722 MB)
Schedule chkdsk for NTFS consistency check at Windows boot time ...
Resetting $LogFile ... (this might take a while)
Relocating needed data ...
ERROR(30): Couldn't update MFT own record: Read-only file system"

Can you help me correct this problem or inform me if this is an ntfsresize limitation or bug ?

Thank you !


Sat Mar 23, 2013 16:10
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Problem using ntfsresize on GPT disk
Hi,

Quote:
ntfsresize -P --force --force /dev/sdd16 -s 179755286527 --no-action
[...]
ERROR(30): Couldn't update MFT own record: Read-only file system"

Can you help me correct this problem or inform me if this is an ntfsresize limitation or bug ?

Hmm. This looks like a bug in a feature which was introduced with this version : the ability to relocate the MFT when its old location was outside the target partition limits.
When running the test run, the partition is mounted read-only to avoid accidental modification... and this safety measure has prevented the bug from destroying your data.

This situation was not supported in older versions, so I have to recommend you try another tool (Windows has its own shrinker), or define a bigger target partition, so that the MFT need not be relocated.

In theory you can do the actual resize, but I would strongly recommend to do a full backup before doing such an attempt.

Regards

Jean-Pierre


Sat Mar 23, 2013 23:06
Profile

Joined: Sat Mar 23, 2013 16:04
Posts: 7
Post Re: Problem using ntfsresize on GPT disk
jpa wrote:
Hi,

Quote:
ntfsresize -P --force --force /dev/sdd16 -s 179755286527 --no-action
[...]
ERROR(30): Couldn't update MFT own record: Read-only file system"

Can you help me correct this problem or inform me if this is an ntfsresize limitation or bug ?

Hmm. This looks like a bug in a feature which was introduced with this version : the ability to relocate the MFT when its old location was outside the target partition limits.
When running the test run, the partition is mounted read-only to avoid accidental modification... and this safety measure has prevented the bug from destroying your data.

This situation was not supported in older versions, so I have to recommend you try another tool (Windows has its own shrinker), or define a bigger target partition, so that the MFT need not be relocated.

In theory you can do the actual resize, but I would strongly recommend to do a full backup before doing such an attempt.

Regards

Jean-Pierre


Going from last to first <g>:

Regarding your last sentence, I am actually attempting to resize a backup partition of my original NTFS partition from a boot CD ( PartedMagic with GParted ) in order to make more space on the backup disk when this error occurs.

When doing the original backup of the NTFS partition from which I boot Windows, from the PartedMagic boot CD using GParted, everything succeeded. But GParted does not allow the end-user to resize during the copy so I must do it afterwards separately. Are you saying that when I attempt to shrink the backup I can try not to shrink it as far as I can in order to try to avoid this 'ntfsresize' bug ? I can try that if this is what you mean.

As I understand your explanation this is a bug in 'ntfsresize' for the version I am running. I hope there are plans to fix this bug in a future ( next ? ) version of ntfs-3g.

For now I will try to mount this partition in Windows from my backup disk and see if it can shrink the partition accordingly.

Thanks very much for your help.


Sun Mar 24, 2013 00:04
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Problem using ntfsresize on GPT disk
Hi,

Quote:
I am actually attempting to resize a backup partition of my original NTFS partition from a boot CD ( PartedMagic with GParted ) in order to make more space on the backup disk when this error occurs.

Resizing the partition will probably not change the size of the backup. Generally, only used zones are saved (but this depends on how you do the backup).
Quote:
Are you saying that when I attempt to shrink the backup I can try not to shrink it as far as I can in order to try to avoid this 'ntfsresize' bug ? I can try that if this is what you mean.

Yes, this depends on where the MFT is located. There is some part of it somewhere between 180GB and 240GB, you may try resizing at 200GB or 220GB, so that the MFT need not be relocated. You can also try defragmenting, as this will group the possible scattered pieces of MFT.
Quote:
As I understand your explanation this is a bug in 'ntfsresize' for the version I am running. I hope there are plans to fix this bug in a future ( next ? ) version of ntfs-3g.

Yes, and I have even attached a proposed fix, can you try it ? Be sure not to corrupt both the original and the backup.

Regards

Jean-Pierre


Attachments:
ntfsresize.relocmft.patch.gz [1.52 KiB]
Downloaded 947 times
Sun Mar 24, 2013 10:47
Profile

Joined: Sat Mar 23, 2013 16:04
Posts: 7
Post Re: Problem using ntfsresize on GPT disk
jpa wrote:
Hi,

Quote:
I am actually attempting to resize a backup partition of my original NTFS partition from a boot CD ( PartedMagic with GParted ) in order to make more space on the backup disk when this error occurs.

Resizing the partition will probably not change the size of the backup. Generally, only used zones are saved (but this depends on how you do the backup).
Quote:
Are you saying that when I attempt to shrink the backup I can try not to shrink it as far as I can in order to try to avoid this 'ntfsresize' bug ? I can try that if this is what you mean.

Yes, this depends on where the MFT is located. There is some part of it somewhere between 180GB and 240GB, you may try resizing at 200GB or 220GB, so that the MFT need not be relocated. You can also try defragmenting, as this will group the possible scattered pieces of MFT.
Quote:
As I understand your explanation this is a bug in 'ntfsresize' for the version I am running. I hope there are plans to fix this bug in a future ( next ? ) version of ntfs-3g.

Yes, and I have even attached a proposed fix, can you try it ? Be sure not to corrupt both the original and the backup.

Regards

Jean-Pierre


From your first answer I do not think I have been clear about what I was trying to resize so I will explain. I made a copy of an NTFS partition on a backup disk. After that I attempted to shrink the copy I had made from within GParted, since the actual used amount for the partition was much less than the size of the partition. GParted uses the 'ntfsresize' command to do this. I attempted the resize on the copy so that I could free up more unallocated space on the backup disk. The original NTFS partiton is completely unaffected by any of this.

I do not do anything with the copy other than keep it in case of an emergency if my original NTFS partition somehow gets corrupted. The backup disk is an external drive which is always turned off when I actually run anything on my computer and is turned on only to do further backups. I do all my backups by booting up from a CD and running in memory the operating system of that CD. The CD happens to be the Parted Magic distribution, which is its own Linux distro with mainly recovery and partitoning tools, of which the most prominent is GParted. I also have the GParted boot CD but I prefer Parted Magic as it is richer in recovery tools.

I appreciate the patch but I have already used a native Windows program, which is also distributed as a Boot CD, to shrink the size of the backup partition. It was successful but the native Windows program ( Minitool Partition Wizard ) has some other problems and is inferior to GParted IMO.

Naturally I would love to see 'ntfsresize' via your patch updated and incorporated into GParted. Like any other end-user I would like to use the minimum number of programs to get things done. I am a very good computer programmer myself and I appreciate the effort you have made to produce the patch so quickly for me to test.


Sun Mar 24, 2013 14:36
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Problem using ntfsresize on GPT disk
Hi,

Quote:
I made a copy of an NTFS partition on a backup disk. After that I attempted to shrink the copy I had made from within GParted, since the actual used amount for the partition was much less than the size of the partition.

If you blindly copy the ntfs partition to a backup disk, used and unused data alike (e.g. by using dd(1)), then by doing an ntfsresize you will indeed free space. And when you restore you have to resize back to the original partition size.

But backup software do not work like that, they only copy used space. A possibility is to backup into a sparse file, showing the apparent original size but having holes in it instead of unused data. In such situation resizing will not save space at all.

Have you read the "man ntfsclone" ?

Regards

Jean-Pierre


Sun Mar 24, 2013 22:42
Profile

Joined: Sat Mar 23, 2013 16:04
Posts: 7
Post Re: Problem using ntfsresize on GPT disk
jpa wrote:
Hi,

Quote:
I made a copy of an NTFS partition on a backup disk. After that I attempted to shrink the copy I had made from within GParted, since the actual used amount for the partition was much less than the size of the partition.

If you blindly copy the ntfs partition to a backup disk, used and unused data alike (e.g. by using dd(1)), then by doing an ntfsresize you will indeed free space. And when you restore you have to resize back to the original partition size.

But backup software do not work like that, they only copy used space. A possibility is to backup into a sparse file, showing the apparent original size but having holes in it instead of unused data. In such situation resizing will not save space at all.

Have you read the "man ntfsclone" ?

Regards

Jean-Pierre


Why do you say that backup software only copies the used space ? If that were the case would not my backup partition be much smaller than my original partition ? When I use GParted to copy the partition the backup copy is the same size as the original. In fact GParted does not even allow me to copy the partition to an unallocated area which is larger than the size of the used data in my original partition but smaller than the total size of the original partition. Some backup software, like the aforementioned Windows/boot product Minitools Partition Wizard, does allow me to copy a partuition to an area of unallocated space which is the same size or more of my original partition's used space but smaller than my original partition's total size. But Minitools has other problems ( among which is that they do not listen to their end-users at all ) and I would rather use GParted.


Tue Mar 26, 2013 01:13
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Problem using ntfsresize on GPT disk
Hi,

Quote:
Why do you say that backup software only copies the used space ?

Yes
Quote:
If that were the case would not my backup partition be much smaller than my original partition ?

Yes, you would backup to a file whose size would depend on the amount of space used.
Quote:
When I use GParted to copy the partition the backup copy is the same size as the original. In fact GParted does not even allow me to copy the partition to an unallocated area which is larger than the size of the used data in my original partition but smaller than the total size of the original partition.

GParted is a partition manager, not a backup tool. It does not have the ability to identify which sectors are used in all the file system types which can be hosted in a managed partition.

Have a look at ntfsclone.

Regards

Jean-Pierre


Tue Mar 26, 2013 13:11
Profile

Joined: Sat Mar 23, 2013 16:04
Posts: 7
Post Re: Problem using ntfsresize on GPT disk
jpa wrote:
Hi,

Quote:
Why do you say that backup software only copies the used space ?

Yes
Quote:
If that were the case would not my backup partition be much smaller than my original partition ?

Yes, you would backup to a file whose size would depend on the amount of space used.
Quote:
When I use GParted to copy the partition the backup copy is the same size as the original. In fact GParted does not even allow me to copy the partition to an unallocated area which is larger than the size of the used data in my original partition but smaller than the total size of the original partition.

GParted is a partition manager, not a backup tool. It does not have the ability to identify which sectors are used in all the file system types which can be hosted in a managed partition.

Have a look at ntfsclone.

Regards

Jean-Pierre


Now I understand.

Despite your saying that GParted is a partition manager and not a backup tool, there is no reason it cannot use 'ntfsclone' to copy an NTFS partition to unallocated space on a hard drive. I am going to suggest to the GParted developers that they use 'ntfsclone' to do so. Then they would be able to copy an NTFS partition to an unallocated area on a disk which is >= the used area of the source NTFS partition but < the total size of the source NTFS partition since 'ntfsclone' will copy only the used data.


Wed Mar 27, 2013 03:29
Profile

Joined: Sat Mar 23, 2013 16:04
Posts: 7
Post Re: Problem using ntfsresize on GPT disk
jpa wrote:
Hi,

Quote:
Why do you say that backup software only copies the used space ?

Yes
Quote:
If that were the case would not my backup partition be much smaller than my original partition ?

Yes, you would backup to a file whose size would depend on the amount of space used.
Quote:
When I use GParted to copy the partition the backup copy is the same size as the original. In fact GParted does not even allow me to copy the partition to an unallocated area which is larger than the size of the used data in my original partition but smaller than the total size of the original partition.

GParted is a partition manager, not a backup tool. It does not have the ability to identify which sectors are used in all the file system types which can be hosted in a managed partition.

Have a look at ntfsclone.

Regards

Jean-Pierre


After reading the man page for 'ntfsclone' it seems that if it is used to copy a partiton from one device to another device it still fills in the used space with zeros so that the size of the destination partition must be as large as the size of the source partition. I think you are saying that when a partition is copied this way it is easier to compress with 'ntfsresize'. But I do not know if the 'ntfsresize' bug which I have encountered would still happen if I used 'ntfsclone' to copy the partition. Are you saying that the 'ntfsresize' bug would not happen in this case ?


Thu Mar 28, 2013 02:26
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Problem using ntfsresize on GPT disk
Hi,

Quote:
After reading the man page for 'ntfsclone' it seems that if it is used to copy a partiton from one device to another device it still fills in the used space with zeros so that the size of the destination partition must be as large as the size of the source partition.

If you copy a partition to another partition the target partition must be the same size as the original partition, but ntfsclone only copies the used sectors. The manual clearly says they are left unchanged when copying to a partition, read it carefully : filling with zeroes only occurs when cloning to standard output, and, in the sparse file case the unused sectors are seen as zeroes, but they are not written, they are not even allocated, they simply are holes in the file.
Code:
ntfsclone  will efficiently clone (copy, save, backup, restore) or res-
cue an NTFS filesystem to a sparse file, image, device  (partition)  or
standard  output.   It  works  at disk sector level and copies only the
used data. Unused disk space becomes zero  (cloning  to  sparse  file),
encoded  with  control  codes  (saving  in  special image format), left
unchanged (cloning to a disk/partition) or filled with  zeros  (cloning 
to standard output).

Quote:
I think you are saying that when a partition is copied this way it is easier to compress with 'ntfsresize'

No at all. I, of course, suggest backing up to a file, and I also say that using ntfsresize for that purpose is irrelevant when using a proper backup software. Below are a three scenarios for you to test :

1) backup the partition "/dev/sdd16" to the (sparse) file "backup", check sizes and restore :
Code:
ntfsclone --overwrite backup /dev/sdd16
du backup
du --apparent-size backup
ntfsclone --overwrite /dev/sdd16 backup

2) same purpose, but using a special backup format :
Code:
ntfsclone --save-image --overwrite backup /dev/sdd16
du backup
ntfsclone --restore-image --overwrite /dev/sdd16 backup

3) same purpose, but using a compressed backup
Code:
ntfsclone --save-image --output - /dev/sdd16 | gzip > backup
du backup
gzip -cd backup | ntfsclone --restore-image --overwrite /dev/sdd16 -

Regards

Jean-Pierre


Thu Mar 28, 2013 13:02
Profile

Joined: Sat Mar 23, 2013 16:04
Posts: 7
Post Re: Problem using ntfsresize on GPT disk
jpa wrote:
Hi,

Quote:
After reading the man page for 'ntfsclone' it seems that if it is used to copy a partiton from one device to another device it still fills in the used space with zeros so that the size of the destination partition must be as large as the size of the source partition.

If you copy a partition to another partition the target partition must be the same size as the original partition, but ntfsclone only copies the used sectors. The manual clearly says they are left unchanged when copying to a partition, read it carefully : filling with zeroes only occurs when cloning to standard output, and, in the sparse file case the unused sectors are seen as zeroes, but they are not written, they are not even allocated, they simply are holes in the file.
Code:
ntfsclone  will efficiently clone (copy, save, backup, restore) or res-
cue an NTFS filesystem to a sparse file, image, device  (partition)  or
standard  output.   It  works  at disk sector level and copies only the
used data. Unused disk space becomes zero  (cloning  to  sparse  file),
encoded  with  control  codes  (saving  in  special image format), left
unchanged (cloning to a disk/partition) or filled with  zeros  (cloning 
to standard output).

Quote:
I think you are saying that when a partition is copied this way it is easier to compress with 'ntfsresize'

No at all. I, of course, suggest backing up to a file, and I also say that using ntfsresize for that purpose is irrelevant when using a proper backup software. Below are a three scenarios for you to test :

1) backup the partition "/dev/sdd16" to the (sparse) file "backup", check sizes and restore :
Code:
ntfsclone --overwrite backup /dev/sdd16
du backup
du --apparent-size backup
ntfsclone --overwrite /dev/sdd16 backup

2) same purpose, but using a special backup format :
Code:
ntfsclone --save-image --overwrite backup /dev/sdd16
du backup
ntfsclone --restore-image --overwrite /dev/sdd16 backup

3) same purpose, but using a compressed backup
Code:
ntfsclone --save-image --output - /dev/sdd16 | gzip > backup
du backup
gzip -cd backup | ntfsclone --restore-image --overwrite /dev/sdd16 -

Regards

Jean-Pierre


If 'ntfsclone' copies only the used sectors why does the destination partition have to be the same size or larger than the target partition ? If only the used sectors are copied then it seems to me as if the destination partition need only be as big as the total size of the used sectors of the source partition. Please explain what I am missing in my assumption ?


Fri Mar 29, 2013 01:20
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Problem using ntfsresize on GPT disk
Hi,

Quote:
If 'ntfsclone' copies only the used sectors why does the destination partition have to be the same size or larger than the target partition ?

Take a scrap book of 100 pages. Write something on pages 1, 22, 76 and 99.
Take another book of 100 pages and make it a copy by copying the same information on the same page numbers.
Now repeat with a target book of 120 pages, then with one of 50 pages...

Once more, before I give up : the target destination needs not be a partition.

Regards

Jean-Pierre


Fri Mar 29, 2013 13:36
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 13 posts ] 


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.