FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Thu May 13, 2021 21:58



Post new topic Reply to topic  [ 13 posts ] 
ntfsresize --force option is ignored 
Author Message

Joined: Thu Jul 28, 2011 17:55
Posts: 6
Post ntfsresize --force option is ignored
The --force option for ntfsresize v2011.4.12AR.4 (libntfs-3g) appears to be ignored.

This can be demonstrated with the following steps:

1) Create a 512 MB partition and format it with the NTFS file system.
NOTE: I created mine on /dev/sda1

2) Shrink the NTFS file system down to 256 MB with the following command:
ntfsresize -P --force /dev/sda1 -s 268435455

3) Notice that even though the --force option was specified, the command still prompts with:
Are you sure you want to proceed (y/[n])?

This problem was originally reported with GParted Live 0.9.0-6 which is based on Debian Live. The GParted bug report can be found at the following link:
Bug 655215 - NTFS partition resize fails

For reference, the following is a log file for the above ntfsresize command:
Code:
user@debian:~$ sudo ntfsresize -P --force /dev/sda1 -s 268435455
ntfsresize v2011.4.12AR.4 (libntfs-3g)
Device name        : /dev/sda1
NTFS volume version: 3.1
Cluster size       : 4096 bytes
Current volume size: 536867328 bytes (537 MB)
Current device size: 536870912 bytes (537 MB)
New volume size    : 268427776 bytes (269 MB)
Checking filesystem consistency ...
Accounting clusters ...
Space in use       : 3 MB (0.5%)
Collecting resizing constraints ...
Needed relocations : 0 (0 MB)
WARNING: Every sanity check passed and only the dangerous operations left.
Make sure that important data has been backed up! Power outage or computer
crash may result major data loss!
Are you sure you want to proceed (y/[n])? y
Schedule chkdsk for NTFS consistency check at Windows boot time ...
Resetting $LogFile ... (this might take a while)
Updating $BadClust file ...
Updating $Bitmap file ...
Updating Boot record ...
Syncing device ...
Successfully resized NTFS on device '/dev/sda1'.
You can go on to shrink the device for example with Linux fdisk.
IMPORTANT: When recreating the partition, make sure that you
  1)  create it at the same disk sector (use sector as the unit!)
  2)  create it with the same partition type (usually 7, HPFS/NTFS)
  3)  do not make it smaller than the new NTFS filesystem size
  4)  set the bootable flag for the partition if it existed before
Otherwise you won't be able to access NTFS or can't boot from the disk!
If you make a mistake and don't have a partition table backup then you
can recover the partition table by TestDisk or Parted's rescue mode.
user@debian:~$


Thu Jul 28, 2011 18:19
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: ntfsresize --force option is ignored
Hi,

Quote:
3) Notice that even though the --force option was specified, the command still prompts with:
Are you sure you want to proceed (y/[n])?


Well, the manual says :
Code:
-f, --force
              Forces ntfsresize to proceed with the resize operation  even  if
              the filesystem is marked for consistency check.

The option -f is not supposed to avoid a request for confirmation...

But, the manual is apparently wrong , and according to the actual code, each "-f" is meant to avoid a single reason for quitting. In your example, your partition was probably dirty (most likely after a previous ntfsresize). In that situation you have to repeat the -f :

Code:
[linux@dimension ntfs-3g]$ touch image.tmp
# create a 100MB ntfs image
[linux@dimension ntfs-3g]$ /sbin/mkntfs -Ff -s512 -c512 image.tmp 100000
[...]
mkntfs completed successfully. Have a nice day.
# resize to 50M, the partition is marked dirty, no question
[linux@dimension ntfs-3g]$ ntfsprogs/ntfsresize -f -s 50M
[...]
Schedule chkdsk for NTFS consistency check at Windows boot time ...
[...]
Successfully resized NTFS on device 'image.tmp'.
[...]
# resize to 40M, it was dirty, then a question.
[linux@dimension ntfs-3g]$ ntfsprogs/ntfsresize -f -s 40M image.tmp
[...]
Are you sure you want to proceed (y/[n])? y
Schedule chkdsk for NTFS consistency check at Windows boot time ...
[...]
# resize to 30M, it was dirty, double the -f, and no question
[linux@dimension ntfs-3g]$ ntfsprogs/ntfsresize -ff -s 30M image.tmp
[...]
Successfully resized NTFS on device 'image.tmp'.
[...]


Now, this is unchanged from the previous ntfsresize, but there is a change : when requested to confirm and there is an error when getting a reply (typically EOF), a reject is assumed for safety.

I can change that if the old behavior is required, but the very question is : should a dirty partition be resized blindly ?

Regards

Jean-Pierre


Thu Jul 28, 2011 22:05
Profile

Joined: Thu Jul 28, 2011 17:55
Posts: 6
Post Re: ntfsresize --force option is ignored
First I wish to begin by thanking you Jean-Peirre for picking up the task of maintaining the ntfsprogs package. This is likely a very large undertaking and is much appreciated.

Also thank you for your prompt response and suggestion on how to work around the current ntfsresize behaviour by using an extra "-f" option.

Now on to your question which I believe has two parts.

Quote:
Now, this is unchanged from the previous ntfsresize, but there is a change : when requested to confirm and there is an error when getting a reply (typically EOF), a reject is assumed for safety.

I can change that if the old behavior is required, but the very question is : should a dirty partition be resized blindly ?


Q1: How should the --force (-f) option behave?

My thoughts are that it makes more sense for the --force (or -f) option to not send a question prompt to the user interface at all. In my opinion this is preferable to sending something like "-ffffffff" in an attempt to guess how many question prompts might arise.

The closest comparison I can think of is the e2fsck command which has an option "-y" that will answer "yes" to all questions. I believe that this is preferable behaviour than having to use "-yyyyyyy..." to try to guess how many prompts might require.

Of course if ntfsresize comes across a situation that will prevent the resize operation from succeeding, then by all means the command should halt and advise the user what actions they might take to resolve the situation.

Q2: Should a dirty partition be resized blindly?

This is a good question.

My past experience has been that using the ntfsresize command multiple times on an NTFS file system is a reasonably safe thing to do. In situations where the NTFS file system was truly in an inconsistent state then the ntfsresize command did report this as the problem and suggested running "chkdsk /f".

Is is possible for the ntfsresize to know if the partition is truly "dirty" or simply marked "dirty" by ntfsresize?
If so then this might help to determine when ntfsresize can or can not proceed with a good chance of success.


Sat Jul 30, 2011 02:05
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: ntfsresize --force option is ignored
Hi,

Quote:
The closest comparison I can think of is the e2fsck command which has an option "-y" that will answer "yes" to all questions. I believe that this is preferable behaviour than having to use "-yyyyyyy..." to try to guess how many prompts might require.

I agree, this would be cleaner. I will add this option in future versions, keeping -f unchanged.
Quote:
Is is possible for the ntfsresize to know if the partition is truly "dirty" or simply marked "dirty" by ntfsresize?

No. "dirty" is a flag meaning "suspicious", with no field telling what is suspicious, and it is a strong option in ntfs-3g not to invent new fields or new meanings for a field.

Actually my question meant : "if a partition is dirty, and the -f option is set, is it correct to not warn the user ?". In the current implementation (as shown in your initial post), the -f prevents both from getting a warning and from aborting. Changing this behavior could cause problems to programs like gparted which analyze the output of ntfsresize.

I could consider a partition being dirty as just a warning, and option -y (or both option -y and -f being set) to mean "do not ask questions and do not stop at warnings", with no disturbance to existing programs setting -f only.

Regards

Jean-Pierre


Sat Jul 30, 2011 09:06
Profile

Joined: Thu Jul 28, 2011 17:55
Posts: 6
Post Re: ntfsresize --force option is ignored
My intention of using the example of e2fsck and the -y option was to simply state that multiple flags (e.g., -f -f -f -f) is not desirable. I did not intend that a new "-y" option was required.


Quote:
Actually my question meant : "if a partition is dirty, and the -f option is set, is it correct to not warn the user ?". In the current implementation (as shown in your initial post), the -f prevents both from getting a warning and from aborting. Changing this behavior could cause problems to programs like gparted which analyze the output of ntfsresize.


From my perspective, if a user specifies the "-f" option, it means that they wish to proceed even if the partition is marked as dirty. When using the command line I think a user would not initially specify "-f". The "-f" would be specified only if the user saw from the ntfsresize output that the file system was dirty, but the user wanted to proceed anyway. Then the user would add the "-f" to the command.

From a GParted perspective, we prefer it if APIs and command line options and behaviour do not change. Of course this is an ideal perspective and we do realize that sometimes things must be changed to allow for future improvements.

To summarize, my preference would be to have the "-f" option proceed and not prompt the user to see if they wish to continue when a dirty file system is discovered. This would be consistent with the manual page entry:
Quote:
-f, --force
Forces ntfsresize to proceed with the resize operation even if
the filesystem is marked for consistency check.


Sun Jul 31, 2011 18:20
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: ntfsresize --force option is ignored
Hi,

Quote:
My intention of using the example of e2fsck and the -y option was to simply state that multiple flags (e.g., -f -f -f -f) is not desirable. I did not intend that a new "-y" option was required.

I understood you disagree with repeated use of -f, but this is how ntfsresize has been since at least ntfsprogs.1.27. I also agree this old behavior is difficult to understand and difficult to explain (must be the reason why it was not documented). I felt the -y was a way to not change the behavior, though satisfying the actual need.
Quote:
From a GParted perspective, we prefer it if APIs and command line options and behaviour do not change. Of course this is an ideal perspective and we do realize that sometimes things must be changed to allow for future improvements.

I agree with this, but what you have experienced is a change in processing errors when ntfsresize tries to get a reply. I feel unsafe when getting an error for an important question means to proceed, but if there is a need, I will reverse it.
Quote:
To summarize, my preference would be to have the "-f" option proceed and not prompt the user to see if they wish to continue when a dirty file system is discovered. This would be consistent with the manual page entry:

The current (and old) behavior is :
Code:
no -f and clean : ask whether to proceed
no -f and dirty : warn and ask whether to proceed
-f and clean : proceed with no question
-f and dirty : do not warn, but ask whether to proceed
-ff and dirty : do not warn and do not ask

You would prefer :
Code:
no -f and clean : do not ask (changed)
no -f and dirty : warn and ask
-f and clean : do not ask (changed)
-f and dirty : do not ask(changed)

I think I will have fewer reports reverting to the old behavior of an errored reply to mean proceed (and fixing the documentation accordingly).

Regards

Jean-Pierre


Sun Jul 31, 2011 21:36
Profile

Joined: Thu Jul 28, 2011 17:55
Posts: 6
Post Re: ntfsresize --force option is ignored
Your summary of the situation is perfect Jean-Pierre.

You stated in two short lists what used to occur and what I was trying to describe. I believe we are on the same page of understanding. :-)

In the end, mine is only an opinion on how to proceed. Since you are maintaining the software, yours is the opinion that matters most.

My preference is not to have to change the GParted code, but I certainly will if need be.

My concerns with the options to proceed are as follows:

A) If the ntfsresize behaviour is reverted back to how it used to work, then I do not need to change any code in GParted. This is the easy, and lazy option for me. :)

B) If the ntfsresize behaviour is kept as it is in the latest code, then I can work around the problem by adding an extra "-f" option to the code in GParted. This is easy to do and should also work with older versions of ntfsresize. This is also a relatively easy GParted code update.

C) If the ntfsresize behaviour is kept as it is in the latest code and a new "-y" option is added, then I have much more work to do. This is because old versions of ntfsresize will probably balk at the new "-y" option. In such a situation I will need to write code to determine which version of ntfsresize is in use, and then apply or not apply the "-y" option based on version.

These above three situations are from my perspective as a maintainer of GParted.

From your perspective of ntfsprogs maintainer, I think it might be best to do what you believe is the best solution for command line users of ntfsresize, and makes the code more easily maintainable.


Wed Aug 03, 2011 17:11
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: ntfsresize --force option is ignored
Hi Curtis,
Quote:
A) If the ntfsresize behaviour is reverted back to how it used to work, then I do not need to change any code in GParted. This is the easy, and lazy option for me. :)

This is not the easiest for me, because I have to explain clearly in the manual what is whimsical. But ok, I will do that to keep the users silent.
Quote:
B) If the ntfsresize behaviour is kept as it is in the latest code, then I can work around the problem by adding an extra "-f" option to the code in GParted. This is easy to do and should also work with older versions of ntfsresize. This is also a relatively easy GParted code update.

Well, though I am reverting back to the old behavior, you should add -ff instead of relying on an unspecified behavior when ntfsresize gets an error when trying to read the reply. I suppose you do not connect the stdin of ntfsresize, hence the error. With -ff no question is asked (I stress : even with the old ntfsresize).
Quote:
C) If the ntfsresize behaviour is kept as it is in the latest code and a new "-y" option is added, then I have much more work to do.

I drop this proposal for now.

Regards

Jean-Pierre


Wed Aug 03, 2011 17:41
Profile

Joined: Thu Jul 28, 2011 17:55
Posts: 6
Post Re: ntfsresize --force option is ignored
Quote:
Well, though I am reverting back to the old behavior, you should add -ff instead of relying on an unspecified behavior when ntfsresize gets an error when trying to read the reply. I suppose you do not connect the stdin of ntfsresize, hence the error. With -ff no question is asked (I stress : even with the old ntfsresize).


Thanks for this suggestion and your prompt handling of this behaviour report. I plan to add your code suggestion to the GParted code.


Mon Aug 08, 2011 01:20
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: ntfsresize --force option is ignored
Hi,

Quote:
A) If the ntfsresize behaviour is reverted back to how it used to work, then I do not need to change any code in GParted. This is the easy, and lazy option for me. :)

Done in ntfs-3g_ntfsprogs-2011.4.12AR.6, however to prevent you from getting lazy, I have added a new option to expand a partition downwards.

Regards

Jean-Pierre


Mon Aug 08, 2011 08:34
Profile

Joined: Thu Jul 28, 2011 17:55
Posts: 6
Post Re: ntfsresize --force option is ignored
Thank you Jean-Pierre for reverting the behaviour of ntfsresize.

Based on your advice, I have also added a second "--force" option to the NTFS file system resize code in GParted.

With winter coming I should be able to spend more time on coding, and hopefully have some time to investigate the new option you added to expand an NTFS partition downwards.

Again, thank you for all your help on this problem. :-)

Sincerely,
Curtis Gedak


Tue Sep 13, 2011 19:31
Profile

Joined: Tue May 17, 2011 21:33
Posts: 5
Post Re: ntfsresize --force option is ignored
Not emitting an interactive prompt when stdin is connected to /dev/null is an old and basic unix philosophy. You should be checking isatty(stdin) and if it isn't, you either print a warning and proceed, or print an error and exit ( to stderr ). You don't print the "Proceed [y/N]" prompt, since there is no human there who can answer. Since the man page says -f should make it proceed even when the dirty flag is set, and people have come to use it like that, and it did seem to work that way before, then there is a bug in the code. The fact that the bug has existed for ages and only recently been triggered doesn't change the fact that it is a bug.


Fri Mar 16, 2012 17:16
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: ntfsresize --force option is ignored
Hi,

Quote:
The fact that the bug has existed for ages and only recently been triggered doesn't change the fact that it is a bug.

I agree, there is a bad design which went unnoticed for years. The current situation is back to the original one, with the manual trying to describe how -f has been implemented.

Now, we have to fix it without breaking something else, and this thread shows the dangers. Testing isatty(stdin) may be the way to go, but there may be GUI encapsulations in which clicking on an OK button pipes a "yes" reply to the stdin. So a new option ("unattended" ?) would be needed to control the isatty() test, adding more crap.

Regards

Jean-Pierre


Fri Mar 16, 2012 20:22
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 13 posts ] 


Who is online

Users browsing this forum: Google [Bot] and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.