FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Fri May 14, 2021 15:48



Post new topic Reply to topic  [ 8 posts ] 
"?" in Windows filenames 
Author Message

Joined: Sun Jan 13, 2008 02:11
Posts: 1
Post "?" in Windows filenames
1. I've mounted my windows partition under Linux and copied some files on it (working on Linux). When using Windows I couldn't either remove or rename files which had characters like "?" or ":" in their filenames. I had to switch back to Linux, find them and rename them "by hand". How can I avoid those type of problems? (any mount flag or sth?)
2. Can anyone give me a link to the documentation pages, because I just can't find it?


Sun Jan 13, 2008 03:00
Profile
Tuxera CTO

Joined: Tue Nov 21, 2006 23:15
Posts: 1648
Post 
Documented at http://ntfs-3g.org/support.html#posixfilenames2


Thu Jan 17, 2008 01:10
Profile

Joined: Wed Mar 12, 2008 02:26
Posts: 5
Post 


This is an insufficient response to this problem. The NT kernel ntfs driver cannot access files with '?' in their name, even through the POSIX subsystem. If the purpose of NTFS-3G is to increase compatibility with NT at the filesystem level, then it might be a good idea to actually increase compatibility with NT at the filesystem level by, you know, actually making it more compatible: adding a mount option to disallow creating incompatible files would further this goal.

If, on the other hand, the purpose is to create a filesystem not directly compatible with NT, but instead inspired by Microsoft's NTFS file system, perhaps it might be a good idea to rename the project. Describing it as NTFS when it is not in fact compatible with the NT ntfs implementation is misleading, to say the least.


Wed Mar 12, 2008 02:35
Profile

Joined: Fri Oct 19, 2007 21:29
Posts: 8
Post Out of bound, out of subject
NTFS is directly inspired from IBM's HPFS, but greatly improved. From this early setup, NTFS has been designed to store metadata from several systems, among which UNIX and NT.

As it stands, when mounted under GNU/Linux, BSD and Mac OS, the volume is mounted under a UNIX or UNIX-like system, with its own rules: some characters are allowed in one, others are not (please note for example, that NTFS can store file case; UNIX user space file access is case discriminant, Windows isn't by default).

Windows can access UNIX file structures; the kernel understands them and the native NTFS driver does, too. However, user space utilities like Windows Explorer and the command line can't make head nor tail from a UNIX file system (you will notice that Windows Explorer will list all files, even 'illegal' ones, and chkdsk won't raise a fuss).

In short: NTFS-3G is a mostly complete user space NTFS driver for UNIX systems. It accesses NTFS like a UNIX system would. For improved compatibility, due to the fact that UNIX has a more fine grained file system access, NTFS-3G allows use of UNIX capabilities on NTFS volumes.

Recently, an ACL to uid/gid translator has been added.

It is your responsibility as a user to make sure that your files can be read from one system to another if you use a mobile device. For maximum compatibility, you should install Microsoft Services for UNIX on your Windows machine to access such devices. If you prefer a reduced compatibility set, mount your file system through a Samba share (it can be done locally), which will cripple down your access to your NTFS file system without too much overhead.

As it stands, it's not NTFS-3G's fault if Windows can't make use of all the features contained in its main partition format.


Fri Mar 14, 2008 02:55
Profile

Joined: Wed Mar 12, 2008 02:26
Posts: 5
Post Re: Out of bound, out of subject
mitch074 wrote:
NTFS is directly inspired from IBM's HPFS, but greatly improved. From this early setup, NTFS has been designed to store metadata from several systems, among which UNIX and NT.


I'm more than well aware of NTFS's history and capabilities.

Quote:
Windows can access UNIX file structures; the kernel understands them and the native NTFS driver does, too. However, user space utilities like Windows Explorer and the command line can't make head nor tail from a UNIX file system (you will notice that Windows Explorer will list all files, even 'illegal' ones, and chkdsk won't raise a fuss).


I've contacted some relevant NTFS developers at Microsoft, and a bug has been entered against CHKDSK for this problem ('?' in filename). NTFS.sys (the kernel driver for the file system) cannot handle files containing character 0x3F, even when talked to directly (i.e. not including any subsystem, Win32, POSIX or otherwise) - confirmed by an NTFS dev.

Quote:
In short: NTFS-3G is a mostly complete user space NTFS driver for UNIX systems.


I agree, it is the most complete, however it can cause disk space loss from a Windows perspective by creating undeletable files.

Quote:
For maximum compatibility, you should install Microsoft Services for UNIX on your Windows machine to access such devices.


Indeed I did install SFU to try and fix my problems, but the POSIX subsystem translates characters like 0x3F by ORing them with 0xF000 to shift them into a private area of Unicode, thereby both complying with POSIX at the interface level and avoiding problems with Windows. Unfortunately, I've been unable to convince the NTFS-3G maintainers to do like Windows does, so the FS corruption will continue for the time being.


Quote:
If you prefer a reduced compatibility set, mount your file system through a Samba share (it can be done locally), which will cripple down your access to your NTFS file system without too much overhead.


Actually, I've created a patch for the FS userspace driver that performs transformations similar to the Windows POSIX subsystem when translating filenames to and from the NT UTF-16LE encoding. That'll serve my purposes pretty well for now, without compromising other features.

Quote:
As it stands, it's not NTFS-3G's fault if Windows can't make use of all the features contained in its main partition format.


It's also not Windows' fault if NTFS-3G has some bugs. Caveat emptor!

-- Barry


Fri Mar 14, 2008 19:10
Profile

Joined: Sat May 29, 2010 14:28
Posts: 2
Post Re: "?" in Windows filenames
Hi. Thanks for the work of many on NTFS-3G, however I must agree that the inclusion of "special characters" when writing to NTFS volumes is quite problematic for those who do move back and forth between *nix and Windows.

Could you please include a switch in NTFS-3G to allow a workaround for this Windows limitation? (i.e. to turn off use of special characters in order to allow compatibility with Windows for the many users who wouldn't have a clue how to re-mount/export their NTFS filesystems using Samba). I believe that implementing this as a switch will not hamper those who want full POSIX namespace support, but it will allow for greater interoperability with Windows for those who wish for that.

Please note the bug report found here for Ubuntu's Rhytmbox, which seems to demonstrate a good case of this issue.
https://bugs.launchpad.net/ubuntu/+source/rhythmbox/+bug/318625?comments=all

I believe that my reasoning in that bug report lines up very closely with that of barrkel and many others.
Quote:
Isn't the whole idea of NTFS-3G to allow proper and full compatibility with the *Windows* implementation of NTFS? Isn't it to allow true interoperability? Thus, given that Windows considers certain characters to be illegal should mean that they are treated as illegal for NTFS drivers on other platforms, purely based on the premise that it is about full interoperability.

Yes, the fact that Windows doesn't like certain characters in its implementation of NTFS is likely an aberration from the POSIX namespace guidelines, but then it is the *reference implementation*.

Additionally, consider that there are many exceptions to rules within the Win32 API. The WINE project is constantly working around exceptions that are undocumented and/or that go against existing documentation on aspects of the API.

Making workarounds in every little app is redundant and counter productive when it could and, in my view, should be solved by making this simple change in NTFS-3G.

Also...
Quote:
A follow-up on my last comment.

Please note that Wine has a "bug-for-bug" policy when it comes to working with the windows API.
See http://wiki.winehq.org/WineFeatures

In other words, if the actual implementation disagrees with the documentation, then the implementation wins. There is a lot of sense to this policy, as the end goal is full interoperability, or "Binary Compatibility" as mentioned on the page I have referenced.

This should be the same with the NTFS-3G driver. If the NTFS documentation says full POSIX namespace support, but the NTFS implementation differs, the implementation should win. In our case, there are characters that are legal in the documentation, even if it is not Microsoft's documentation, but every known official implementation of NTFS does not allow full POSIX namespace support.

For the sake of full interoperability, this bug should not be fixed in individual apps, but in the NTFS-3G code, just like similar issues are dealt with on a "bug-for-bug" basis in the Wine code.

See here for many other people who think that this situation is problematic;
http://ubuntuforums.org/archive/index.php/t-1253837.html
http://superuser.com/questions/90227/files-with-illegal-filenames
https://bugs.launchpad.net/ubuntu/+bug/230906
http://superuser.com/questions/31587/how-to-force-windows-xp-to-rename-a-file-with-a-special-character

See also;
http://support.microsoft.com/kb/320081
http://support.microsoft.com/kb/120716/
These both suggest using a POSIX tool to delete or rename files with special characters, which are not available as standard in any Windows version. This in itself shows that there is a serious usability issue here.
http://superuser.com/questions/49679/how-to-batch-rename-files-copied-from-osx-to-windows-with-in-filenames
This suggests using various 3rd party tools to delete or rename, since the default windows tools don't truly support the POSIX namespace.

Please, please, please, could you implement a switch for those who wish to have a hobbled version of NTFS? I believe that would be in the spirit of *nix platforms that aim to not hem people in. Please allow people to choose this restriction if they so desire... :)


Sat May 29, 2010 15:52
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: "?" in Windows filenames
Hi,

Quote:
I must agree that the inclusion of "special characters" when writing to NTFS volumes is quite problematic for those who do move back and forth between *nix and Windows.

Could you please include a switch in NTFS-3G to allow a workaround for this Windows limitation? (i.e. to turn off use of special characters in order to allow compatibility with Windows for the many users who wouldn't have a clue how to re-mount/export their NTFS filesystems using Samba)

This is available in the release candidate advanced ntfs-3g-2010.5.16AR.1, see http://pagesperso-orange.fr/b.andre/adv ... fs-3g.html
The option to use is "windows_names", documented in the manual of the said version.

Regards

Jean-Pierre


Tue Jun 01, 2010 21:31
Profile

Joined: Sat May 29, 2010 14:28
Posts: 2
Post Re: "?" in Windows filenames
Jean-Pierre (and other NTFS-3G team members)

Thank you for your reply. It is greatly appreciated!

The inclusion of this switch in the new version of NTFS-3G is a good step forward, and hopefully including it as a switch won't break anything by default.

Kind Regards,

Rygle


Mon Jun 07, 2010 14:34
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 8 posts ] 


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.