FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Mon Jun 21, 2021 03:46



Post new topic Reply to topic  [ 9 posts ] 
Are special characters allowed in NTFS? 
Author Message

Joined: Sat Jan 19, 2013 15:14
Posts: 4
Post Are special characters allowed in NTFS?
First, excuse me if this is not the right place for my question, but I haven't found a more suitable forum.

By reading the FAQ, I have learned that NTFS-3G allows special characters in file and folder names, such as a backslash or colon. In Windows, these characters are not allowed. Now my question is: Is this a limitation of the general NTFS specification, or is this limitation specific to Windows?


Sat Jan 19, 2013 15:21
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Are special characters allowed in NTFS?
Hi,

Quote:
In Windows, these characters are not allowed. Now my question is: Is this a limitation of the general NTFS specification, or is this limitation specific to Windows?

I cannot answer about the NTFS specification, as I do not have a copy thereof. I can only say there is nothing in the NTFS data layout known to prevent from using any legal Unicode character (even '/' and the null character), but each operating system may impose its own restrictions.

With ntfs-3g, depending on your needs, you can either use Unix rules (two forbidden chars) or the Windows ones (eleven forbidden chars).

Regards

Jean-Pierre


Sat Jan 19, 2013 22:41
Profile

Joined: Sat Jan 19, 2013 15:14
Posts: 4
Post Re: Are special characters allowed in NTFS?
Thanks for your reply. Your info on the different rulesets is very helpful.


Sat Jan 19, 2013 22:45
Profile

Joined: Wed Jan 30, 2013 17:09
Posts: 3
Post Re: Are special characters allowed in NTFS?
I've been trying to back up a linux system to an ntfs disk and a few of my linux files have names that include characters in the 129 to 255 range.

(I've been using rsync but a simple "cp" behaves the same way)

I cannot create any files that contain these characters - I wrote a test program that loops through 128..255 and does a fopen of a file with that character as its name - it creates 128 files OK on a regular linux filesystem, none at all on a mounted ntfs drive, and if i port the program to windows, it creates 94 files successfully. So ntfs/windows supports at least 94 of those values.

How do I create these files on an ntfs-3g mounted drive? Is there a specific mount option I need to give?

thanks

G
PS If you want to explore this yourself here's the trivial test code... create an "in" directory before you run it...
Code:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(int argc, char **argv) {
  int i;
  FILE *f;
  char fname[5];
  strcpy(fname, "in/?");
  for (i = 128; i < 256; i++) {
    fname[3] = i;
    f = fopen(fname, "w");
    if (f == NULL) {
      fprintf(stdout, "%d failed\n", i);
    } else {
      fputc(i, f); fputc('\n', f);
      fclose(f);
    }
  }
  exit(0);
  return(1);
}


Wed Jan 30, 2013 17:21
Profile

Joined: Sat Jan 19, 2013 15:14
Posts: 4
Post Re: Are special characters allowed in NTFS?
How about backing up to a single archive file and then copying this file to the NTFS partition?


Fri Feb 01, 2013 01:40
Profile

Joined: Wed Jan 30, 2013 17:09
Posts: 3
Post Re: Are special characters allowed in NTFS?
geoyo wrote:
How about backing up to a single archive file and then copying this file to the NTFS partition?


I would prefer to find out how to create proper filenames containing these characters. The documentation seems to suggest it should work but it doesn't. The underlying filesystem definitely supports them and windows sort-of supports them, so either linux supports them (in which case I must be missing some mount option?) ... or ... it's a bug/omission in ntfs-3g and should be reported and fixed.

(Aso the file system I'm backing up is about 3Tb and I don't fancy handling a 3Tb zip or tar file.)

Graham


Fri Feb 01, 2013 06:44
Profile

Joined: Sat Jan 19, 2013 15:14
Posts: 4
Post Re: Are special characters allowed in NTFS?
Maybe this will help? http://www.unix.com/unix-advanced-exper ... cipts.html

I'm not a unix/linux expert, though... just did a google search for linux + extended ASCII


Fri Feb 01, 2013 17:32
Profile

Joined: Wed Jan 30, 2013 17:09
Posts: 3
Post Re: Are special characters allowed in NTFS?
geoyo wrote:
Maybe this will help? http://www.unix.com/unix-advanced-exper ... cipts.html

I'm not a unix/linux expert, though... just did a google search for linux + extended ASCII


Unfortunately if it were that simple I wouldn't be posting here asking for help from the ntfs-3g experts.

I spent a lot of time Googling and playing with locales and --iconv on rsync.

The page you pointed at isn't ntfs-3g specific. My problem is - I can handle these filenames just fine on ext3, reiserfs, etc etc. It works, more or less, on an NTFS file system under Windows too. Just not in ntfs-3g on linux.

It's not a shell issue, and it's not an issue of entering or displaying the characters, and I don't want to use 2-byte sequences because the files I'm backing up/mirroring don't use 2-byte sequences, and I don't want to rename the files in either the source or the destination trees.

It *shouldn't* be a locale issue because I just want 8-bit transparency. The characters in these filenames are effectively meaningless.

On the surface it looks like a problem with ntfs-3g, but there's always a chance that I overlooked some option that enables the behaviour I expect. If there is such an option it's likely either an environment variable or a mount option.

If you're interested in seeing this problem for yourself, run the code I posted above on linux in an ext3 or other regular filesystem, and then try to copy the files it creates to a directory on a mounted ntfs-3g filesystem.

I've just now fetched the ntfs-3g source so hopefully I can work this out for myself over the weekend (unless some kind soul knows the answer and can give me back my weekend :-) )

One thing that I didn't think of until just now - the ntfs-3g I have installed on my SuSE system may be fairly old. I've just now rebuilt ntfs-3g from the latest source and when I get hope later today and can power up my ntfs drive (it's in one of those external USB drive caddies) I'll check to see if this problem still happens with the latest version.

regards,

Graham


Fri Feb 01, 2013 18:41
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Are special characters allowed in NTFS?
Hi,

Quote:
It *shouldn't* be a locale issue because I just want 8-bit transparency. The characters in these filenames are effectively meaningless

ntfs stores the file names as Unicode characters encoded as utf16, and ntfs-3g normally requires the names to be defined as valid utf8, so that they can be translated both ways to utf16.
Quote:
I've been trying to back up a linux system to an ntfs disk and a few of my linux files have names that include characters in the 129 to 255 range.

The correct way of doing that is to translate your existing file names to utf8, then to do the backup. You may avoid modifying your existing file system by creating a shell script to copy each file to a target file which has a valid utf8 name.

Now, there still exists an old "locale" mount option for ntfs-3g, so that file names encoded with that locale will be translated to utf16. This option is now undocumented because it has led to complains for invisible files and undeletable directories. Among the issues are the fact that Windows does not use the iso-8859 encodings which Linux generally uses, and the fact that files created by Windows may be untranslatable to the selected locale. So this option is strongly discouraged, but it may be of some help in you case.

Regards

Jean-Pierre


Mon Feb 04, 2013 13:08
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 9 posts ] 


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.