FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Sat May 15, 2021 03:23



Post new topic Reply to topic  [ 8 posts ] 
"Invalid or incomplete multibyte or wide character" Issue 
Author Message

Joined: Fri Jan 30, 2009 13:30
Posts: 2
Post "Invalid or incomplete multibyte or wide character" Issue
Hello,

I'm having problems copying filenames with non-standard characters onto an NTFS partition. I've searched these forums and read FAQs but unfortunately I still receive this error:

$ cp /mnt/nfs/Price£10.txt /data/
cp: cannot stat `/data/Price\24310.txt': Invalid or incomplete multibyte or wide character

My setup is as follows:

Ubuntu 8.10 Desktop - 2.6.27-11-generic - x64

/mnt/nfs - this is a mountpoint to another Linux box using NFS. To mount, I simply used:
Code:
sudo mount 192.168.x.x:/files /mnt/nfs/


/data - this is a separate hard drive in my Linux box that is formatted with NTFS. This is mounted during boot and is configured in /etc/fstab as the following:
Code:
/dev/sdb1 /data ntfs-3g defaults 0 0


I haven't used the locale option because I'm using ntfs-3g 2009.1.1 as this now has "Built-in, transparent UTF-8 conversion support".

Code:
$ sudo ntfs-3g --help

ntfs-3g 2009.1.1 integrated FUSE 27 - Third Generation NTFS Driver


Interestingly, when I map to the /data mountpoint as a share from Vista, I can copy this file into the ntfs volume without any errors, yet doing an ls of this file shows the following:

Code:
$ ls -al
-rwxrwxrwx 1 root root    0 2009-01-30 11:33 Price£10.txt


Another filename called "Siúil-A-Rún.txt" appears as the following if copied using a Vista share:

Code:
-rwxrwxrwx 1 root root 5280 2005-11-16 20:32 Siúil-A-Rún.txt


My Ubuntu locale settings are:

Code:
$ sudo locale
LANG=en_GB.UTF-8
LC_CTYPE="en_GB.UTF-8"
LC_NUMERIC="en_GB.UTF-8"
LC_TIME="en_GB.UTF-8"
LC_COLLATE="en_GB.UTF-8"
LC_MONETARY="en_GB.UTF-8"
LC_MESSAGES="en_GB.UTF-8"
LC_PAPER="en_GB.UTF-8"
LC_NAME="en_GB.UTF-8"
LC_ADDRESS="en_GB.UTF-8"
LC_TELEPHONE="en_GB.UTF-8"
LC_MEASUREMENT="en_GB.UTF-8"
LC_IDENTIFICATION="en_GB.UTF-8"
LC_ALL=


Any help you can offer would be great.

Many thanks,
Paul.


Fri Jan 30, 2009 14:02
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: "Invalid or incomplete multibyte or wide character" Issue
Hi Paul,

Quote:
$ cp /mnt/nfs/Price£10.txt /data/
cp: cannot stat `/data/Price\24310.txt': Invalid or incomplete multibyte or wide character

True.

Your file names are supposed to be UTF8 encoded and "\243" is not a correct UTF8 char. The correct encoding for "£" is "\302\243" as you can display in "ls -l", so the file name was correctly translated.

Assuming you typed the file name in the cp example, you just have to configure your terminal to used UTF8.

Regards

Jean-Pierre


Fri Jan 30, 2009 15:40
Profile

Joined: Fri Jan 30, 2009 13:30
Posts: 2
Post Re: "Invalid or incomplete multibyte or wide character" Issue
Hi jpa, thanks for your help.

Indeed this has helped, I'm using Putty and the Character Set Translation was set to ISO-8859-1 (Latin-1 West Europe). Chaging this to UTF-8 helped show the correct characters and prevent the error from occuring.

Many thanks,
Paul.


Mon Feb 02, 2009 12:41
Profile

Joined: Wed May 27, 2009 07:13
Posts: 2
Post Re: "Invalid or incomplete multibyte or wide character" Issue
Hello,


I have nearly the same problem, running a backup script from a cron-job:
cp: cannot create directory `/mnt/usb/backup-27-05-2009/daten/dosapp/Startmen\374': Invalid or incomplete multibyte or wide character


I'm stuck at telling my cron script to use the correct encoding.


Wed May 27, 2009 07:16
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: "Invalid or incomplete multibyte or wide character" Issue
Hi,

Your backup directory name is generated dynamically, probably from a pattern hard coded in some shell script.
Try grepping for "Startmen" to locate and fix the pattern.
If it is generated by a binary program, you may have to dynamically translate its output.

Regards

Jean-Pierre


Wed May 27, 2009 09:59
Profile

Joined: Wed May 27, 2009 07:13
Posts: 2
Post Re: "Invalid or incomplete multibyte or wide character" Issue
adding

Code:
export LANG=de_DE@euro

and changing my mount line to

Code:
ntfs-3g /dev/sda1 /mnt/usb -o locale=de_DE@euro

in my script seems to have solved the problems!


Thu May 28, 2009 07:06
Profile

Joined: Wed Feb 02, 2011 04:58
Posts: 1
Post Re: "Invalid or incomplete multibyte or wide character" Issue
I have recently purchased an Iomega Black Belt external hard drive which is formatted with NTFS. I've been trying to copy FLAC files off my Fedora Core box onto the drive via ntfs-3g to no avail:

[root@localhost ~]# cp /var/tmp/Dr.\ Dog\ -\ 05\ -\ Where\'d\ All\ the\ Time\ Go¦.flac /media/Iomega_HDD_/
cp: cannot stat `/media/Iomega_HDD_/Dr. Dog - 05 - Where\'d All the Time Go\277.flac': Invalid or incomplete multibyte or wide character

I know this has to with the file name and Unicode support. Even after changing my Putty terminal to use UTF-8, the same error is encountered.

System:

[root@localhost ~]# cat /etc/redhat-release
Fedora release 11 (Leonidas)

[root@localhost ~]# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Mounting options located in syslog via hot plug:

Feb 1 23:39:31 localhost ntfs-3g[2613]: Version 2010.5.16 integrated FUSE 27
Feb 1 23:39:31 localhost ntfs-3g[2613]: Mounted /dev/sdb1 (Read-Write, label "Iomega_HDD", NTFS 3.1)
Feb 1 23:39:31 localhost ntfs-3g[2613]: Cmdline options: rw,nosuid,nodev,uhelper=devkit,uid=500,gid=500,dmask=0077
Feb 1 23:39:31 localhost ntfs-3g[2613]: Mount options: rw,nosuid,nodev,uhelper=devkit,silent,allow_other,nonempty,relatime,fsname=/dev/sdb1,blkdev,blksize=4096,default_permissions
Feb 1 23:39:31 localhost ntfs-3g[2613]: Global ownership and permissions enforced, configuration type 1

Errror in syslog:

Feb 1 23:46:31 localhost ntfs-3g[2613]: Could not convert filename to Unicode: 'Dr. Dog - 05 - Where'd All the Time Go¦.flac': Invalid or incomplete multibyte or wide character

I haven't really messed with /etc/fstab as that file typically isn't used for external drives (or is it?).

Please assist as I'm really at wit's end but would prefer to not format to FAT32.

-slainte


Wed Feb 02, 2011 09:21
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: "Invalid or incomplete multibyte or wide character" Issue
Hi,

Quote:
[root@localhost ~]# cp /var/tmp/Dr.\ Dog\ -\ 05\ -\ Where\'d\ All\ the\ Time\ Go¦.flac /media/Iomega_HDD_/
cp: cannot stat `/media/Iomega_HDD_/Dr. Dog - 05 - Where\'d All the Time Go\277.flac': Invalid or incomplete multibyte or wide character

Your file name must have an illegal utf-8 character. I see (not reliably because of cut-and-paste and possible translations in the process) the sequence '¦.' where '¦' may be 0xa6, the beginning of a two-byte character whose second character cannot be '.' (after 0xa6 you must have a byte in range 0x80-0x9f)
Not being a valid character, it cannot be interpreted and translated to utf-16 as required by ntfs.

To be sure of the diagnosis, please post the file name as hexadecimal, as in
Code:
[linux@pavilion2 ~]$ echo 'Go¦.flac' | od -t x1
0000000 47 6f a6 2e 66 6c 61 63 0a
0000011

Quote:
I know this has to with the file name and Unicode support. Even after changing my Putty terminal to use UTF-8, the same error is encountered.

If you access through putty, create the file on ext3 in order to display its name after it has been translated by putty :
Code:
[linux@pavilion2 ~]$ ls Go* | od -t x1
0000000 47 6f a6 2e 66 6c 61 63 0a
0000011


Regards

Jean-Pierre

*edit*

I oversaw the error message from cp, in which the said character is returned as \277 this means the character 0xaf, not 0xa6, which confirms the diagnosis because the following character, if it were a legal one, would also have been displayed in octal.


Wed Feb 02, 2011 09:49
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 8 posts ] 


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.