FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Thu May 13, 2021 21:43



Post new topic Reply to topic  [ 79 posts ]  Go to page Previous  1, 2, 3, 4  Next
Operation not supported (45) - ACL problem 
Author Message

Joined: Mon Sep 13, 2010 18:42
Posts: 42
Post Re: Operation not supported (45) - ACL problem
Hi Jean-Pierre,

Thanks for the responses. Yes, it turns out that her account is in the "Administrators" group. I think that this would often be the case with many home/small-business PC users so perhaps this somewhat mitigates the issue of not being able to set the SID permissions in a small Windows environment. In a larger Windows environment I think it's reasonable to expect that some kind of domain server will be available from where you can potentially query SIDs using something like winbind if using Linux. I would expect that it is probably also possible to run commands directly on the Windows domain server to extract a similar output of the SIDs.

jpa wrote:
I have analyzed the consequences of options fmode=004 and dmode=005 to define protections to be "or-ed" in chmod and setxattr on files and directories. This is apparently doable and clean (several days needed). The consequence is you will have to do chmod's in your restore script on Linux, but the backup files will be readable on any Windows computer.

I think this is a useful feature to have to cover any similar usecases where other for some reason the user doesn't have admin rights on Windows.

jpa wrote:
There is no gcc cross-compiler for sparc on a standard PC ?

It might be possible to use Qemu which would have the additional benefit of also providing a real testing/debugging environment:
http://readynas.com/forum/viewtopic.php?f=35&t=30963
although reading the posts in that thread, it doesn't sound like it's easy to get working. I might give it a go if I feel strong enough!

In the meantime, as you suggest, let's wait for a compiled from Netgear version.

Regards,

Mark


Thu Oct 21, 2010 20:32
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Operation not supported (45) - ACL problem
Hi Mark,

Quote:
Thanks for the responses. Yes, it turns out that her account is in the "Administrators" group. I think that this would often be the case with many home/small-business PC users so perhaps this somewhat mitigates the issue of not being able to set the SID permissions in a small Windows environment.

That was an unexpected conclusion in an environment with strict permission controls.
Quote:
In a larger Windows environment I think it's reasonable to expect that some kind of domain server will be available from where you can potentially query SIDs using something like winbind if using Linux. I would expect that it is probably also possible to run commands directly on the Windows domain server to extract a similar output of the SIDs.

Sure. Having unified SIDs eliminates the major problem you were facing.
Quote:
I think this is a useful feature to have to cover any similar usecases where other for some reason the user doesn't have admin rights on Windows.

But this comes at a lower priority now.
Quote:
It might be possible to use Qemu which would have the additional benefit of also providing a real testing/debugging environment:

I think qemu emulates the sparc processor, so you have to copy the full OS image. Just installing the compiler might be easier (see http://www.cis.upenn.edu/~milom/cross-compile.html ). Being in user space, compiling ntfs-3g does not require the kernel headers to be available.

Regards

Jean-Pierre


Fri Oct 22, 2010 09:13
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Operation not supported (45) - ACL problem
Hi,

I have installed a cross compiler for sparc-linux. Before I try to build ntfs-3g, can you check the "hello world" executable attached ?
It would also be useful to know which is the glibc version installed on your NAS.

Regards

Jean-Pierre


Attachments:
hello.gz [1.35 KiB]
Downloaded 837 times
Fri Oct 22, 2010 21:03
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Operation not supported (45) - ACL problem
Hi Mark,

I have uploaded an ntfs-3g version for Sparc-linux to http://b.andre.pagesperso-orange.fr/spa ... .2AR.1.zip
Unpack it anywhere, no need to install, start it by
Code:
sparc-ntfs-3g -o <options> <device> <mountpoint>

I am not sure about the glibc version you have, if they are not compatible, I will statically link the one I have (glibc-2.4-4)

Of course this is fully untested. Better first try the read-only mode.

*edit*

If the above fails, please retry with http://b.andre.pagesperso-orange.fr/spa ... aurora.zip
This one uses glibc-2.3.3 but has no requirement beyond glibc-2.3.2. This is the same requirement as the package which Netgear posted on their forum.

Still untested, be careful.

Have a nice week-end

Regards

Jean-Pierre


Sat Oct 23, 2010 09:53
Profile

Joined: Mon Sep 13, 2010 18:42
Posts: 42
Post Re: Operation not supported (45) - ACL problem
Hi Jean-Pierre,

Sorry, I was away for a couple of days. The first version failed due to a missing GLIBC dependency as soon as I tried to mount. The second version mounted successfully, but failed but with a new error during the cp:
Code:
# cp -p test /USB_HDD_5/marktest/
cp: closing `/USB_HDD_5/marktest/test': Socket operation on non-socket


From the debug log it looks like it failed at FLUSH and SETXATTR isn't even called:
Code:
WARNING: blksize option is ignored because ntfs-3g must calculate it.
utime_omit_ok: 0
Version 2010.10.2AR.1 integrated FUSE 28
Mounted /dev/sda1 (Read-Write, label "OneTouch4", NTFS 3.1)
Cmdline options: debug,acl,user_xattr,rw,sync,noatime,allow_other,blksize=4096
Mount options: acl,rw,sync,allow_other,allow_other,nonempty,noatime,fsname=/dev/
sda1,blkdev,blksize=4096
User mapping built, Posix ACLs in use, configuration type 7
unique: 1, opcode: INIT (26), nodeid: 0, insize: 56
INIT: 7.8
flags=0x00000003
max_readahead=0x00020000
   INIT: 7.8
   flags=0x00000041
   max_readahead=0x00020000
   max_write=0x00020000
   unique: 1, error: 0 (Success), outsize: 40
unique: 2, opcode: LOOKUP (1), nodeid: 1, insize: 49
LOOKUP /marktest
   NODEID: 2
   unique: 2, error: 0 (Success), outsize: 136
unique: 3, opcode: GETATTR (3), nodeid: 2, insize: 40
   unique: 3, error: 0 (Success), outsize: 112
unique: 4, opcode: GETATTR (3), nodeid: 2, insize: 40
   unique: 4, error: 0 (Success), outsize: 112
unique: 5, opcode: LOOKUP (1), nodeid: 2, insize: 45
LOOKUP /marktest/test
   NODEID: 3
   unique: 5, error: 0 (Success), outsize: 136
unique: 6, opcode: GETATTR (3), nodeid: 3, insize: 40
   unique: 6, error: 0 (Success), outsize: 112
unique: 7, opcode: SETATTR (4), nodeid: 3, insize: 128
   unique: 7, error: 0 (Success), outsize: 112
unique: 8, opcode: OPEN (14), nodeid: 3, insize: 48
   unique: 8, error: 0 (Success), outsize: 32
OPEN[0] flags: 0x40001 /marktest/test
unique: 9, opcode: GETATTR (3), nodeid: 3, insize: 40
   unique: 9, error: 0 (Success), outsize: 112
unique: 10, opcode: FLUSH (25), nodeid: 3, insize: 64
FLUSH[0]
   unique: 10, error: -38 (Socket operation on non-socket), outsize: 16
unique: 11, opcode: RELEASE (18), nodeid: 3, insize: 64
RELEASE[0] flags: 0x40001
   unique: 11, error: 0 (Success), outsize: 16
unique: 12, opcode: FORGET (2), nodeid: 1, insize: 48
FORGET 1/1
unique: 13, opcode: DESTROY (38), nodeid: 0, insize: 40
Unmounting /dev/sda1 (OneTouch4)
   unique: 13, error: 0 (Success), outsize: 16


Regards,

Mark


Sun Oct 24, 2010 21:03
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Operation not supported (45) - ACL problem
Hi Mark,

Quote:
The first version failed due to a missing GLIBC dependency as soon as I tried to mount.

I suspected this would be the case.
Quote:
The second version mounted successfully,

Good news.
Quote:
but failed but with a new error during the cp:
cp: closing `/USB_HDD_5/marktest/test': Socket operation on non-socket
From the debug log it looks like it failed at FLUSH and SETXATTR isn't even called:

Hmm. Not seen before.
Probably a bug in the sync mount option (new code). Try without the option.

Regards

Jean-Pierre


Sun Oct 24, 2010 21:17
Profile

Joined: Mon Sep 13, 2010 18:42
Posts: 42
Post Re: Operation not supported (45) - ACL problem
Good news! The developer who previously helped to build a version for me got his dev environment work and built another version for me which works. I guess there must have been some issue with the cross-compilation previously.

Code:
# cp -p test /USB_HDD_5/marktest
(No error!!)

Also, not only does ls work, but getfacl now also works:
Code:
# ls -l /USB_HDD_5/marktest/test
-rw-rw-r--    1 root     root            0 Oct 24 20:13 /USB_HDD_5/marktest/test

# getfacl /USB_HDD_5/marktest/test
getfacl: Removing leading '/' from absolute path names
# file: USB_HDD_5/marktest/test
# owner: root
# group: root
user::rw-
user:mark:rw-
group::r--
mask::rw-
other::r--


Here's the debug log:
Code:
WARNING: blksize option is ignored because ntfs-3g must calculate it.
utime_omit_ok: 0
Version 2010.10.2AR.1 integrated FUSE 28
Mounted /dev/sda1 (Read-Write, label "OneTouch4", NTFS 3.1)
Cmdline options: debug,acl,user_xattr,rw,sync,noatime,allow_other,blksize=4096
Mount options: acl,rw,sync,allow_other,allow_other,nonempty,noatime,fsname=/dev/
sda1,blkdev,blksize=4096
User mapping built, Posix ACLs in use, configuration type 7
unique: 1, opcode: INIT (26), nodeid: 0, insize: 56
INIT: 7.8
flags=0x00000003
max_readahead=0x00020000
   INIT: 7.8
   flags=0x00000041
   max_readahead=0x00020000
   max_write=0x00020000
   unique: 1, error: 0 (Success), outsize: 40
unique: 2, opcode: LOOKUP (1), nodeid: 1, insize: 49
LOOKUP /marktest
   NODEID: 2
   unique: 2, error: 0 (Success), outsize: 136
unique: 3, opcode: GETATTR (3), nodeid: 2, insize: 40
   unique: 3, error: 0 (Success), outsize: 112
unique: 4, opcode: GETATTR (3), nodeid: 2, insize: 40
   unique: 4, error: 0 (Success), outsize: 112
unique: 5, opcode: LOOKUP (1), nodeid: 2, insize: 45
LOOKUP /marktest/test
   NODEID: 3
   unique: 5, error: 0 (Success), outsize: 136
unique: 6, opcode: GETATTR (3), nodeid: 3, insize: 40
   unique: 6, error: 0 (Success), outsize: 112
unique: 7, opcode: SETATTR (4), nodeid: 3, insize: 128
   unique: 7, error: 0 (Success), outsize: 112
unique: 8, opcode: OPEN (14), nodeid: 3, insize: 48
   unique: 8, error: 0 (Success), outsize: 32
OPEN[0] flags: 0x40001 /marktest/test
unique: 9, opcode: GETATTR (3), nodeid: 3, insize: 40
   unique: 9, error: 0 (Success), outsize: 112
unique: 10, opcode: FLUSH (25), nodeid: 3, insize: 64
FLUSH[0]
   unique: 10, error: -90 (Function not implemented), outsize: 16
unique: 11, opcode: RELEASE (18), nodeid: 3, insize: 64
RELEASE[0] flags: 0x40001
   unique: 11, error: 0 (Success), outsize: 16
unique: 12, opcode: SETATTR (4), nodeid: 3, insize: 128
   unique: 12, error: 0 (Success), outsize: 112
unique: 13, opcode: SETXATTR (21), nodeid: 3, insize: 116
   unique: 13, error: 0 (Success), outsize: 16
unique: 14, opcode: STATFS (17), nodeid: 0, insize: 40
   unique: 14, error: 0 (Success), outsize: 96
unique: 15, opcode: GETATTR (3), nodeid: 1, insize: 40
   unique: 15, error: 0 (Success), outsize: 112
unique: 16, opcode: GETATTR (3), nodeid: 1, insize: 40
   unique: 16, error: 0 (Success), outsize: 112


Sun Oct 24, 2010 21:19
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Operation not supported (45) - ACL problem
Hi again,

Quote:
Also, not only does ls work, but getfacl now also works:

A long awaited one....

Code:
# cp -p test /USB_HDD_5/marktest
(No error!!)

This is much strange, because the log says :
Code:
unique: 10, opcode: FLUSH (25), nodeid: 3, insize: 64
FLUSH[0]
   unique: 10, error: -90 (Function not implemented), outsize: 16

And both builds are using the same code 2010.10.2AR.1 The above error is issue by the fuse library code, I would not suspect the compiler (rather the old fuse kernel, the sync also done when unmounting, and this is known to be wrong).

Regards

Jean-Pierre


Sun Oct 24, 2010 21:38
Profile

Joined: Mon Sep 13, 2010 18:42
Posts: 42
Post Re: Operation not supported (45) - ACL problem
Looking at the debug log it seems that FLUSH is still not implemented. Even so, it looks like this version could be a workable solution. Thanks very much for your help and persistence which got us to this point.

A few questions to tie up the loose ends:
1. Is it possible to fix FLUSH as well (for completeness)?
2. When do you anticipate being able to provide a stable (non-advanced release) version that incorporates these fixes?
3. Since the mount options for NTFS-3G are not supplied via /etc/fstab, which file are they provided in? (I will need to ask Netgear to modify this file)
4. Should I ask for the following options? "user_xattr,rw,sync,noatime,allow_other"
5. The problem is that the majority of users will simply connect an NTFS drive and expect that backups should work without any further configuration. Unfortunately this is not the case. Unless they create a dummy UserMapping file on the drive they will get "Operation not supported" errors, correct? So I suppose that I should also request that Netgear create a "NTFS-3G-UserMapping" file somewhere and also specify the mount option "usermapping=<absolute path to NTFS-3G-UserMapping>". Do you agree that would be the best approach?

Regarding performance, I tried copying a 734042112 byte file
Code:
# time cp BIG /USB_HDD_5/marktest/

real    11m49.882s
user    0m0.390s
sys     0m41.930s

Here are the results copying the same file using version 2010.8.8
Code:
# time cp BIG /USB_HDD_5/marktest/

real 3m12.523s
user 0m0.570s
sys 0m44.370s


Unfortunately the write performance of the new version is extremely low: 1034033 bytes/sec. With version 2010.8.8 I was getting 3812750 bytes/sec almost 4 times the performance. I'm pretty sure that the developer compiled using optimized mode in both cases:
Code:
# file ntfs-3g
ntfs-3g: ELF 32-bit MSB executable, SPARC, version 1 (SYSV), for GNU/Linux 2.2.0, dynamically linked (uses shared libs), stripped



Thanks,

Mark


Sun Oct 24, 2010 22:13
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Operation not supported (45) - ACL problem
Hi again,

Well, I have part of the explanation.

First, the flush is unrelated to the sync option, and it is not implemented. In this situation, fuse throws an error ENOSYS, which you normally see as "not implemented" in the log, but this error is filtered somewhere and is not forwarded to the application.

Now the version I compiled was based on my errno.h, which says :
Code:
#define ENOSYS          38      /* Function not implemented */

The log shows the error -38, but it is translated as "socket operation", and this error is forwarded to the application, which aborts.

In the version compiled by Netgear, the error is shown as -90 and translated to "not implemented". On my system, I have
Code:
#define EMSGSIZE        90      /* Message too long */


Conclusion : we have a different numbering of errors.

I thought the error numbering was the same on all Linuxes !

Do you know which distribution your NAS is based on ?

Regards

Jean-Pierre


Sun Oct 24, 2010 22:14
Profile

Joined: Mon Sep 13, 2010 18:42
Posts: 42
Post Re: Operation not supported (45) - ACL problem
jpa wrote:
Do you know which distribution your NAS is based on ?

Debian "sarge" I believe.


Sun Oct 24, 2010 22:21
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Operation not supported (45) - ACL problem
Hi again,

Quote:
1. Is it possible to fix FLUSH as well (for completeness)?

I do not think this is useful. It is a part of the closing routine, and the buffers are flushed on closing.
Quote:
2. When do you anticipate being able to provide a stable (non-advanced release) version that incorporates these fixes?

I have not negotiated with Tuxera which will be incorporated in the next versions. I am to make a proposal next week, then we will enter the Tuxera quality assurance procedures. This may take a couple of months... (positive user experience helps).
Quote:
3. Since the mount options for NTFS-3G are not supplied via /etc/fstab, which file are they provided in? (I will need to ask Netgear to modify this file)

They normally are provided in fstab for fixed disks. For pluggable devices, they are predefined by rules in some file unknown to me (at least in recent kernel versions).
Quote:
4. Should I ask for the following options? "user_xattr,rw,sync,noatime,allow_other"

user_xattr, rw and noatime are default options. sync is a very new option, try it out first, it may have consequences on performances. allow_other will probably have no effect in your case (are plain users allowed to mount ?).
Quote:
5. The problem is that the majority of users will simply connect an NTFS drive and expect that backups should work without any further configuration. Unfortunately this is not the case. Unless they create a dummy UserMapping file on the drive they will get "Operation not supported" errors, correct?

No, they will be allowed to write anything anywhere, and the files will be readable by anybody on any machine. That is what most users want.
Quote:
So I suppose that I should also request that Netgear create a "NTFS-3G-UserMapping" file somewhere and also specify the mount option "usermapping=<absolute path to NTFS-3G-UserMapping>". Do you agree that would be the best approach?

This way, they will force the use of protections and ACLs. Negear know better than me the needs of their customers, in terms of security and interoperating with Windows. I can tell what is technically possible (and open to proposals).
Quote:
Regarding performance, I tried copying a 734042112 byte file
real 11m49.882s
Here are the results copying the same file using version 2010.8.8
real 3m12.523s
Unfortunately the write performance of the new version is extremely low: 1034033 bytes/sec. With version 2010.8.8 I was getting 3812750 bytes/sec almost 4 times the performance.

Oh ! Please retry with no sync option.
Quote:
Debian "sarge" I believe.

Ok, I will get their include files.

Regards

Jean-Pierre


Sun Oct 24, 2010 22:54
Profile

Joined: Mon Sep 13, 2010 18:42
Posts: 42
Post Re: Operation not supported (45) - ACL problem
jpa wrote:
user_xattr, rw and noatime are default options. sync is a very new option, try it out first, it may have consequences on performances.

Oh ! Please retry with no sync option.

On the AR release the "sync" option definitely seems to adversely affect performance. With "sync" switched off I get the following performance:
Code:
real 3m3.850s

On the 2010.8.8 release it seems that the "sync" option has no effect on performance whatsoever:
Code:
real 3m14.627s

Is the "sync" option working at all in 2010.8.8? If it is, then why is there such a performance penalty in the latest version? And finally, what do we give up if the "sync" option is off (does it mean that the user might corrupt the filesystem by disconnecting the USB drive since some data may not have been flushed to the device)?
jpa wrote:
allow_other will probably have no effect in your case (are plain users allowed to mount ?).

Mounting occurs automatically when a drive is attached to the NAS so I don't think users have any control over this through the standard "Frontview" user interface. When logged in via SSH, all commands are run as "root" so once again mount would only be executed by root. I suppose that Netgear probably are simply using the default options for NTFS-3G (and "allow_other" is a default option) .
jpa wrote:
mjw wrote:
5. The problem is that the majority of users will simply connect an NTFS drive and expect that backups should work without any further configuration. Unfortunately this is not the case. Unless they create a dummy UserMapping file on the drive they will get "Operation not supported" errors, correct?

No, they will be allowed to write anything anywhere, and the files will be readable by anybody on any machine. That is what most users want.

I think that this will not be suitable in the case of the ReadyNAS since the primary purpose of an NTFS external USB drive would be to backup the internal EXT3 RAID drives. Two commands are available for the backup:
1. cp -p
2. rsync -aAv
Both of the above commands will cause backup errors unless NTFS-3G is guaranteed to be running in UserMapping mode. For this reason I think there is no choice except to ask Netgear to create a dummy UserMapping file somewhere on the ext3 partition and point NTFS-3G to use it via the "usermapping" option. If there was some other option that could avoid the need for this (perhaps an "acl" option) it might be more convenient and allow NTFS-3G to work in a more standard way (since no other filesystems use a UserMapping file as a way of indicating "acl" mode).

Regards,

Mark


Sun Oct 24, 2010 23:38
Profile

Joined: Mon Sep 13, 2010 18:42
Posts: 42
Post Re: Operation not supported (45) - ACL problem
By the way, is there a way to attach files on this forum? I emailed the developer about errno.h and he has just sent me all the errno.h files on his system but I have no easy way to share this with you.

Also, if you would like to perform some tests yourself (or maybe compile some new versions), he has offered to give you access to his environment. His email address is
stefan AT whocares.de

Regards,

Mark


Sun Oct 24, 2010 23:43
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Operation not supported (45) - ACL problem
Hi,

Quote:
On the 2010.8.8 release it seems that the "sync" option has no effect on performance whatsoever:

It surely has no effect because it is not implemented. This make me worrying about its use : why did you use the sync option ? If many users are doing so, I am to be flooded with complains when the option is released in a stable version.
Quote:
Is the "sync" option working at all in 2010.8.8? If it is, then why is there such a performance penalty in the latest version? And finally, what do we give up if the "sync" option is off (does it mean that the user might corrupt the filesystem by disconnecting the USB drive since some data may not have been flushed to the device)?

This causes "All I/O to the filesystem should be done synchronously" (see mount(8)), When you write data into a file, the system waits until the data are written, the time stamps and file size are written in the parent directory, the allocation bitmap and other metadata are updated, etc. Without the sync, the system does not wait and the updates which occur before the data are actually written (typically in the parent directory and metadata) cancel the writing of obsolete data.
The sync option shortens the life of the storage device. It should only be used with databases, because the database managers reserve the space in advance (so data updating implies not bitmap updating, no file size updating, and the timestamps are unimportant), and they can use their own transaction log after an unexpected failure because they can be sure of the sequencing of writes to device.
Even with the sync, when you unplug the device while it is being written to, you can create inconsistency.
Quote:
I think that this will not be suitable in the case of the ReadyNAS since the primary purpose of an NTFS external USB drive would be to backup the internal EXT3 RAID drives. Two commands are available for the backup:
1. cp -p
2. rsync -aAv
Both of the above commands will cause backup errors unless NTFS-3G is guaranteed to be running in UserMapping mode.

I agree, but the errors you see are caused by your (unwanted) use of Posix ACLs.
Quote:
For this reason I think there is no choice except to ask Netgear to create a dummy UserMapping file somewhere on the ext3 partition and point NTFS-3G to use it via the "usermapping" option.

If they want to allow the use of Posix ACLs, they currently have to.
Quote:
If there was some other option that could avoid the need for this (perhaps an "acl" option) it might be more convenient and allow NTFS-3G to work in a more standard way (since no other filesystems use a UserMapping file as a way of indicating "acl" mode).

The very problem here is the discrepancy of user identification in your environment. The other filesystems do not have to be compatible with the Windows users identification.
Quote:
By the way, is there a way to attach files on this forum?

Yes, but you have to compress them (.gz .zip .rar etc.) I attach a description of how I set up the cross-compiler for sparc.
Quote:
I emailed the developer about errno.h and he has just sent me all the errno.h files on his system but I have no easy way to share this with you.

After googling I saw that sparc-linux uses a specific variant, probably for better compatibility with Solaris (eg http://www.mail-archive.com/git-commits ... 40591.html )
Quote:
Also, if you would like to perform some tests yourself (or maybe compile some new versions), he has offered to give you access to his environment.

Nice proposal. I am to contact him.

This may be the end of chapter one. Please test carefully the new version, and if some new problem arises, we will open chapter two.

Thank you for your cooperation and feedback. I learned a lot about the consequences of using Samba without unified SIDs, and this may lead to new developments in ntfs-3g.

Regards

Jean-Pierre


Attachments:
sparc.inf.gz [1.84 KiB]
Downloaded 872 times
Mon Oct 25, 2010 10:10
Profile

Joined: Mon Sep 13, 2010 18:42
Posts: 42
Post Re: Operation not supported (45) - ACL problem
Hi Jean-Pierre,

I have attached the errorno.h files. Thanks again for all your help. I will continue testing the version you have recommended.

jpa wrote:
This make me worrying about its use : why did you use the sync option ? If many users are doing so, I am to be flooded with complains when the option is released in a stable version.

I don't think that this is an issue. There is an option in the Netgear Frontview web user interface that says the following:
Frontview wrote:
Enable fast USB disk writes. This option speeds up USB write access by accessing the USB device in asynchronous mode. If you enable this option, do not remove the USB device without properly unmounting it. Failure to do so can compromise data integrity on the device.

I currently have this switched off on my device which would perhaps explain why the "sync" command is being passed as a mount option to ntfs-3g. I may be able to verify this for sure at some point when the NAS is not in use.

Regarding performance, I realize that the Readynas device I have has a pretty weak CPU, but even so I was wondering if there is anything you can do to improve write performance. Someone did a comparison of ntfs-3g vs Tuxera vs Paragon back in May this year on a MacBook Pro. Did you see it? I can only find it in google's webcache now:
http://webcache.googleusercontent.com/s ... clnk&gl=us

In any case, as you can see from the chart below, it seems that on identical hardware the Paragon driver was able to achieve 3.6 times the write performance of ntfs-3g (version 10.1.16):
Image

This would seem to suggest that there is some potential for further optimizing the performance of NTFS-3G so that it comes closer to saturating the USB connection. Do you have any thoughts on that? Also, why is it that Tuxera has higher performance - I thought that Tuxera is based on NTFS-3G?

Thanks,

Mark


Attachments:
errno.h.tar.gz [5.15 KiB]
Downloaded 826 times
Mon Oct 25, 2010 17:08
Profile

Joined: Mon Sep 13, 2010 18:42
Posts: 42
Post Re: Operation not supported (45) - ACL problem
P.S. I also looked at your FAQ regarding performance (http://www.tuxera.com/community/ntfs-3g-faq/#slow). On my Readynas, it is possible to achieve 10MB/second writes (with "fast USB writes" enabled) to an EXT3 formatted USB drive which is approximately double the write speed of NTFS-3G on the Readynas (this has been confirmed by many users so it's not isolated to my Readynas). The Readynas does support USB 2.0.

I unsuccessfully tried to check whether DMA is switched on:
Code:
# hdparm /dev/sda1

/dev/sda1:
readonly      =  0 (off)
readahead     = 256 (on)
geometry      = 60801/255/63, sectors = 976773104, start = 63
tera:~# hdparm -d /dev/sda1

/dev/sda1:
HDIO_GET_DMA failed: Invalid argument


Mon Oct 25, 2010 17:50
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Operation not supported (45) - ACL problem
Hi Mark,

Quote:
I have attached the errorno.h files.

Thank you. In the meantime I have got the full set of included files from Stefan, so I do not need your copy any more.
Quote:
I don't think that this is an issue. There is an option in the Netgear Frontview web user interface that says the following:

This would be a real issue if some distribution is setting the sync by default.
Quote:
Regarding performance, I realize that the Readynas device I have has a pretty weak CPU, but even so I was wondering if there is anything you can do to improve write performance.

I have made significant improvements in the past. But you know, improvements are harder and harder to get.
Quote:
Someone did a comparison of ntfs-3g vs Tuxera vs Paragon back in May this year on a MacBook Pro. Did you see it? I can only find it in google's webcache now:

Yes, I saw a summary which was posted here.
Quote:
This would seem to suggest that there is some potential for further optimizing the performance of NTFS-3G so that it comes closer to saturating the USB connection. Do you have any thoughts on that? Also, why is it that Tuxera has higher performance - I thought that Tuxera is based on NTFS-3G?

Basically Tuxera and Paragon get better performances because they are in-kernel versions, whereas ntfs-3g is a user-space one. This causes context switches and a longer data flow. There may also be differences specific to MacOSX which I do not know about. Anyway ntfs-3g will never catch up to Tuxera ntfs, the improvements on the open-source ntfs-3g are also fed into the closed-source Tuxera ntfs.
Quote:
On my Readynas, it is possible to achieve 10MB/second writes (with "fast USB writes" enabled) to an EXT3 formatted USB drive which is approximately double the write speed of NTFS-3G on the Readynas

The gap is likely to be present for a long time with ntfs-3g. You may want to switch to Tuxera ntfs.
Quote:
I unsuccessfully tried to check whether DMA is switched on:

I have no idea why the command failed. Was the device mounted ?

Regards

Jean-Pierre


Mon Oct 25, 2010 18:45
Profile

Joined: Mon Sep 13, 2010 18:42
Posts: 42
Post Re: Operation not supported (45) - ACL problem
jpa wrote:
Basically Tuxera and Paragon get better performances because they are in-kernel versions, whereas ntfs-3g is a user-space one. This causes context switches and a longer data flow.

I initially suspected that this may be the case, but was perhaps misinformed by the NTFS-3G FAQ (http://www.tuxera.com/community/ntfs-3g-faq/#kernel) which claims that there is no reason why NTFS-3G cannot be as well-performing as it would be if it were purely in the kernel:
Quote:
When will be the driver ported to the kernel?

When it becomes clear that a huge, complex, feature rich and general purpose file system can not be as reliable and well-performing in hybrid space as purely in the kernel. At the moment there are no such strong indications. In fact, more and more experiences show just the opposite.

The FAQ even seems to suggest that when it becomes clear that performance would be better if implemented inside the kernel then the NTFS-3G driver will be ported to the kernel. Perhaps there are commercial reasons for not doing this (for example to provide sufficient product differentiation between NTFS-3G and Tuxera). It is understandable to be concerned that Tuxera sales could be affected if NTFS-3G performance was comparable.
jpa wrote:
The gap is likely to be present for a long time with ntfs-3g. You may want to switch to Tuxera ntfs.

Unfortunately I have very little influence on Netgear regarding moving to a non-free licensed driver version. You can see that even after 1 week Netgear still hasn't even been able to produce a .deb of your latest release (something that took Stefan no time at all). Perhaps Tuxera sales should reach out to Netgear directly since the main Readynas competitor Qnap have moved to Paragon in the last year and Qnap users are now benefiting from 99 MB/s performance for NTFS drives connected via eSATA (http://www.smallnetbuilder.com/index.ph ... &chart=114). Netgear users are struggling along with about 10 MB/s. I would think that Netgear will have to do something to compete even if it costs a little money.
jpa wrote:
mjw wrote:
I unsuccessfully tried to check whether DMA is switched on:

I have no idea why the command failed. Was the device mounted ?

[/quote]
Yes the device was mounted. Perhaps now that you will shortly have a Sparc development environment it might be possible for you to run some performance profiling of your own to see if there are any quick wins. I realize that the chances of finding significant performance improvements at this point are slim, but if you are able to find any this would probably be the number one most requested feature of NTFS-3G that I have seen posted in the Readynas online forums.


Mon Oct 25, 2010 20:28
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Operation not supported (45) - ACL problem
Hi Mark,

Quote:
I initially suspected that this may be the case, but was perhaps misinformed by the NTFS-3G FAQ (http://www.tuxera.com/community/ntfs-3g-faq/#kernel) which claims that there is no reason why NTFS-3G cannot be as well-performing as it would be if it were purely in the kernel:

This is an old remark based on some view at the time. I tend to not agree now, however I have not used Tuxera ntfs, so I cannot tell from experience.
Quote:
The FAQ even seems to suggest that when it becomes clear that performance would be better if implemented inside the kernel then the NTFS-3G driver will be ported to the kernel.

This is what Tuxera has done, but the port is closed-sourced.
Quote:
It is understandable to be concerned that Tuxera sales could be affected if NTFS-3G performance was comparable.

Tuxera has never asked not to port ntfs-3g to in-kernel interfacing. I have personally no appropriate expertise to do that.
Quote:
Perhaps Tuxera sales should reach out to Netgear directly since the main Readynas competitor Qnap have moved to Paragon in the last year and Qnap users are now benefiting from 99 MB/s performance for NTFS drives connected via eSATA

(forwarded to Tuxera)
Quote:
Unfortunately I have very little influence on Netgear regarding moving to a non-free licensed driver version. You can see that even after 1 week Netgear still hasn't even been able to produce a .deb of your latest release (something that took Stefan no time at all).

I found this rather efficient for an unusual customer request. Only the hobbyists do not consider what will be the return.
Quote:
Perhaps now that you will shortly have a Sparc development environment it might be possible for you to run some performance profiling of your own to see if there are any quick wins. I realize that the chances of finding significant performance improvements at this point are slim, but if you are able to find any this would probably be the number one most requested feature of NTFS-3G that I have seen posted in the Readynas online forums.

I have done a lot of profiling last year, and I know exactly where the time is spent in ntfs-3g when doing my own kind of job (development, lots of small files). I have improved what I can, there is no quick-wins any more, and I have difficulties improving more (outside some specific usage pattern to be identified). Sorry about it.

Regards

Jean-Pierre


Mon Oct 25, 2010 21:59
Profile

Joined: Mon Sep 13, 2010 18:42
Posts: 42
Post Re: Operation not supported (45) - ACL problem
Hi Jean-Pierre,

jpa wrote:
Quote:
2. When do you anticipate being able to provide a stable (non-advanced release) version that incorporates these fixes?

I have not negotiated with Tuxera which will be incorporated in the next versions. I am to make a proposal next week, then we will enter the Tuxera quality assurance procedures. This may take a couple of months... (positive user experience helps).

So far the 2010.10.2AR.1 release is working without problems and I have had no further errors from my ReadyNAS backup jobs. I finally got a response back from Netgear. Unfortunately it seems that they are not interested in incorporating this fix officially into their product firmware until it becomes a stable release version.

Did you complete your discussions with Tuxera and will this fix be incorporated into the next stable release?

Thanks,

Mark


Tue Nov 09, 2010 20:30
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Operation not supported (45) - ACL problem
Hi Mark,

Quote:
Did you complete your discussions with Tuxera and will this fix be incorporated into the next stable release?

The proposal is still under discussion, including the fix you need. Be patient.

Regards

Jean-Pierre


Tue Nov 09, 2010 22:46
Profile

Joined: Mon Sep 13, 2010 18:42
Posts: 42
Post Re: Operation not supported (45) - ACL problem
Hi Jean-Pierre,

I noticed a new stable release "STABLE Version 2011.1.15 (January 23, 2011)". It sounds from the release notes that this version contains all the fixes that helped to resolve the problems in this thread. Is that the case? If so, I will let Netgear know that there is a new stable release that they can start testing with.

Thanks!


Sun Jan 30, 2011 21:07
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Operation not supported (45) - ACL problem
Hi,

Quote:
It sounds from the release notes that this version contains all the fixes that helped to resolve the problems in this thread. Is that the case?

This release should contain all the fixes I know about. Just note the use of Posix ACLs is still a compile-time option, but I have published a prelinked version for Sparc with the Posix ACLs enabled.
Quote:
If so, I will let Netgear know that there is a new stable release that they can start testing with.

Fine. Thanks.

Regards

Jean-Pierre


Sun Jan 30, 2011 22:36
Profile

Joined: Mon Sep 13, 2010 18:42
Posts: 42
Post Re: Operation not supported (45) - ACL problem
Great news. Where can I download the Sparc version? I looked here - did I miss it?
http://www.tuxera.com/community/ntfs-3g-download/


Sun Jan 30, 2011 23:22
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 79 posts ]  Go to page Previous  1, 2, 3, 4  Next


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.