FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Fri Jul 30, 2021 21:36



Post new topic Reply to topic  [ 4 posts ] 
La vitesse limité malgré le disque utilisé 
Author Message

Joined: Mon Nov 30, 2009 21:35
Posts: 38
Post La vitesse limité malgré le disque utilisé
Chers Monsieurs,
j'ai utilisé le NTFS-Generation Troisième au cours de deux ans, et mes disques étaient deja tres agés.

Mais ce semaine-la j'ai incorporé en appareil de stockage meilleur, de 2012.
Puis j'ai découvert quelque chose tres étonnante: le probleme (je considére ce qu'il y a un problème) persiste.
Malgré le disque active la vitesse reste limitée au moins de 14 mégaoctet par seconde.

J'ai analysé en plus peu avec l'option de "debug" activée, et voici le resultat:
--------------------------------------------------------------------------

par console
Code:
WRITE[0] 4096 bytes to 59019264
   WRITE[0] 4096 bytes
   unique: 21353, error: 0 (Победа), outsize: 24
unique: 21354, opcode: WRITE (16), nodeid: 34, insize: 4160
WRITE[0] 4096 bytes to 59023360
   WRITE[0] 4096 bytes
   unique: 21354, error: 0 (Победа), outsize: 24
unique: 21355, opcode: WRITE (16), nodeid: 34, insize: 4160
WRITE[0] 4096 bytes to 59027456
   WRITE[0] 4096 bytes
   unique: 21355, error: 0 (Победа), outsize: 24
unique: 21356, opcode: WRITE (16), nodeid: 34, insize: 4160
WRITE[0] 4096 bytes to 59031552
   WRITE[0] 4096 bytes
   unique: 21356, error: 0 (Победа), outsize: 24
unique: 21357, opcode: WRITE (16), nodeid: 34, insize: 4160
WRITE[0] 4096 bytes to 59035648
   WRITE[0] 4096 bytes
   unique: 21357, error: 0 (Победа), outsize: 24
unique: 21358, opcode: WRITE (16), nodeid: 34, insize: 4160
WRITE[0] 4096 bytes to 59039744
   WRITE[0] 4096 bytes
   unique: 21358, error: 0 (Победа), outsize: 24
unique: 21359, opcode: FLUSH (25), nodeid: 34, insize: 64
FLUSH[0]
   unique: 21359, error: -38 (Функция не реализована), outsize: 16
unique: 21360, opcode: RELEASE (18), nodeid: 34, insize: 64
RELEASE[0] flags: 0x8002
   unique: 21360, error: 0 (Победа), outsize: 16


Le programme n'enregistre jamais plus que 4 kilooctets par l'interrogation. Chaque fois.
Donc le buffer est limité a niveau de 4,096 octets.
J'ai vérifié encore une fois. Je pense que cela pourrait être ici:

fuse-2.8.1/lib/cuse-lowlevel.c et cuse-lowlevel.c
Code:
   if (bufsize < FUSE_MIN_READ_BUFFER) {
      fprintf(stderr, "fuse: warning: buffer size too small: %zu\n",
         bufsize);
      bufsize = FUSE_MIN_READ_BUFFER;
   }

   bufsize -= 4096;
   if (bufsize < f->conn.max_write)
      f->conn.max_write = bufsize;


Il paraît que le disque est saturé des interrogations minuscules, par ce que l'usage de chaque processeur ne gagne plus de 60 pour-cents.

La librairie de FUSE sans doute a une option de "max_write",

fuse-2.8.1/lib/cuse-lowlevel.c
Quote:
fprintf(stderr,
" -o max_write=N set maximum size of write requests\n"
" -o max_readahead=N set maximum readahead\n"
" -o async_read perform reads asynchronously (default)\n"
" -o sync_read perform reads synchronously\n"
" -o atomic_o_trunc enable atomic open+truncate support\n"
" -o big_writes enable larger than 4kB writes\n"
" -o no_remote_lock disable remote file locking\n");


Mais cet argument n'est passé par le NTFS-Generation Troisième.

ntfs-3g/libfuse-lite/libfuse-lite.c
Quote:
static const struct fuse_opt fuse_mount_opts[] = {
FUSE_MOUNT_OPT("allow_other", allow_other),
FUSE_MOUNT_OPT("allow_root", allow_root),
FUSE_MOUNT_OPT("blkdev", blkdev),
FUSE_MOUNT_OPT("fsname=%s", fsname),
FUSE_OPT_KEY("allow_other", KEY_KERN_OPT),
FUSE_OPT_KEY("allow_root", KEY_ALLOW_ROOT),
FUSE_OPT_KEY("blkdev", KEY_FUSERMOUNT_OPT),
FUSE_OPT_KEY("fsname=", KEY_FUSERMOUNT_OPT),
FUSE_OPT_KEY("large_read", KEY_KERN_OPT),
FUSE_OPT_KEY("blksize=", KEY_KERN_OPT),
FUSE_OPT_KEY("default_permissions", KEY_KERN_OPT),
FUSE_OPT_KEY("max_read=", KEY_KERN_OPT),
FUSE_OPT_KEY("max_read=", FUSE_OPT_KEY_KEEP),
FUSE_OPT_KEY("user=", KEY_MTAB_OPT),
FUSE_OPT_KEY("-r", KEY_RO),
FUSE_OPT_KEY("ro", KEY_KERN_FLAG),
FUSE_OPT_KEY("rw", KEY_KERN_FLAG),
FUSE_OPT_KEY("suid", KEY_KERN_FLAG),
FUSE_OPT_KEY("nosuid", KEY_KERN_FLAG),
FUSE_OPT_KEY("dev", KEY_KERN_FLAG),
FUSE_OPT_KEY("nodev", KEY_KERN_FLAG),
FUSE_OPT_KEY("exec", KEY_KERN_FLAG),
FUSE_OPT_KEY("noexec", KEY_KERN_FLAG),
FUSE_OPT_KEY("async", KEY_KERN_FLAG),
FUSE_OPT_KEY("sync", KEY_KERN_FLAG),
FUSE_OPT_KEY("dirsync", KEY_KERN_FLAG),
FUSE_OPT_KEY("atime", KEY_KERN_FLAG),
FUSE_OPT_KEY("noatime", KEY_KERN_FLAG),
FUSE_OPT_KEY("-h", KEY_HELP),
FUSE_OPT_KEY("--help", KEY_HELP),
FUSE_OPT_KEY("-V", KEY_VERSION),
FUSE_OPT_KEY("--version", KEY_VERSION),
FUSE_OPT_END
};



Ne pourriez-vous expliquer, s'il vous plait, pourquoi il agit comme ca?
Est ce qu'il ya quelque probleme technique avec des blocs grands en NTFS-3G?
Autrement comment de passer l'option de "large write" utilisant les options de NTFS-3G?

Note: le cluster de NTFS on tous des disques est 64K.


Fri Nov 02, 2012 15:03
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: La vitesse limité malgré le disque utilisé
Hi,

Quote:
Chers Monsieurs,

Though French is my favourite language, I will use the customary language of this forum. Thank you for your understanding.
Quote:
Le programme n'enregistre jamais plus que 4 kilooctets par l'interrogation. Chaque fois.
Donc le buffer est limité a niveau de 4,096 octets

In your situation, the program does indeed only record 4KB per write call.
Quote:
J'ai vérifié encore une fois. Je pense que cela pourrait être ici:

You checked and found a possible cause, but I think the sequence you showed does the opposite, it prevents from using buffers smaller than 4KB.
Quote:
Code:
static const struct fuse_opt fuse_mount_opts[] = {

These are the options passed to mount(8), the buffer sizes are not mentioned because ther are unrelated to mounting, they are fuse specific options.
Quote:
Ne pourriez-vous expliquer, s'il vous plait, pourquoi il agit comme ca?

Why is it so ? It is a default choice made by the fuse project, which is not so bad for the most common cluster size.
Quote:
Est ce qu'il ya quelque probleme technique avec des blocs grands en NTFS-3G?
Autrement comment de passer l'option de "large write" utilisant les options de NTFS-3G?

Technical problems in using bigger buffers ? Oh yes, there probably are, nevertheless reading the manual may sometimes be useful.
Quoting from the ntfs-3g manual :
Code:
       big_writes
              This option prevents fuse from splitting write buffers  into  4K
              chunks,  enabling  big  write buffers to be transferred from the
              application in a single step (up to some system limit, generally
              128K bytes).

Now, you have to understand that the write buffers are controlled by the application which writes. If you use fprintf(3) for writing, the buffer size is defined by the macro BUFSIZ defined in <stdio.h>, unless you define another buffer by setbuffer(3).

The command cp(1) apparently uses 64K buffers, tar(1) has an option -b to define the buffer size, dd(1) has the option bs to defined the buffer size, ...

Quote:
Note: le cluster de NTFS on tous des disques est 64K.

If you are using 64KB clusters, you should really use the option big_writes.

Regards

Jean-Pierre


Fri Nov 02, 2012 18:36
Profile

Joined: Mon Nov 30, 2009 21:35
Posts: 38
Post Re: La vitesse limité malgré le disque utilisé
Dear jpa, thanks for answering soon,

jpa wrote:
nevertheless reading the manual may sometimes be useful.
Quoting from the ntfs-3g manual :
Code:
       big_writes
              This option prevents fuse from splitting write buffers  into  4K
              chunks,  enabling  big  write buffers to be transferred from the
              application in a single step (up to some system limit, generally
              128K bytes).



Sadly, no a such an option in 2010.2.6AR.1 which i have still used.
I've read the Release story carefully, but didn't found when a big_writes parameter was first introduced.


I have upgraded to 2012.1.AR15.8

Code:
WRITE[0] 61440 bytes to 43315200
   WRITE[0] 61440 bytes
   unique: 3108, error: 0 (Победа), outsize: 24
unique: 3109, opcode: GETXATTR (22), nodeid: 42, insize: 68
   unique: 3109, error: -61 (Нет доступных данных), outsize: 16
unique: 3110, opcode: WRITE (16), nodeid: 42, insize: 61504
WRITE[0] 61440 bytes to 43376640
   WRITE[0] 61440 bytes
   unique: 3110, error: 0 (Победа), outsize: 24
unique: 3111, opcode: GETXATTR (22), nodeid: 42, insize: 68
   unique: 3111, error: -61 (Нет доступных данных), outsize: 16
unique: 3112, opcode: WRITE (16), nodeid: 42, insize: 61504
WRITE[0] 61440 bytes to 43438080
   WRITE[0] 61440 bytes
   unique: 3112, error: 0 (Победа), outsize: 24


Got a considerable improvement in speed with big_writes in fstab - now the write speed is limited only by a network throughput, i.e 26-30 Mb/sec.
Should note also that the cpu usage by ntfs-3g has been dropped seriously - now less than 28% on Atom 1600.

Thank you again
and merci.


Fri Nov 02, 2012 20:47
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: La vitesse limité malgré le disque utilisé
Hi,

Quote:
I've read the Release story carefully, but didn't found when a big_writes parameter was first introduced.

It was introduced in 2011.4.12AR.1 as shown on http://jp-andre.pagesperso-orange.fr/changelog.html

Quote:
WRITE[0] 61440 bytes to 43315200

Your buffer size is 15*4096 bytes, if your cluster size is 16*4096, most writes will affect two clusters. If you can get the same values for both sizes, you will probably get some improvement.

Regards

Jean-Pierre


Fri Nov 02, 2012 22:27
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.