FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Sun Jun 13, 2021 04:44



Post new topic Reply to topic  [ 7 posts ] 
NTFS-3G poor write performance when Cool&Quiet is activated 
Author Message

Joined: Wed Jan 05, 2011 15:38
Posts: 4
Post NTFS-3G poor write performance when Cool&Quiet is activated
Hi,

i have set up a Software Raid 5 on a standard ubuntu 10.10 Installation, which can deliver Data at a rate up to 200MB/s and more, so everthing is fine with the Raid.
Now i'm trying to copy or move Files with sizes up to 16GB to an external esata Drive, which ist formatted with ntfs on an aligned partition starting on sector 2048, size 1TB.

When Cool&Quiet is activated, the write performance is low as 37MB/s, so the 1TB needs 7hrs to be filled.
The CPU is a Dual-Core 2,8Ghz Athlon 7850, which idles at 1,4Ghz all the time during copying and no higher CPU usage (60%) or clock-advance to 2,8Ghz can be seen.

Deactivation of Cool&Quiet in the Motherbord Bios jumps the write rate to 67MB/s, CPU usage still ~ 60%, changing the CPU clock by setting the Multiplier always changes the write rate, 50MB/s on 2Ghz, 70MB/s on 3Ghz, CPU usage still ~ 60% ...

To find out, whats going on, i tested the same HDD formatted with ext4, write rate is always 100MB/s, which is max for this drive, doesnt matter C&Q active or not ...

To me it looks like ntfs-3g cannot ask the CPU/C&Q for higher clock or usage to achieve an adequate write rate.
I would say its a bug, not a feature ... , is there any solution to this ?

The system data:
Asus M4N78 PRO, Dual-Core 2,8Ghz, 4GB RAM, ext esata Drive Samsung HD103UJ, ubuntu 10.10, Kernel 2.6.35.24.28, ntfs-3g 2010.8.8

German discussion in ubuntuusers:
http://forum.ubuntuusers.de/topic/softw ... atte-zu-l/

with kind regards from Germany
Andi


Wed Jan 05, 2011 16:28
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: NTFS-3G poor write performance when Cool&Quiet is activated
Hi Andi,

Quote:
To me it looks like ntfs-3g cannot ask the CPU/C&Q for higher clock or usage to achieve an adequate write rate.
I would say its a bug, not a feature ... , is there any solution to this ?

ntfs-3g has definitely no interface to request any increase or decrease of clock speed. IMHO some specialized process measures the idle state rate of the CPUs and adjust the clock speed accordingly.

What you are experiencing is probably a consequence of ntfs-3g being single-threaded. When you see 60%, this probably means one CPU used by the single-threaded ntfs-3g process at 100%, while the other CPU serves the other processes at 20%. At higher clock speeds, everything is quicker but the proportions remain the same. If throughput decreases when activating Cool&Quiet, its logic is probably affected by a CPU working at a moderate load.

If you have mainly big files, you may get some improvement by using ntfs-3g-2010.10.2AR.3 (http://www.tuxera.com/community/ntfs-3g-advanced/)

Regards

Jean-Pierre


Wed Jan 05, 2011 23:25
Profile

Joined: Wed Jan 05, 2011 15:38
Posts: 4
Post Re: NTFS-3G poor write performance when Cool&Quiet is activated
Hi Jean-Pierre,

thanks for answer,

i took 2 screenshots, that you can see, how the CPU is used.
Both are made with C&Q activated, one is writing to an ntfs, second is writing to ext4.
C&Q deavtivation gives the same picture with a higher throughput (ntfs), as i wrote earlier.
It is the same HDD, just changed the filesystem on it.

On one hand i can understand, that writing to ntfs may use more ressources,
on the other i can not understand, that it dont use all or most of the ressources, that are available.

It looks like the CPU usage is limited somehow to app. 60%

btw i can not install ntfs-3g-2010.10.2AR.3 at the moment, converting the rpm to a deb and installing failed.
is there a deb repo ? or some other way ?

Regards

Andi


Attachments:
copy to ntfs.png
copy to ntfs.png [ 130.42 KiB | Viewed 18531 times ]
copy to ext4.png
copy to ext4.png [ 132.94 KiB | Viewed 18531 times ]
Thu Jan 06, 2011 22:00
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: NTFS-3G poor write performance when Cool&Quiet is activated
Hi Andi,

Quote:
i took 2 screenshots, that you can see, how the CPU is used.
Both are made with C&Q activated, one is writing to an ntfs, second is writing to ext4.

So both CPU loads are in the 60% area, which means my explanation does not hold... This tends to move the suspicion away from ntfs-3g.
Quote:
btw i can not install ntfs-3g-2010.10.2AR.3 at the moment, converting the rpm to a deb and installing failed.
is there a deb repo ? or some other way ?

I have also put a plain static-linked binary for such situation : the Knoppix one on the same page. You do not need to install anything or uninstall your current version (instructions in the readme file), but as the CPU load is not at stake, you are unlikely to get a significant change in behavior.

Regards

Jean-Pierre


Thu Jan 06, 2011 22:38
Profile

Joined: Wed Jan 05, 2011 15:38
Posts: 4
Post Re: NTFS-3G poor write performance when Cool&Quiet is activated
OK,

tested ntfs-3g-2010.10.2AR.3 out of the tarball without installing, no improvements ...

any idea, what to look next ... ;-)

btw do i understand right, you think it is something else than ntfs-3g,

but its the same hardware, the same nautilus, the same folder on the same source, i want to copy,

just the filesystem on the target drive is changed, so ntfs-3g has come to work, what else could be different ... ???

Regards

Andi


Fri Jan 07, 2011 00:19
Profile

Joined: Wed Jan 05, 2011 15:38
Posts: 4
Post Re: NTFS-3G poor write performance when Cool&Quiet is activated
Hi Jean-Pierre,

one good news, one bad news ...

Cool'n'Quiet is out of the race for Troubleshooting, there are different ways to regulate the core clocks.

I was expecting, a preset called "ondemand" does the right thing, but "conservative" does :roll: , now both core clock step up to 3Ghz as soon as the copy job starts.

This means also, ntfs-3g is not responsible for the missing speed up :mrgreen:

The bad news is, we are sticking with this max 60% CPU usage, it is still there. (ends up in max 67MB/s writing out of 100 possible))

There are 2 questions :

Why is CPU load that high, is there a way to reduce the demand of CPU-Power, that ntfs-3g is using this time ? (do you see any potential by reprogramming it ?)

If it needs to be that high, why is it not using all the power that is there ?

Am i right, that ntfs writing is not (longer) a Kernel job, is there something in the system that limits the power to this job/process ??? Is it adjustable ???

If you also know somebody else to ask these, please let me know

Regards

Andi


Fri Jan 07, 2011 19:39
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: NTFS-3G poor write performance when Cool&Quiet is activated
Hi Andi,

Quote:
I was expecting, a preset called "ondemand" does the right thing, but "conservative" does :roll: , now both core clock step up to 3Ghz as soon as the copy job starts.

Good catch !
Quote:
Why is CPU load that high, is there a way to reduce the demand of CPU-Power, that ntfs-3g is using this time ? (do you see any potential by reprogramming it ?)

No obvious way to reduce the CPU load. Using bigger write buffer may be a solution (but buffer size cannot be controlled by cp or rsync). Reprogramming is probably not compatible with your time frame (but if you pinpoint a specific defect, it might be fixable).
Quote:
Am i right, that ntfs writing is not (longer) a Kernel job, is there something in the system that limits the power to this job/process ??? Is it adjustable ???

ntfs-3g is a user-space driver, this means each write request has to go through an interprocess communication which is not needed by in-kernel driver. This explains an extra CPU load when compared against in-kernel drivers.
Quote:
If you also know somebody else to ask these, please let me know

Tuxera ntfs is an in-kernel driver, you may ask them for a try (see Tuxera home page).

Regards

Jean-Pierre


Fri Jan 07, 2011 23:06
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.