FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Thu May 13, 2021 21:03



Post new topic Reply to topic  [ 4 posts ] 
NTFS cp -pu problem 
Author Message

Joined: Sat Dec 11, 2010 06:13
Posts: 2
Post NTFS cp -pu problem
Problem
Trying to cp -pu from an NTFS partition to an ext3 partition, the timestamps of source & dest look OK, shown by terminal ls -l and by nautilus columns Date Modified and Date Accessed.

So cp's -p option works. (I mention that because old forums said that was a problem - looks like now fixed)
But cp's -u option says that the same file should not get copied again unless it's changed.

But it does get copied again, and again.

The old files that I put when I made the partition with gparted and partimage don't copy again, but any that I have created or modified since then just keep getting copied each time.



Just to remind,
-p same as --preserve=mode,ownership,timestamps
-u, --update copy **only** when the SOURCE file is newer than the
destination file or when the destination file is missing



So here's too many details, maybe some relevant.

For a couple of years I've been doing
cp -puv -t $dest $source
from one ext3 partition to another, no problem.

But now I've copied my whole data disk to an NTFS partition on a USB
portable hard drive. NTFS so I can use it at work with Win boxes.

I want to copy files from this NTFS partition to an ext3 destination
with timestamps intact, so that next week when I do the same thing, I
only copy new or changed files.

Demo* code is
source=/media/data/aaa_S500_data/* # the NTFS partition
dest='/home/garry/Desktop/junk' # a ext3 partition
cp -puv -t $dest $source
#*Demo Really I look at 30GB and 20,000 files, that's why I don't want to be copying the same thing repeatedly.

Here is a demo source directory
garry@S500-lucid:~$ ls -l /media/data/aaa_S500_data/
total 64
-rwxrwxrwx 1 garry garry 38254 2010-12-02 06:20 elephantGreen.ico
-rwxrwxrwx 1 garry garry 5978 2010-10-25 17:27 elephant.ico
-rwxrwxrwx 1 garry garry 69 2010-11-30 05:55 readme.txt
-rwxrwxrwx 1 garry garry 766 2002-10-14 04:27 wdlogo.ico
-rwxrwxrwx 1 garry garry 5978 2010-10-25 17:27 wooly-mammoth.ico
garry@S500-lucid:~$


So into an empty directory, '/home/garry/Desktop/junk' I copy
garry@S500-lucid:~$ cp -puv -t '/home/garry/Desktop/junk'
/media/data/aaa_S500_data/*
`/media/data/aaa_S500_data/elephantGreen.ico' -> `/home/garry/Desktop/junk/elephantGreen.ico'
`/media/data/aaa_S500_data/elephant.ico' -> `/home/garry/Desktop/junk/elephant.ico'
`/media/data/aaa_S500_data/readme.txt' -> `/home/garry/Desktop/junk/readme.txt'
`/media/data/aaa_S500_data/wdlogo.ico' -> `/home/garry/Desktop/junk/wdlogo.ico'
`/media/data/aaa_S500_data/wooly-mammoth.ico' -> `/home/garry/Desktop/junk/wooly-mammoth.ico'
garry@S500-lucid:~$

Then without changing any files in $source, and if I do
cp -puv -t '/home/garry/Desktop/junk' /media/data/aaa_S500_data/*
again, nothing should be copied.

But the two files that were created / altered on the NTFS partition
after[size=50][size=50][/size][/size] it got created with gparted & partimage get copied again.

garry@S500-lucid:~$ cp -puv -t '/home/garry/Desktop/junk' /media/data/aaa_S500_data/*
`/media/data/aaa_S500_data/elephantGreen.ico' -> `/home/garry/Desktop/junk/elephantGreen.ico'
`/media/data/aaa_S500_data/readme.txt' -> `/home/garry/Desktop/junk/readme.txt'
garry@S500-lucid:~$

and again

garry@S500-lucid:~$ cp -puv -t $dest $source
`/media/data/aaa_S500_data/elephantGreen.ico' -> `/home/garry/Desktop/junk/elephantGreen.ico'
`/media/data/aaa_S500_data/readme.txt' -> `/home/garry/Desktop/junk/readme.txt'



here is my /etc/fstab
proc /proc proc nodev,noexec,nosuid 0 0
UUID=a49e1eb6-72bc-465d-9852-ce99403d303f / ext3 errors[size=50]=remount-ro 0 1
UUID=f13a83d6-fdfd-4dc7-b67c-a3d47c9fd727 none swap sw 0 0
UUID=07B3B28C580F4BEA /media/data ntfs-3g defaults,umask=000,uid=1000,gid=1000 0 1
#UUID=07B3B28C580F4BEA /media/data ntfs-3g defaults,relatime,umask=000,uid=1000,gid=1000 0 1
#UUID=07B3B28C580F4BEA /media/data ntfs defaults 0 1
UUID=c1841d07-7991-4e9c-8d35-969cb49ccf28 /media/toybox_data ext3 defaults 0 2


I tried
touch --reference '/media/data/aaa_S500_data/newfile' '/home/garry/Desktop/junk/newfile'.
It copied timestamps ok but that made no difference to the cp -u problem.


What I'm using:-
ubuntu 10.04
ntfs-3g 2010.3.6 #came with ubuntu
ntfs-3g 2010.10.2 #installed today, no diffferent
GNU Bash-4.1
cp in GNU coreutils 7.4


So is this a proper bug? Looks like it to me, but I'm amazed if nobody else found it first.

regards
Garry


Sat Dec 11, 2010 06:35
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: NTFS cp -pu problem
Hi,

Quote:
The old files that I put when I made the partition with gparted and partimage don't copy again, but any that I have created or modified since then just keep getting copied each time.

Do you remember the old times when you were copying to FAT and half the files were copied again and again ? This was because FAT could only store even seconds and files with an odd number of seconds had their timestamp rounded down.

Similarly, ext3 and ntfs have different time resolutions : ext3 resolves to one second and ntfs resolves to 100 nanoseconds.
Code:
# create on /shared which is on ntfs
[linux@pavilion2 ntfs-3g]$ touch /shared/doc/try
[linux@pavilion2 ntfs-3g]$ ls -l --full-time /shared/doc/try
-rw-r--r-- 1 linux linux 0 2010-12-11 09:15:10.963284000 +0100 /shared/doc/try
# copy to /tmp which is on ext3
[linux@pavilion2 ntfs-3g]$ cp -p /shared/doc/try /tmp/try
[linux@pavilion2 ntfs-3g]$ ls -l --full-time /shared/doc/try /tmp/try
-rw-r--r-- 1 linux linux 0 2010-12-11 09:15:10.963284000 +0100 /shared/doc/try
-rw-r--r-- 1 linux linux 0 2010-12-11 09:15:10.000000000 +0100 /tmp/try
# copy from ntfs to ntfs
[linux@pavilion2 ntfs-3g]$ cp -p /shared/doc/try /shared/doc/try2
[linux@pavilion2 ntfs-3g]$ ls -l --full-time  /shared/doc/try /shared/doc/try2
-rw-r--r-- 1 linux linux 0 2010-12-11 09:15:10.963284000 +0100 /shared/doc/try
-rw-r--r-- 1 linux linux 0 2010-12-11 09:15:10.963284000 +0100 /shared/doc/try2

Now, you will get the same problem the other way with ext4 whose timestamps resolve to one nanosecond.

This does not happen on Windows because timestamps are rounded upward. This can also be done on ntfs-3g (useful when copying from ext4), but ext3 is out of my reach.

Back to your situation, if your original file was on ext3, you copy to ntfs, then back to ext3, it should not be copied again. Your problems stems from the fact you have modified your file on ntfs (using Windows, but would be the same on Linux), and cp -pu cannot copy the exact timestamp to ext3.

Apart from migrating to ext4 I see no immediate solution.

Regards

Jean-Pierre


Sat Dec 11, 2010 10:40
Profile
Tuxera CTO

Joined: Tue Nov 21, 2006 23:15
Posts: 1648
Post Re: NTFS cp -pu problem
Please note, the discussion is also ongoing here:

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=7612

Regards, Szaka


Sat Dec 11, 2010 13:36
Profile

Joined: Sat Dec 11, 2010 06:13
Posts: 2
Post Re: NTFS cp -pu problem
Thank you Jean-Pierre

Quote:
ext3 and ntfs have different time resolutions : ext3 resolves to one second and ntfs resolves to 100 nanoseconds.

Now, you will get the same problem the other way with ext4 whose timestamps resolve to one nanosecond.

This does not happen on Windows because timestamps are rounded upward. This can also be done on ntfs-3g (useful when copying from ext4), but ext3 is out of my reach.

Back to your situation, if your original file was on ext3, you copy to ntfs, then back to ext3, it should not be copied again. Your problems stems from the fact you have modified your file on ntfs (using Windows, but would be the same on Linux), and cp -pu cannot copy the exact timestamp to ext3.

Apart from migrating to ext4 I see no immediate solution.



I made a ext4 partition and tested, copy both ways from ntfs -> ext4 and back, and it all works, copy when it should, not when it should not. Problem solved.


If I understand what you've said, 2.000000 sec is earlier than 2.1111111 second, therefore the 2.1111111 second NTFS file is copied over the 2.0000000 ext3 file again. And again.

I don't understand, I touch'ed a dest file on ext3 so it's got tomorrow's date, and cp -p still copies to it, although not updating (downdating?) it's timestamp. Someone said "That sounds very much like a bug, perhaps a different bug."

Anyway, what I've got looks like it will work, so thanks Jean-Pierre, Szaka, Paul Eggert for http://lists.gnu.org/archive/html/bug-c ... 00012.html

regards Garry


Sun Dec 12, 2010 12:37
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.