FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Wed May 12, 2021 02:06



Post new topic Reply to topic  [ 11 posts ] 
Getting EA_ATTR and EA_INFORMATION on a file object 
Author Message

Joined: Thu Feb 27, 2014 22:24
Posts: 3
Post Getting EA_ATTR and EA_INFORMATION on a file object
I'm trying to copy a file(actually it is a directory object) from one ntfs partition to another using ntfs-3g. The directory in question is the C:\Windows\csc\v2.0.6 directory. This directory happens to have windows extended attributes assigned to it. My copy must include these extended attributes in order for the directory to be correct on the target volume.

I can see the extended attributes when I use the ntfsinfo. They show up as $EA_INFORMATION and $EA with all of their correct data.

My problem is that I can not get this information from ntfs-3g. I can not find anywhere in the source where any of the getfattr commands would return this information.

Is there any way that I can retrieve this information, and also just as important, be able to set this information?


Thu Feb 27, 2014 22:36
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Getting EA_ATTR and EA_INFORMATION on a file object
Hi,

Quote:
I can see the extended attributes when I use the ntfsinfo. They show up as $EA_INFORMATION and $EA with all of their correct data.

The EA were designed in ntfs for use with OS2, and Windows did not use them until Windows 8.
Note that the Posix extended attributes are not put into EAs, but in named streams.

Do you have any information on EAs ? How can you tell the data is correct ?
Quote:
Is there any way that I can retrieve this information, and also just as important, be able to set this information?

This has been on my todo list for over a year. So far I only have a single example, and I have no idea how the EAs are stored when there are several extended attributes (and whether this is allowed). Above all, I have no idea about the meaning of the value : does it have to be consistent with some information, what are the consequences of copying an EA to another file, etc. Please share what you know.

My single example so far is : name = "$KERNEL.PURGE.APPXFICACHE" value = 0xd797b2327c5fcd01

Regards

Jean-Pierre


Fri Feb 28, 2014 10:28
Profile

Joined: Thu Feb 27, 2014 22:24
Posts: 3
Post Re: Getting EA_ATTR and EA_INFORMATION on a file object
Thanks for the reply.

EAs are actually being used right now in Windows 7 by the Client Side Services (CSC), also know as the Offline file services. You can see them by doing the ntfsinfo on /windows/csc/v2.0.6 and get their complete details by doing the ntfscat on the same file and ask for the EA and EA_INFORMATION attributes.

The EA_ATTR field is pretty accurate with the data that is contained in the fields. They essentially are a repository of any data you want, named whatever you want and you just need to make sure that if you are adding a new one to a file that you set the offset to the previous EA to the size of the previous EA then add your EA to the end of the list.

There are no real rules about putting an EA on a different file. They usually have no file specific information in them. Instead they are used by other products to store their data. For instance the CSC keeps its database in the EA for the directory v2.0.6. When it is starting up its services or creating a sync object with a server it will look up the data in the EA to ensure to process it correctly. The nice part about this set of EAs is that I believe there are two of them. So they are a good example to use. And they are available on any Windows 7 box. So no special applications needed in order to see them and play with them.

My suggestion initially would be that you treat it like you do the reparse data. Namely provide a simple APIs for getting the entire entry for both the EA and EA_INFORMATION, and then setting the entire entry for both. Not sure exactly what it would mean for setting the data since I have not looked at that aspect of the source code. But getting the data would simply be looking up the data in the ntfs object the same way that the ntfsinfo and ntfscat do. I would add them as new "extended attributes" (I know the name will be confusing), and retrieved using the system.extended_attribute_information and system.extended_attribute_data calls.


Fri Feb 28, 2014 16:22
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Getting EA_ATTR and EA_INFORMATION on a file object
Hi,

Quote:
Instead they are used by other products to store their data. For instance the CSC keeps its database in the EA for the directory v2.0.6. When it is starting up its services or creating a sync object with a server it will look up the data in the EA to ensure to process it correctly.

And there is no Win32 function to read or write EAs (apart from two functions for backing up and restoring a file)... which make the EAs nice places to store malware.
Or is there ?
Quote:
I would add them as new "extended attributes" (I know the name will be confusing), and retrieved using the system.extended_attribute_information and system.extended_attribute_data calls.

Can you please post at least an example with two different pairs of (name, value) stored in the same file ?

Regards

Jean-Pierre


Fri Feb 28, 2014 22:36
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Getting EA_ATTR and EA_INFORMATION on a file object
Hi,

Attached is a patch for a preliminary implementation of the extended attribute system.ntfs_ea to set or retrieve the full set of EA's in a file. There are still a lot of todo's, most of them for processing error conditions.

*edit* patch deleted.

Please try and check whether this suits your needs.

I have found the Win32 functions ZwSetEaFile() and ZwQueryEaFile() to set and retrieve EAs, nevertheless I am still in need of real examples. This would save time guessing what are the underlying requirements (alignments, etc.)

The ability to set or retrieve a single EA might be done (much) later if there is a real need.

Regards

Jean-Pierre


Sat Mar 01, 2014 15:30
Profile

Joined: Thu Feb 27, 2014 22:24
Posts: 3
Post Re: Getting EA_ATTR and EA_INFORMATION on a file object
Jean-Pierre,

Thanks for the update. We are currently under test with it.

Here is dump of an EA from the debugger. This is the EA information for the directory c:\windows\csc\v2.0.6. The directory exists on all Windows 7 file systems. Note that after the dump of the data I have done a dump type of each of the EAs so you can get a feel for what the fields look like. Note that the data in the data field can be anything at all, and is only valid to the creator of the EA. Most times no one else will ever interpret the data.


0: kd> db 0xfffffa80`066ce010 L1d0
fffffa80`066ce010 94 00 00 00 00 31 58 00-43 38 41 30 35 42 43 30 .....1X.C8A05BC0
fffffa80`066ce020 2d 33 46 41 38 2d 34 39-45 39 2d 38 31 34 38 2d -3FA8-49E9-8148-
fffffa80`066ce030 36 31 45 45 31 34 41 36-37 36 38 37 2e 43 53 43 61EE14A67687.CSC
fffffa80`066ce040 2e 44 41 54 41 42 41 53-45 00 06 00 00 02 50 00 .DATABASE.....P.
fffffa80`066ce050 58 00 00 00 00 00 00 00-00 00 00 90 4f 58 00 00 X...........OX..
fffffa80`066ce060 00 00 00 90 4f 58 00 00-00 00 00 00 00 00 00 00 ....OX..........
fffffa80`066ce070 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
fffffa80`066ce080 00 00 00 00 00 00 00 00-00 00 0d b1 12 56 13 f6 .............V..
fffffa80`066ce090 df 11 b4 67 8b a9 f4 05-e1 8f 00 00 00 00 00 00 ...g............
fffffa80`066ce0a0 00 00 00 00 c0 00 00 00-00 34 80 00 43 38 41 30 .........4..C8A0
fffffa80`066ce0b0 35 42 43 30 2d 33 46 41-38 2d 34 39 45 39 2d 38 5BC0-3FA8-49E9-8
fffffa80`066ce0c0 31 34 38 2d 36 31 45 45-31 34 41 36 37 36 38 37 148-61EE14A67687
fffffa80`066ce0d0 2e 43 53 43 2e 44 41 54-41 42 41 53 45 45 58 31 .CSC.DATABASEEX1
fffffa80`066ce0e0 00 06 00 00 02 50 00 58-00 00 ac 3b 42 00 00 00 .....P.X...;B...
fffffa80`066ce0f0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
fffffa80`066ce100 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
fffffa80`066ce110 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
fffffa80`066ce120 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
fffffa80`066ce130 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
fffffa80`066ce140 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
fffffa80`066ce150 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
fffffa80`066ce160 00 00 00 00 00 00 00 00-00 30 38 00 43 38 41 30 .........08.C8A0
fffffa80`066ce170 35 42 43 30 2d 33 46 41-38 2d 34 39 45 39 2d 38 5BC0-3FA8-49E9-8
fffffa80`066ce180 31 34 38 2d 36 31 45 45-31 34 41 36 37 36 38 37 148-61EE14A67687
fffffa80`066ce190 2e 43 53 43 2e 45 50 4f-43 48 45 41 00 06 00 00 .CSC.EPOCHEA....
fffffa80`066ce1a0 02 10 00 38 00 0f b1 12-56 13 f6 df 11 b4 67 8b ...8....V.....g.
fffffa80`066ce1b0 a9 f4 05 e1 8f 00 00 00-00 00 00 00 00 00 00 00 ................
fffffa80`066ce1c0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
fffffa80`066ce1d0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
0: kd> dt fffffa80`066ce010 _FILE_FULL_EA_INFORMATION
+0x000 NextEntryOffset : 0x94
+0x004 Flags : 0 ''
+0x005 EaNameLength : 0x31 '1'
+0x006 EaValueLength : 0x58
+0x008 EaName : [1] "C"
0: kd> dt fffffa80`066ce010+94 _FILE_FULL_EA_INFORMATION
+0x000 NextEntryOffset : 0xc0
+0x004 Flags : 0 ''
+0x005 EaNameLength : 0x34 '4'
+0x006 EaValueLength : 0x80
+0x008 EaName : [1] "C"
0: kd> dt fffffa80`066ce010+94+C0 _FILE_FULL_EA_INFORMATION
+0x000 NextEntryOffset : 0
+0x004 Flags : 0 ''
+0x005 EaNameLength : 0x30 '0'
+0x006 EaValueLength : 0x38
+0x008 EaName : [1] "C"

I'll let you know what the results of our testing show.

Thanks

Chip


Tue Mar 04, 2014 19:08
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Getting EA_ATTR and EA_INFORMATION on a file object
Hi,

Quote:
Here is dump of an EA from the debugger.

This is interesting. I suppose this is a memory dump of what ZwQueryEaFile() returned on Windows (and not a dump from disk), as the last "NextEntryOffset" has been set to zero. In particular this example shows my assumptions about alignments were wrong (so I remove the former patch from this forum).

Attached is a patch for a full implementation of extended attribute functions to get, set and remove EAs.

The main difference from ZwQueryEaFile() and ZwSetEaFile() is that the Linux functions use the on-disk representations, and the "NextEntryOffset" field should always be the aligned size and never zeroed. (Apparently ZwQueryEaFile() does not returned a zeroed field when there is a single EA). I have also attached the result of your example, which shows this difference.

Please report about any problems ASAP, I will not be able to come back to this for a couple of weeks.

Regards

Jean-Pierre


Attachments:
example.gz [291 Bytes]
Downloaded 963 times
ea.patch.gz [3.82 KiB]
Downloaded 1010 times
Wed Mar 05, 2014 11:06
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Getting EA_ATTR and EA_INFORMATION on a file object
Hi,

Please apply the attached patch over the global EA patch posted above

Regards

Jean-Pierre


Attachments:
ntfsinfo.c.patch.gz [252 Bytes]
Downloaded 1023 times
Wed Mar 05, 2014 12:35
Profile

Joined: Wed Mar 12, 2014 21:41
Posts: 3
Post Re: Getting EA_ATTR and EA_INFORMATION on a file object
Jean-Pierre, thanks for your help with this. We needed to make a couple of changes to the EA code in xattrs.c to get the patch to work for us, but it does work successfully now.

Attached is a patch with our changes. Some are bug fixes, a couple of others you may or may not want to take due to design guidelines. Basically, we wrapped a couple of the ntfs_attr_readall() calls in ntfs_attr_exist() in order to avoid a syslog message every time ntfs_get_ntfs_ea() was called for a file/dir that had no EA_INFORMATION attribute - ntfs_attr_readall() logs every time the first ntfs_attr_open() call doesn't find anything.


Attachments:
File comment: This patch applies to xattrs.c that has been already modified with the second ea.patch from 3/5/14.
xattrs.c.patch.gz [860 Bytes]
Downloaded 1063 times
Wed Mar 12, 2014 22:21
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Getting EA_ATTR and EA_INFORMATION on a file object
Hi,

Thanks for the proposed patch. I cannot examine it now, will do later.

Regards

Jean-Pierre


Thu Mar 13, 2014 09:29
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Getting EA_ATTR and EA_INFORMATION on a file object
Hi,

The only problem I see with your patch is, when trying to get a non-existing EA, a zero-sized attribute is returned instead of ENODATA. Thanks for the proposal.

Attached is an updated global patch, to be applied to original source code.

Regards

Jean-Pierre


Attachments:
ea.patch.new.gz [4.06 KiB]
Downloaded 1046 times
Tue Mar 25, 2014 13:23
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 11 posts ] 


Who is online

Users browsing this forum: Google [Bot] and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.