FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Tue Sep 21, 2021 10:17



Post new topic Reply to topic  [ 4 posts ] 
Reading and creating reparse points 
Author Message

Joined: Sun Feb 17, 2013 23:37
Posts: 2
Post Reading and creating reparse points
Hi,

I used to use the ntfs3g tool suit to back up my vista partition at the file level. The backup process went sort of like this:

1. VSS Snapshot and mount vista partition.
2. Create backup using DAR + cygwin from Vista.
3. Run ntfs3g secaudit from Vista.
3. Record all attributes (hidden/system) from Vista.
4. Use self written program to record reparse points (from Vista).

To restore, I used:

1. Restore backup from Linux.
2. Restore security audit from Linux.
3. Once booted into Vista, recreate reparse points from step 4 above.
4. Reset all attributes from Vista.


I had to reinstall Vista a while back. After then, I forgot to take backups.
I resized the partition and Vista refuses to boot off it, even after a chkdsk.

The thing is that the partition can be read perfectly well with Linux. I can copy the files across. I can copy the security and attribute meta data.

The problem is that I don't know to deal with the reparse points from Linux. I can identify reparse points using 'find -type l'. From that I can dump the reparse extended reparse data. However it does not tell me whether the reparse point is a symlink or junction. Unless somebody can tell me how do do that from the binary data outputted from getfattr.

The other problem is that I'm not sure how I can create reparse points with ntfs-3g, if I can. Can I just create blank directories and then set the reparse data on those directories?


Richard


Mon Feb 18, 2013 04:19
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Reading and creating reparse points
Hi,

Quote:
The problem is that I don't know to deal with the reparse points from Linux. I can identify reparse points using 'find -type l'. From that I can dump the reparse extended reparse data. However it does not tell me whether the reparse point is a symlink or junction. Unless somebody can tell me how do do that from the binary data outputted from getfattr.

The reparse points have a tag which identify what they mean (0xA0000003 for a junction, 0xA000000C for a symbolic link), which can be seen as the first four bytes returned by getfattr. There are reparse points which are neither junctions nor symbolic links.
Quote:
The other problem is that I'm not sure how I can create reparse points with ntfs-3g, if I can. Can I just create blank directories and then set the reparse data on those directories?

Reparse points can be set on files or directories. For the ones which translate to a symbolic link, you can create a void file or a void directory (depending on whether the target is a file or a directory), then set the reparse point.

On http://jp-andre.pagesperso-orange.fr/ad ... fs-3g.html there is a downloadable tool.zip file which contains several ntfscp programs or shell commands to copy a directory tree with all the attributes. They will however not copy the boot structure needed for an OS to boot.

Regards

Jean-Pierre


Mon Feb 18, 2013 09:54
Profile

Joined: Sun Feb 17, 2013 23:37
Posts: 2
Post Re: Reading and creating reparse points
Quote:
There are reparse points which are neither junctions nor symbolic links.

Thankfully a standard installation only uses those two types of reparse points. I don't think it creates any other types, unless the user specifically does so.
Quote:
On http://jp-andre.pagesperso-orange.fr/ad ... fs-3g.html there is a downloadable tool.zip file which contains several ntfscp programs or shell commands to copy a directory tree with all the attributes.

Thanks. That ntfscp program is a lifesaver. I managed to copy the data over and Vista was able to boot without any filesystem errors.

The original problem was that I had shrinked my Vista partition with gparted. And after I got a c000021a bsod error. Vista tried to run the autochk program but was unable to do so, due to the fact that it could not mount the partition that autochk was located on. I had also lost the OEM Vista DVD. I only had an XP CD and the chkdsk didn't seem to repair the problem.
Quote:
They will however not copy the boot structure needed for an OS to boot.

As long as you copy over the mbr signature and keep the partition offset the same bootmgr will boot that partition. If you do happen to change those two figures you can always update them in the BCD hives.

I made 2 minor modifications to ntfscp.c (see patch at end of post)

1. Enable large file support. ntfscp would hit an overflow error copying files >2GB. So I changed the IO syscalls to their 64 bit width versions and I used the stat64 struct instead of stat. (Requires flag -D_FILE_OFFSET_BITS=64)

2. Change the buffer size for main data. 64k is fine for the extended attributes, but I think it would take a long time for the large files I was copying, so I ramped it up to 16M. The problem was that the #define used for buffer size was the same as the maximum attribute size, BUFLTH. So I has to malloc a seperate buffer for the main data stream.

Overall, I think your ntfscp is a great tool for recovering ntfs partitions. Especially in cases where users have lost the original installation media or when chkdsk/autochk fails to repair the filesystem inconsistencies.

Even though the documentation says that ntfscp is a demonstration program, I don't see why it can't be released as an offical tool as part of one of the ntfs packages. It seems that the only people who know about ntfscp (not the one in ntfsprogs) are the people who you've told on this forum. I think it is a shame that such a useful tool is given so little exposure.

Do you think it would be possible to move it to the ntfg-3g package or ntfstools package as an "official" tool/program?


Richard

======
Patch:
Code:
--- cmp/ntfscp.c   2011-09-10 16:54:52.000000000 +0100
+++ tools/ntfscp.c   2013-02-19 00:06:53.809425010 +0000
@@ -52,6 +52,10 @@
  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
  */

+#define __USE_LARGEFILE64
+#define _LARGEFILE_SOURCE
+#define _LARGEFILE64_SOURCE
+
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
@@ -95,6 +99,9 @@
BOOL opts; /* skip sparse files */
int optv; /* verbose */

+#define CPBUFSZ 0x1000000
+char* cpBuf;
+
BOOL shouldcopy(const char *source, u32 attrib)
{
   BOOL ok;
@@ -236,7 +243,7 @@
  */

BOOL attribcopy(const char *source, const char *destination, char *buf,
-      const struct stat *pst, const u64 stamps[])
+      const struct stat64 *pst, const u64 stamps[])
{
   struct utimbuf utb;
   BOOL err;
@@ -348,14 +355,14 @@
  */

BOOL actualcopy(const char *source, const char *destination, char *buf,
-      const struct stat *pst, const u64 stamps[], u32 attrib)
+      const struct stat64 *pst, const u64 stamps[], u32 attrib)
{
   int s;
   int d;
   BOOL err;
-   unsigned int count;
-   unsigned int written;
-
+   unsigned long count;
+   unsigned long written;
+       
   if (optv)
      printf("file %s to %s\n",source,destination);
   err = TRUE;
@@ -368,9 +375,10 @@
            err = sparsecopy(s,d,buf);
         } else {
            do {
-               count = read(s,buf,BUFLTH);
+               //count = read(s,buf,BUFLTH);
+               count = read(s,cpBuf,CPBUFSZ);
               if (count)
-                  written = write(d,buf,count);
+                  written = write(d,cpBuf,count);
            } while (count && (written == count));
            err = count && (written != count);
         }
@@ -386,7 +394,7 @@
            destination);
      close (s);
   } else
-      fprintf(stderr,"*** Could not open %s\n",source);
+          fprintf(stderr,"*** Could not open %s\n%s\n",source, strerror(errno));
   return (err);
}

@@ -397,7 +405,7 @@

BOOL filecopy(const char *source, const char *destination,
      const char *parentdest, char *buf,
-      const struct stat *pst, const u64 stamps[], u32 attrib)
+      const struct stat64 *pst, const u64 stamps[], u32 attrib)
{
   u32 dirattrib;
   u32 revattrib;
@@ -454,9 +462,9 @@
      const char *parentdest, char *buf);

BOOL dircopy(const char *source, const char *destination, char *buf,
-      const struct stat *pst, const u64 stamps[])
+      const struct stat64 *pst, const u64 stamps[])
{
-   struct stat st;
+   struct stat64 st;
   char *innersrc;
   char *innerdst;
   struct dirent *de;
@@ -466,7 +474,7 @@
   if (optv)
      printf("dir %s to %s\n",source,destination);
   err = FALSE;
-   if (!stat(destination,&st)
+   if (!stat64(destination,&st)
      /*
       * make sure the directory is created writeable, even
       * if a default ACL says otherwise.
@@ -525,9 +533,9 @@
  */

BOOL linkcopy(const char *source, const char *destination, char *buf,
-      const struct stat *pst, const u64 stamps[])
+      const struct stat64 *pst, const u64 stamps[])
{
-   struct stat st;
+   struct stat64 st;
   int f;
   u32 attrib;
   BOOL err;
@@ -556,7 +564,7 @@
         } else {
            if (mkdir(destination,
                  pst->st_mode & 07777)) {
-               if (lstat(destination,&st)
+               if (lstat64(destination,&st)
                  || (((st.st_mode & S_IFMT)
                     != S_IFDIR)
                     && ((st.st_mode & S_IFMT)
@@ -611,7 +619,7 @@
BOOL copy(const char *source, const char *destination,
      const char *parentdest, char *buf)
{
-   struct stat st;
+   struct stat64 st;
   u64 stamps[3];
   u32 attrib;
   int size;
@@ -620,7 +628,7 @@

   err = FALSE;
      /* collect times before making any other access */
-   s = lstat(source,&st);
+   s = lstat64(source,&st);
   if (!s
      && (lgetxattr(source,"system.ntfs_times",(char*)stamps,24) == 24))
      switch(st.st_mode & S_IFMT) {
@@ -650,9 +658,11 @@
         break;
      }
   else {
-      fprintf(stderr,"*** Cannot access %s\n",source);
+          fprintf(stderr,"*** Cannot access %s\nErro: %s\n",source, strerror(errno));
      err = TRUE;
   }
+        fflush(stdout);
+        fflush(stderr);
   return (err);
}

@@ -729,6 +739,7 @@

   err = TRUE;
   xarg = getoptions(argc, argv);
+        cpBuf = malloc(sizeof(char) * CPBUFSZ);
   if (xarg && (xarg == argc - 2)) {
      buf = (char*)malloc(BUFLTH);
      parentdest = (char*)malloc(strlen(argv[xarg+1]) + 1);


Thu Feb 21, 2013 04:21
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1286
Post Re: Reading and creating reparse points
Hi Richard,

Quote:
The original problem was that I had shrinked my Vista partition with gparted. And after I got a c000021a bsod error.

Gparted relies on ntfsresize for shrinking ntfs partitions. If this occurred with a recent ntfsresize, I am interested in the details.
Quote:
I made 2 minor modifications to ntfscp.c (see patch at end of post)

Thanks for sharing.
Quote:
Overall, I think your ntfscp is a great tool for recovering ntfs partitions. Especially in cases where users have lost the original installation media or when chkdsk/autochk fails to repair the filesystem inconsistencies.

This goes beyond expectations, ntfscp was not designed to copy a full file system, ntfsclone is safer for that purpose.
Quote:
Do you think it would be possible to move it to the ntfg-3g package or ntfstools package as an "official" tool/program?

Well, ntfscp is fine for copying user files, but it will quickly show its limits for copying system files (for instance when getting/setting DOS names on hardlinked files, very common since Windows 7). Ntfsclone is safer. You may also consider wimlib which uses a Microsoft format.

Regards

Jean-Pierre


Thu Feb 21, 2013 20:17
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.