Dual Hibernation, almost! How 2 force unmount/close handles?
Hello again!
I have started this new topic because the previous thread (HIBERFILE.SYS) had many incorrect information by me. If you are Jean-Pierre you can just skip to the Problem: - Question: at the end. For others -- here is to re-cap, so far...
Goal:
* To hibernate both Windows and Ubuntu. Switch between them - DONE
* To share my D: Data drive between Windows and Ubuntu - Almost ? (with some caveats)
My setup:
Platforms: Windows 8.1, Ubuntu 14.10
Shared D:\ "Data" drive in the middle - The D: drive is formatted to NTFS filesystem
So Far we have discovered that before hibernation on Windows side, the D:\ data NTFS volume must be fully unmounted and taken offline with the command:
mountvol.exe D: /P
/P is necessary because the /D switch only removes the drive letter / directory mappings. /D switch did not actually dismount the volume (for an internal disk).
mountvol.exe /?
/P Removes the volume mount point from the specified directory,
dismounts the volume, and makes the volume not mountable.
You can make the volume mountable again by creating a volume
mount point.
We also found that after using /P the volume *IS* mountable again afterwards. Contrary to the help text saying it cannot. It will re-mount very easily in normal way. No issues.
Now. Some limitations in Windows side. Caveats.
Giving enough time to dismount:
In Windows, we can only listen for WM_POWEREVENT. That is the only notification (i am aware of) which is saying we are going to hibernate (OR sleep). On Windows 8 and 8.1 we are told that there is only 2.0 seconds from receiving this message until the system goes into hibernation. I have tested this and can say that is very true. Although maybe in practice it can sometimes be a little more than 2.0 seconds, not more than 2.5 sec maximum. IN THEORY we might use SetThreadExecutionState to delay hibernation further until we are truly finished dismounting out NTFS volumes. Like this:
DllCall("SetThreadExecutionState","UInt",ES_CONTINUOUS | ES_SYSTEM_REQUIRED | ES_DISPLAY_REQUIRED | ES_AWAYMODE_REQUIRED)
MsgBox, Press OK to allow sleep
DllCall("SetThreadExecutionState","UInt",ES_CONTINUOUS)
However I have tried this and it does not work for our needs. Because Windows has already started serving applications the WM_POWEREVENT notification. It is too late. We can block hibernation only before the hibernate action is initiated. And of course we do not before time when that is going to happen. So we must ensure that all our neededntfs volumes are dismounted within 2 seconds. Otherwise the filesystem will be left in a bad state when the memory is frozen (still unmounting.
Recommendations:
For this reason we recommend NOT to use slow disk for this (for example over USB interface). And to only use ONE big shared NTFS data disk shared between the 2 OSES. And not trying to be sharing or unmounting 2+ NTFS disks with this method. In mountvol.exe /? help Microsoft says it will take care of forcing closed all open file handles by applications. And guarantees closing (not in a specific time). In my early test the /P (unmount+ take offline) happens fast enough and seems to work OK. However I have not tested with a heavily loaded system. Anyway it is best to wait for idle times before choosing to hibernate windows. IF there is some delay unmounting, then should be safe as in linux ntfs-3g will probably detect the disk was not cleanly dismounted yet.
Windows - Distinguishing between hibernation and sleep:
Unfortunately this is not possible either. I have spent 1 whole day just trying to do this. What we find is that there is a KernelPower info msg written to the SystemEventLog. Which can be read using the cmdline utility 'wevtutil.exe'. For example the following command will tell us if there was such a new event written to the windows event log within the last 15 seconds:
# targetstate: hibernate=5 sleep=4
wevtutil.exe qe System "/q:*[System[Provider[@Name='Microsoft-Windows-Kernel-Power'] and (Level=4) and TimeCreated[timediff(@SystemTime) <= 15000]]] and *[EventData[Data[@Name='TargetState']=5]]"
The result is that if we run the above command immediately after receiving the WM_POWEREVENT, then unfortunately there is nothing in the event log yet. After windows is resumed from it's hibernation, only then do we see the log message has been written. By that time we must have already needed to dismount the volume. SO the implications is it is possible to distinguish between resuming from either hibernation OR sleep. But not entering hibernation OR sleep. The conclusion is we must mount / dismount out NTFS volume regardless for both of these power states. Ideally we would avoid this on sleep because it is only needed for hibernation. Bah ^*&$ M%£&Soft. Luckily if we do not sleep very often, then it does not matter. And the added unmount / remounting does not negatively impact the sleeping times too much since the system waits 2 seconds anyway.
===
In windows, install autohotkey program. version 1.16+. Then open Task Scheduler program, and add a system task with fill admin privelidges. To open and run this .AHK autokotkey file (NOT .exe). The task can be started at windows startup and run as 'system' account, een when no user is logged in. This ensures that the Data drive will be dismounted even hibernating from the login screen.
YOU MUST REPLACE THE ntfs VolumeName UUID and Drive letter of your own shared NTFS Data drive (cannot be C drive!). use mountvol.exe /? to list them.
hibernate_hook.ahk:
; hibernation hook
#SingleInstance, force
; Listen to the Windows power event "WM_POWERBROADCAST" (ID: 0x218):
OnMessage(0x218, "func_WM_POWERBROADCAST",10)
Return
; on sleep - 4, on wake 18, 7
; on hibernate - 4, on resume 18, 7
func_WM_POWERBROADCAST(wParam, lParam)
{
If (lParam = 0) {
If (wParam = 4) ;PBT_ APM SUSPEND
{
objShell := ComObjCreate("WScript.Shell")
objExec := objShell.Exec("mountvol.exe D: /P")
Return
}
Else If (wParam = 7) ;PBT_APM RESUME SUSPEND
{
objShell := ComObjCreate("WScript.Shell")
objExec := objShell.Exec("mountvol.exe D: \\?\Volume{b03e36d5-51f3-11e4-8263-001eecd73a1f}\")
Return
}
; Else If (wParam = 18) ;PBT_ APM RESUME AUTOMATIC
; {
; objShell := ComObjCreate("WScript.Shell")
; objExec := objShell.Exec("mountvol.exe D: \\?\Volume{b03e36d5-51f3-11e4-8263-001eecd73a1f}\")
; }
}
Return
}
And that takes care of windows hibernation site. Now for the linux side, in Ubuntu.
LINUX SIDE:
We also need a counterpart script for linux hibernation. This still has an issue and might not be correct yet. Here is a temporary link:
https://gist.github.com/anonymous/d12357711ad679fb4d06Where "/Data" disk entry is wrtten in the fstab file /dev/sda9 the same NTFS D: drive as we have on Windows 8.
If no file handles are open on the disk at the time of linux hibernation, then the first command "umount /Data" will call ntfs-3g driver and unmount the ntfs disk cleanly. No problems whatsoever. I can hibernate between Linux and Windows many times and all Data is preserved.
Problem:
However the entire point of hibernation is to be able to resume running applications / have files open in the workspace. So what is really needed is for the linux ntfs driver to do for me (as 'root' user) is the same sort of thing as what mountvol.exe /P on Windows will do: The un-mount command must force-close all currently open file handles / file descriptiors and invalidate them. Flush current writes but make any new handles impossible to create. And further disk writes from those previous handles impossible / invalid / ignored.
Unfortunately in this situation with open file handles ('lsof /dev/sda9'). Then the normal 'umount /Data' command refuses to unmount forcefully. And the --lazy option seems not to work re-mounting afterwards (some bizarre error message).
Searching stackoverflow for that bizarre error message (caused by --lazy), I have then found this interesting command instead:
fusermount -uz "/Data"
My Question:
Although I don't understand what such 'fusermount' command is doing at all. But seems a bad idea because the disk unmounts. The mountpoint disappears from 'df' command. However if I have an open Bash window, then it still seems to have a handle on the stale directory. And will ls, mkdir will can continue to write data on to the NTFS disk. This very bad and results in corruption of the ntfs disk (after resuming the session from hibernation, when the bash window or whatever application continues to be running). This looks like some limitation in ubuntu's ntfs driver - unless there is some way around the problem?
In an ideal world, I would like to be able to run some command as root that force-closes all open handles + unmounts the NTFS volume immediately. Making it non-writable. Otherwise such a busy application may immediately re-open a new handle to write to. Is there any way on Linux I can do such things? (reliably!)
Or if the function does not exist today, can it ever be added in the future to such FUSE user-land drivers ?
Many thanks for any input / comments.