Hard Light Productions Forums
Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Topic started by: jr2 on August 07, 2012, 11:54:35 pm
-
Ran across this whilst trying to look up the VirtualStore location, thought I'd run it by here... is this something FSO should use? What are you guys' thoughts (sorry if I'm being a clueless n00b, feel free to tell me if this isn't relevant to FSO's development).
http://windowsteamblog.com/windows/b/developers/archive/2009/08/04/user-account-control-data-redirection.aspx
User Account Control Data Redirection
To making sure your application is ready (compatible) for Windows 7, I am going to start providing additional information about the few compatibility topics we introduced in the “Is Your Application Ready for Windows 7 RTM? (http://windowsteamblog.com/blogs/developers/archive/2009/07/23/is-your-application-ready-for-windows-7-rtm.aspx)” post. This post focuses UAC Virtualization or more known as Data Redirection.
What Are You Talking About?
Today, many applications are still designed to write files to the Program Files, Windows directories, or system root (typically the C drive) folders. Some applications are designed to update Microsoft® Windows registry values, specifically values in HKLM/Software. But there is one problem: the files or registry values are not created or updated. You may ask, “What’s going on? My application goes through the code and does not report an error. So where are my files?”
To be specific, you may experience one or more of the following symptoms:
- Your application writes to Program Files, Windows directories, or the system root (typically the C drive) folders, but you can't find your files in these locations
- Your application writes to the Windows registry, specifically to HKLM/Software, but you can't see the registry updates
- You switch to another user account, and your application is unable to find files that were written to Program Files, Windows directories, or the system root (typically the C drive) folders, or it finds older versions of these files
- After turning User Account Control (UAC) on or off, your application is unable to find files in the Program Files or Windows directories
If this happens to your application, it is experiencing UAC Virtualization (AKA Data Redirection). The following information provides you with everything you need to know in order to detect this application compatibility problem, offers some solutions, and provides additional information about the specific nature of the compatibility problem.
The Real Issue: UAC Virtualization
Prior to Windows Vista, administrators typically ran applications. As a result, applications could freely read and write system files and registry keys. If standard users ran these applications, they would fail due to insufficient access. Windows Vista improved application compatibility for standard users by redirecting writes (and subsequent file or registry operations) to a per-user location within the user’s profile.
For example, if an application attempts to write to C:\Program Files\Contoso\Settings.ini, and the user does not have permissions to write to that directory (the Program Files), the write operation will be redirected to C:\Users\Username\AppData\Local\VirtualStore\Program Files\Contoso\settings.ini. If an application attempts to write to HKEY_LOCAL_MACHINE\Software\Contoso\ in the registry, it will automatically be redirected to HKEY_CURRENT_USER\Software\Classes\VirtualStore\MACHINE\Software\Contoso or HKEY_USERS\UserSID_Classes\VirtualStore\Machine\Software\Contoso.
The following figure illustrates the two components of the Windows Virtualization process: file virtualization and registry virtualization:
(http://i46.tinypic.com/2yxgrah.gif) (http://msdn.microsoft.com/en-us/library/bb756960.aspx)
To learn more about UAC virtualization and new UAC technologies, see “New UAC Technologies for Windows Vista” at http://msdn.microsoft.com/en-us/library/bb756960.aspx (http://msdn.microsoft.com/en-us/library/bb756960.aspx).
Solution
Virtualization is intended only to assist in application compatibility with existing programs. New applications designed for Microsoft Windows 7 should NOT perform write operations to sensitive system areas, nor should they rely on virtualization to provide redress for incorrect application behavior. Always develop applications for use with standard user privileges and don’t count on the application running under administrator privileges. Test your application with standard user privileges and not administrator privileges.
If you are experiencing UAC virtualization with applications developed prior to Windows 7, re-design your applications to write files to the appropriate locations. When updating existing code to run on Windows 7, you should:
- Ensure that during run-time, applications store data only in per-user locations or in computer locations within %alluserprofile% that have properly set access control list (ACL) settings. For more information about ACLs, see Access Control Lists (http://msdn.microsoft.com/en-us/library/aa374872.aspx).
- Determine the known folder to which you want to write the data files. Generic data used by all users should be written to a global public location that is shared by all users. All other data should be written to a per-user location.
- Generic data files can include, but are not limited to log files, settings files (INI/XML), saved state applications such as saved games, and so on
- User documents are different; they should be saved to the Documents folder (or to a location the user specifies)
- Ensure that you do not hard-code paths once you have determined the appropriate locations. Instead, use one of the following programming models and APIs to retrieve the correct paths of specific Windows known folders:
- C/C++ native applications: Use the SHGetKnownFolderPath (http://msdn.microsoft.com/en-us/library/bb762188(VS.85).aspx) function that retrieves the full path of a known folder identified by the folder's KNOWNFOLDERID (http://msdn.microsoft.com/en-us/library/bb762584(VS.85).aspx), a GUID parameter indicating the known location you would like to obtain:
- FOLDERID_ProgramData – Shared program data directory for all users
- FOLDERID_LocalAppData – Per-user program data directory (non-roaming)
- FOLDERID_RoamingAppData – Per-user program data directory (roaming)
- Managed Code: Use the System.Environment.GetFolderPath (http://msdn.microsoft.com/en-us/library/system.environment.getfolderpath.aspx) function. GetFolderPath takes a parameter indicating the Known Location you would like to obtain
- Environment.SpecialFolder.CommonApplicationData – Shared program data directory for all users
- Environment.SpecialFolder.LocalApplicationData – Per-user program data directory (non-roaming)
- Environment.SpecialFolder.ApplicationData – Per-user program data directory (roaming)
- If none of the above-mentioned options are available, use the environment variable:
- %ALLUSERSPROFILE% – Shared program data directory for all users
- %LOCALAPPDATA% – Per-user program data directory (non-roaming) - Windows Vista or later
- %APPDATA% – Per-user program data directory (roaming) - Windows Vista or later
Steps to Determine the Most Appropriate Solution
So far, we have presented the symptoms associated with UAC virtualization, explained why redirection is taking place, and suggested a solution. This section contains tests and procedures you should perform in order to pinpoint the real problem and plot a way to resolve it.
Test #1:
- Launch the application with standard user privileges
- Run through the scenario that results in a write operation to any given protected folder such as the Program Files or system root (C:\) directories
- Expected results: The application “succeeds” in writing the file to the protected folder; however, you can't find the file in the expected location
- Conclusion: This suggests that UAC data virtualization is redirecting the file to a different location
Test #2:
- Using Windows Explorer, search for your files in the VirtualStore folder
- The VirtualStore folder is a folder in your profile which stores redirected files
- The VirtualStore folder’s name and location are subject to change in later versions of Windows
- First, attempt to find it under %localappdata%\VirtualStore
- If you are unable to find it there, try issuing dir %userprofile%\yourfile.dat /s /a at a command line (usually found C:\Users\<user name>\AppData\Local\VirtualStore)
- Expected results: You should find your file at the virtual store folder or sub folders
- Conclusion: This is proof that UAC virtualization is taking place
Test #3:
- Log in as an administrator and launch your application with administrator privileges.
- To launch the application with administrator privileges, right-click the file executable and select Run as Administrator
- When an application runs with administrator privileges, virtualization is turned off, and the application now has sufficient privileges to write to the protected folders
- Do not rely on permanently marking your application to require administrator privileges as a way to bypass virtualization, as doing so will leave the system more vulnerable to attack, will prevent the application from running with limited or standard user privileges, and will needlessly annoy users with a UAC prompt
- Expected results: The application succeeds in writing the file to the protected folder, and you can find the file at the expected location
- Conclusion: Running the application with administrator privileges turns off virtualization and grants privileges
Test #4:
- Launch Process Monitor (ProcMon)
- To learn more about Process Monitor, visit the Windows Sysinternals Process Monitor Web site (http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx) on Microsoft TechNet
- During the hands-on lab, we will walk you through a detailed step-by-step process to use Process Monitor
- Configure filtering to show only file operations and only those operations performed by your process
- When you launch Process Monitor, the Process Monitor Filter dialog box appears
- Add the following entry to the filter: "Column=Process Name, Relation=is, Value=YourApp.exe, Action=Include"
(http://i48.tinypic.com/jagdv7.png)
- To invoke the Process Monitor Filter dialog box again, click this toolbar button (http://i46.tinypic.com/55koet.png)
- After clicking OK, configure Process Monitor to log only file events by enabling the following toolbar buttons as shown: (http://i50.tinypic.com/5b2wye.png)
- Press CTRL-X to clear the log; press CTRL-E to toggle logging on/off
- Expected results: You will see file operations whose results are REPARSE; the next line (with result SUCCESS) will usually be the redirected operation:
(http://i50.tinypic.com/10zvmaf.png)
- Double-click the line representing the operation with REPARSE as the result and click the Stack tab to show the call stack at the time of the operation:
(http://i48.tinypic.com/27xowu0.png)
- The pink K and blue U letters on the left of each stack frame show whether the stack frame is in kernel mode (K) or in user mode (U);in this case, we are interested only in user-mode stack frames
- In this example, the SaveFile function (at frame 21) in BrokenAppNative.exe is the one performing the operation which will be redirected
- You should configure symbols for a more meaningful display' for more information about configuring symbols, refer to the Debugging Tools for Windows Web site (http://www.microsoft.com/whdc/devtools/debugging/default.mspx)
- Conclusion: This test proves that UAC Virtualization did take place and shows you what operations in your application need to be corrected
Test #5:
- Add a manifest to your application which contains a UAC directive
- This will mark your application as UAC-aware and will disable UAC virtualization
- A manifest is an XML document that developers embed as a resource in a DLL or .exe file, but can be a standalone file named YourApp.exe.manifest or YourDLL.dll.manifest
- Manifests can contain a variety of information that usually pertains to application compatibility, such as the exact version of Visual C++ runtime to load, the version of Common Controls Library to load, as well as UAC settings
- Read more about UAC settings in the manifest in the Windows Vista Developer Story: Create and Embed an Application Manifest (UAC) (http://msdn.microsoft.com/en-us/library/bb756929.aspx)
- Expected results: The application now fails to write to any of the protected folders, returning an “access denied” error
- This is a GOOD result, because the UAC data virtualization didn’t kick into action
- As the developer, you should be able to recognize this (because you marked the application as UAC-aware in the manifest) and avoid writing to any of the protected areas
- Conclusion: Windows enables virtualization because the application isn’t marked as UAC-aware. Marking your application as UAC-aware disables virtualization. If your app tries to write to protected store while marked as UAC-aware, you will get an access denied exception
Hands On Labs and Additional Material
you can download (http://windowsteamblog.com/cfs-file.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/developers.CodeSamples.AppCompat/Data-Redirection.zip) this doc, a presentation describing UAC Virtualization, and two full hands on labs on this topic (http://windowsteamblog.com/cfs-file.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/developers.CodeSamples.AppCompat/Data-Redirection.zip), one for managed code and one for native, from here (http://windowsteamblog.com/cfs-file.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/developers.CodeSamples.AppCompat/Data-Redirection.zip).
Tools
In order to detect UAC virtualization we use the following tools:
- Process Monitor, a free and advanced Microsoft tool that monitors and logs file system, registry, network, and process activity
- To learn more about Process Monitor, see the Process Monitor page (http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx) at the Windows Sysinternals Web site
- Download Process Monitor from Microsoft TechNet (http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx)
- Standard User Analyzer, part of the Microsoft Application Compatibility Toolkit, is a free tool that monitors resource (file, registry, and others) usage of a given application and reports activity that is responsible for Standard User problems
- To learn more about Standard User Analyzer see “Standard User Analyzer” (http://technet.microsoft.com/en-us/library/cc766021.aspx) at Microsoft TechNet
- To obtain Standard User Analyzer, download the Microsoft Application Compatibility Toolkit (http://www.microsoft.com/downloads/details.aspx?FamilyID=24da89e9-b581-47b0-b45e-492dd6da2971&displaylang=en)
Additional Resources
- Windows Vista Application Compatibility: UAC: Standard User Changes – http://msdn.microsoft.com/en-us/library/bb963893.aspx
- Common file and registry virtualization issues in Windows Vista – http://support.microsoft.com/kb/927387
- New UAC Technologies for Windows Vista – http://msdn.microsoft.com/en-us/library/bb756960.aspx
- Inside Windows Vista User Account Control – http://technet.microsoft.com/en-us/magazine/2007.06.uac.aspx
-
Don't think this is an issue unless you absolutely must install FSO into Progam Files, and no other folder will satisfy your ALL APPLICATIONS ARE INSTALLED IN PROGRAM FILES! OCD/fetish. :P
-
Speaking as someone who uses fs2o primarily on linux, why bother with this?
-
That.. is a very good question.
-
FSO does not really need to be aware of UAC. We'll probably move away from the "put everything in FS2DIR/data or the registry" model on Windows to something similar to what we're already doing on MacOS and Linux anyway, that way, all those problems are going to go away.
Not to mention that the only reason why we advise people not to put FSO into Program Files is because it makes modding much simpler.
-
I was referring to this problem:
Having FS2 in \Program Files means that your log file was in (as I posted before) \Users\<username>\AppData\Local\VirtualStore\Program Files (x86)\GOG.com\FreeSpace2\data\fs2_open.log
This is because programs are not allowed to write to the program files directory unless they have administrative permission.
-
It's a problem only to people that are noobish enough to keep installing stuff in Program Files. Especially old games. What year is this, really ?
-
The year of GOG.com default installation locations? :P
-
It's a problem only to people that are noobish enough to keep installing stuff in Program Files. Especially old games. What year is this, really ?
Assuming that being newbie-friendly isn't a worthy goal in itself is a good way to end up a shrinking dispossessed minority in most things in life, y'know.
-
It's a problem only to people that are noobish enough to keep installing stuff in Program Files. Especially old games. What year is this, really ?
The year when pretty much everything installs to Program Files by default, and why bother changing it? :p
(I'm still on XP, though, so I blissfully run everything as admin and ignore the gaping security holes.)
-
Mattthegeek:
Usability is a good thing to have. It's nice that you think that obscure paths are not a big deal, but be aware that we are not writing a program for the supergeek user subset.
Please, for the love of kittens, at least try to understand that arrogance is not a nice trait, and that making our community exclusionary in nature is not a good thing.
As for the overall discussion: It is teh pointless. Yes, we know that UAC's folder virtualization has its issues. Yes, we are aware of ways to get around it. Yes, we do plan on implementing them at some point.
Oh, and jr2? There's this section from the big FSO FAQ that was written to address this very thing: *NOTE FOR PEOPLE USING WINDOWS VISTA OR 7:
If you installed FS2 to C:\Program Files\ (or C:\Program Files (x86)\ on 64-bit versions of Windows), the log file will not be in FS2\data\, but rather in a "virtualized" folder. This can be found by either going to the data folder, and clicking on the "Compatibility Files" button in the Explorer window, or by manually navigating to C:\Users\<Your Username>\AppData\Local\VirtualStore\Program Files (x86)\<Rest of the path to your FS2 directory>\data
-
If the GOG.com installer installs by default to that location, we could nicely ask them to change it. We've had a line of dialog with them before, y'know.
-
jr2: yes, FSO will be UAC-aware once we get to it.
This topic (http://www.hard-light.net/forums/index.php?topic=80191.msg1590044#msg1590044) has come up several times before. The answer is the same as the last time you asked.
-
This topic (http://www.hard-light.net/forums/index.php?topic=80191.msg1590044#msg1590044) has come up several times before. The answer is the same as the last time you asked.
Oops... :nervous: I thought I had brought up a similar topic, however I guess it was rather the same topic. I should have searched rather than relying on memory. :nono: My apologies. :(