Windows nt virtual memory

Every time Windows NT creates a new process it associates a new empty address space with that process. Normally, some code, typically from an executable image file, is then placed in that address space. Threads are in turn associated with a particular process. Hence, there can be many threads within a single process, all sharing the same address space. Note that each of these threads is running independently, and hence can independently generate page faults within this common address space. Once Windows NT intercepts the page fault and captures the necessary information such as where the fault occurred and why , it transfers control to the Memory Manager.

Inside Windows NT, page faults occur for a number of reasons. One very common reason is that the virtual page is not currently resident in physical memory. In this case, the Memory Manager is responsible for allocating a physical page and retrieving the correct contents from wherever it is actually stored. Typically, this is on a disk somewhere, although it could be via some remote connection to a file server.

Whether a page is resident is indicated in its page table entry using the Valid bit. Recall that in the Intel example only 20 bits of the 32 bit virtual address were actually needed to locate the physical page, since the last 12 bits of information were the offset within the page. Thus, the other 12 bits of information can be used to keep track of details about the particular physical page such as if the page is valid or not. In the cases where the page is not valid, the hardware cannot actually associate any meaning with any of the other bits.

Returning to our page fault example, the Memory Manager uses the other 31 bits of information to determine where the page is actually located.

For example, were the data inside the paging file, this information would identify which paging file and where within the paging file. Given this information the Memory Manager can retrieve the data from the disk. This points to two of the vital roles of the Memory Manager: handling page faults and allocating physical memory.

In addition, the Memory Manager is also responsible for managing address spaces, the underlying memory management hardware, and even the mapping of files as if they were pieces of memory! In addition to supporting all of these features, the Memory Manager is also responsible for doing this very efficiently, because speed within the VM system is critical to the overall performance of the operating system.

This makes the actual implementation of the VM system both vital and complex. Indeed, of all of the pieces of the operating system, the Memory Manager is the single most complex piece. Virtual Address Spaces. At this point, then, with a basic description of why Virtual Memory is important, we can turn our attention to describing the Windows NT Memory Manager.

In Windows NT 4. This access is enforced by the underlying hardware using some of those precious "spare bits" in the page table entry. All pages are assumed to be accessible from kernel mode, although even from kernel mode pages need not be in use, which is different than not being valid since valid means "resident in physical memory". Thus, the 4GB virtual address range of the computer system is divided into two pieces.

One piece represents those addresses that can be accessed from user mode. The other piece represents everything else. Any time the operating system is about to run a thread in a different process, it must switch out the set of mapping tables it is using to translate virtual to physical addresses. As a convenience for the operating system, only some of the virtual to physical mappings are actually changed. The benefit of this approach is that any virtual addresses within this constant range are valid in any process context.

This feature proves to be very useful when writing system software. Very recently, Microsoft introduced Enterprise Server, a new version of Windows NT that allows you to "tune" the allocation of virtual addresses between user mode and kernel mode.

Figure 2 graphically describes this new memory division. The fundamental difference is that the range of addresses valid in arbitrary process context is now much smaller. Correspondingly, the range of addresses valid in a specific process context is now much larger.

For some specialized applications, such as databases, the ability to have a larger address space allows them to operate much more efficiently. Additionally, perhaps you have read or heard a bit about some of the features that will be present in Windows NT Version 5. One of those features is the extension of the virtual memory address space so that it now allows up to 32 GB of virtual memory, rather than the older 4GB limitation.

In theory, it has always been possible to have more than 4GB of physical memory, but just not very common. The future is likely to make this even more common. Changing to a 64 bit virtual address from a 32 bit virtual address has a profound effect on the operating system. During the transition, the operating system will need to facilitate working with both applications and systems software to ensure backwards compatibility.

Eventually, of course, the operating system will be fully 64 bit enabled and the backwards compatibility will not be necessary. In NT 5. The virtual address space has been extended to support 64 bit virtual addresses for "aware" application programs. Internally, the Memory Manager provides support to use 35 bits of the total 64 bit virtual address space. To preserve compatibility with existing applications and systems software the additional 28 GB of virtual address space begins at 4GB and continues to addresses of up to 32GB.

This is shown graphically in the Figure 3. Of course, like anything written about Windows NT 5. The maximum working set quota cannot be exceeded. When a process tries to add more pages than allowed by its maximum working set quota, the system will trim out old pages to make room for the new pages.

Thus, no single process is allowed to consume all available physical memory. Of course, as we mentioned before, these values are dynamic. The Memory Manager monitors the behavior of the system and of the specific process and adjusts these quotas as necessary. For example, a process that is generating an excessive number of page faults will be allowed to grow, in hopes of decreasing the number of page faults it is generating.

The limits on the working set size of a process or the Cache Manager, which has its own specific working set are important to ensure that no single process steals too much memory and thus makes all other processes suffer as a result.

However, by itself, such limits are not enough to guarantee that physical memory is shared fairly between processes. To further facilitate this process, the kernel actually has a background worker thread known as the Balance Set Manager that is responsible for performing a variety of resource pruning tasks. One of these is trimming the working set of all the processes that might be consuming physical memory. The Balance Set Manager runs periodically as needed. Thus, when the system isn?

However, once the demands for physical memory exceed the available pool, the Balance Set Manager will be called upon to? Each process is analyzed by the Balance Set Manager and as necessary it will trim the working set of the particular process. As we discussed earlier in this article, each process has a minimum working set quota and the Balance Set Manager may reclaim physical pages of memory from the process although it will not normally decrease the amount of physical memory below that minimum working set quota.

In fact, the reclamation of pages is done using a two step process that attempts to ensure that only physical pages that are relatively unimportant are actually reclaimed from a process. As the Balance Set Manager scans through the working set of the given process, each page table entry is examined, and the Balance Set Manager looks for pages that have the?

Such pages can be reclaimed by the Balance Set Manager by making the page transitional that is, no longer? If the page has the? Under certain circumstances, the Balance Set Manager will reclaim all the physical memory for a process.

This occurs when the process has not been active for a period of time. In these circumstances, the Balance Set Manager will reclaim all of the physical pages of the process.

If that process is then later rescheduled, the Memory Manager must re-create the data for those physical pages and read it in from disk. For example, the Winlogon program where you type your name and password, and choose a domain to log into the system is typically paged out of memory since it is not normally needed.

When a physical page is reclaimed by the operating system, something must be done with the data on that particular page. For some pages, the data is already stored in a file on the disk. For example, the executable code for a particular program is stored in a file located on a disk somewhere either locally or across the network.

However, application programs also create dynamic information that must be stored in a safe location whenever the physical page containing the data is reclaimed. Clearly, this cannot be stored in the original executable file because it is shared with all running instances of the same program. Thus, the Memory Manager uses paging files to store such data. A paging file is nothing more than a disk-based file under the exclusive control of the Memory Manager.

While the file systems allow the paging file to be called using any name, the Memory Manager uses the name pagefile. Indeed, the Memory Manager can even change the size of the paging file, if needed, to increase the amount of disk space and hence the amount of?

As we discussed in the first part of this article The NT Insider, Volume 5, Number 2 the page table entry for a given process may actually describe where the data is located within the paging file. Thus, if the application program accesses that page, the page itself will be marked as invalid but the Memory Manager will notice that this page must be stored in the paging file.

In that instance, the Memory Manager can then interpret the remaining 31 bits of the page table entry to tell it where the data is located within the paging file. This is graphically demonstrated in Figure 2. Indeed, the existing Memory Manager implementation allows Windows NT to use more than a single paging file, although the file systems do limit support to one paging file per file system volume.

As shown in Figure 2 , there are four bits reserved for describing which paging file is being used for a given entry? As we mentioned earlier, some data is restored directly from the original file from which it came. All such files are accessed using a memory mapping technique, where the file appears as if it were entirely in memory, even though it is paged in from disk as needed.

A memory mapped file could be read-only, as it is for executable programs, or modifiable as it is for data files. There are actually a number of ways that a memory mapped file can be accessed read-only, read-write, or for execution , but the Memory Manager groups these into two distinct types of access.

One type of access used for executables uses copy-on-write technology to ensure the original file is not modified although the application can change its own data and the other type of access used for data files simply writes changes back to the original file. While changing the code in an executable program would seem to be uncommon, in fact it occurs regularly and thus it is necessary for Windows NT to support it.

This ensures that applications relying upon the ability to modify code work properly. One example of such an application is a debugger. Debuggers typically create breakpoints within the code by modifying the existing application code loaded into memory. When the breakpoint occurs, the debugger substitutes the correct code for the breakpoint and restarts execution.

Of course, it would be a terrible idea to write the breakpoint back to the original program file? Worse yet, they wouldn? Another example of code modification, indeed probably the most common case of code modification is for the import table associated with a DLL. This table must be overwritten with the correct addresses of the functions as they were loaded into memory, not as they were in the original DLL file located on disk.

Sometimes no modification is needed because the DLL was loaded into memory at the same location specified when it was linked. Indeed, this is so much faster, Microsoft actually recommends in the SDK documentation that application programmers try to load their DLLs at separate locations to minimize the? The first time an executable page is modified, it generates a page fault because the page is marked as?

The Memory Manager then:. Unlike executable images, however, changes to memory mapped data files must be written back to disk eventually.

Thus, such pages are initialized so that they may be modified by the application using the memory mapping. As changes are made to the page, the hardware or possibly the operating system updates the page table entry to note that the physical page has been modified.

Thus, when the Memory Manager or Balance Set Manager attempts to reclaim the page, it will be written back to disk before it can be reclaimed. Note that if the system has enough physical memory it might be quite a long time before the Memory Manager attempts to reclaim that page. Thus, applications must ensure that the data is flushed to disk by using the appropriate APIs.

One interesting problem that occurs with memory mapped files is that a single file could be in use by more than one process at any point in time. For example, two programs might be using the same DLL. In that case it would be a waste of resources to maintain two separate copies of the DLL in memory. A more serious case would be if two applications or two instances of the same application opened a data file and each memory mapped it.

If there are two copies of that file in memory, there is absolutely no guarantee as to the order in which updates to the file will be written back to disk? Unfortunately, since individual pages could be written in different orders, there is no guarantee that the contents of such a file would even represent what either application thought was written to disk.

Figure 4. Of course, Windows NT has a clever solution to this problem? Let's first take a look at the PFN database. We know the virtual address where this structure is supposed to start, but let's look for it the long way, anyway.

We can easily find the beginning of this structure by using the? The poi command treats its parameter as a pointer and retrieves the value located at that pointer. First we need to figure out the page frame number in physical memory that the target's PTE refers to, and then we can index into the PFN database by that number. Looking back on what we learned from the previous article , we can grab the PTE information about the target page very easily using the handy! The above result would indicate that the backing page frame number for the target is 0xb.

That should be the index into the PFN database that we'll need to use. Remember that we'll need to multiply that index by the size of an nt! This looks like a valid PFN entry. We can verify that we've done everything correctly by first doing the manual calculation to figure out what the address of the PFN entry should be, and then comparing it to where WinDbg thinks it should be.

So based on the above, we know that the nt! This can be done by using the! Here we can see that WinDbg also indicates that the PFN entry is at 0xfffffaf , just like our calculation, so it looks like we did that correctly. An interlude about working sets Phew - we've done some digging around in the PFN database now, and we've seen how each entry in that database stores some information about the physical page itself. Let's take a step back for a moment, back into the world of virtual memory, and talk about working sets.

Each process has what's called a working set, which represents all of the process' virtual memory that is subject to paging and is accessible without incurring a page fault.

Some parts of the process' memory may be paged to disk in order to free up RAM, or in a transition state, and therefore accessing those regions of memory will generate a page fault within that process. In layman's terms, a page fault is essentially the architecture indicating that it can't access the specified virtual memory, because the PTEs needed for translation weren't found inside the paging structures, or because the permissions on the PTEs restrict what the application is attempting to do.

When a page fault occurs, the page fault handler must resolve it by adding the page back into the process' working set meaning it also gets added back into the process' paging structures , mapping the page back into memory from disk and then adding it back to the working set, or indicating that the page being accessed is invalid.

Figure 7: An example working set of a process, where some rarely accessed pages were paged out to disk to free up physical memory. Figure 8: Two processes may have different views of their user space virtual memory, but they get a shared view of the kernel space virtual memory.

Figure One of the processes no longer has a valid PTE for the shared memory, so Windows instead uses the prototype PTE to ascertain the true state of the physical page. This way, the page fault handler is able to examine the prototype PTE to see if the physical page is still valid and resident or not. If it is still resident, then the page fault handler can simply mark the process' version of the PTE as valid again, resolving the soft fault.

If the prototype PTE indicates it is Not Valid , then the page fault handler must fetch the page from disk. We can continue our adventures in WinDbg to explore this further, as it can be a tricky concept.

Based on what we know about shared memory, that should mean that the PTE referenced by the PFN entry for the entry point of notepad. Let's look at the fully expanded nt! We can compare that with the nt! It looks like the Prototype bit is not set on either of them, and they're both valid. This makes perfect sense. The shared page still belongs to notepad. The notepad. The same isn't true for a region of memory that can't be shared. We can use the! Because this memory is not shareable, the process' paging structures are able to manage the hardware PTE directly.

It's worth exploring different regions of memory this way, just to see how the paging structures and PFN entries are set up in different cases. As mentioned above, the VAD tree is another important consideration when dealing with user-mode memory as in many cases, it will actually be a VAD node which indicates where the prototype PTE for a given shared memory region resides.

In these cases, the page fault handler will need to refer to the process' VAD tree and walk the tree until it finds the node responsible for the shared memory region.

Figure The FirstPrototypePte member of the VAD node points to a region of memory that has a contiguous block of prototype PTEs that represent shared memory within that virtual address range.

Let's quickly switch back to the application so that we can let it create our initial buffer. To do this, we're simply allocating some memory and then accessing it to make sure it's resident.

Your buffer may differ. We should hop back into our debugger now and check out that memory address. As mentioned before, it's important to remember to switch back into the process context of memdemo.

We have no idea what context we could have been in when we interrupted execution, so it's important to always do this step. When we wrote memdemo. It would ensure that when the breakpoint was hit, we were already in the correct context. For the purposes of this article, though, it's best to practice swapping back into the correct process context, as during most live analysis we would not have the liberty of throwing int3 exceptions during the program's execution.

Looks like that was a success - we can even see the 0xff byte that we wrote to it. Let's have a look at the backing PTE for this page using the! That's good news. It seems like the Valid V bit is set, which is what we expect. Let's look at the PFN database entry using!

We can go to the address of the PTE at 0xffffed00ff0a8e00 and cast it as an nt! We see that it's Valid , Dirty , Accessed , and Writeable , which are all things that we expect. The Accessed bit is set by the hardware when the page table entry is used for translation. If that bit is set, it means that at some point the memory has been accessed because the PTE was used as part of an address translation.

Software can reset this value in order to track accesses to certain memory. Similarly, the Dirty bit shows that the memory has been written to, and is also set by the hardware. We see that it's set for us because we wrote our 0xff byte to the page.

Now let's let the application execute using the g command. We're going to let the program page out the memory that we were just looking at, using the following code: Once that's complete, don't forget to switch back to the process context again.

We need to do that every time we go back into the debugger! Now let's check out the PTE with the! We see now that the PTE is no longer valid, because the page has been trimmed from our working set; however, it has not been paged out of RAM yet. This means it is in a transition state, as shown by WinDbg. We can verify this for ourselves by looking at the actual PTE structure again. So because the memory hasn't yet been paged out, if our program were to access that memory, it would cause a soft page fault that would then simply mark the PTE as valid again.

If we examine the PFN entry with! Now let's press g again and let the app continue. It'll create a shared section of memory for us. In order to do so, we need to create a file mapping and then map a view of that file into our process.