## Segment Registers: Real mode vs. Protected mode

Content by: johnfine@erols.com
HTML formatting by: volunteer_needed

This page discusses the major difference between real mode and protected mode, which is the behavior of segment registers.

- The simple but wrong version
- The way it really works
- Examples (why you can't trust the simple version)
  - Switching to pmode
  - Switching from pmode
  - Flat real mode
  - CPU reset
  - Changing the GDT or LDT
- Related topics
  - segment wraparound
  - NULL Selector

### The simple but wrong version

Most memory accesses implicitly or explicitly use a segment register. If you don't understand that aspect of it yet, please look at Segment register basics.

In real mode the CPU shifts the segment register value left by four places (multiplying it by 16) and adds the 16 bit offset to get a 20 bit physical address.

In pmode the value in a segment register is not a segment number. It is a "selector". Bit 2 of the selector tells the CPU whether to use the GDT or LDT. Bits 3 to 15 of the selector index to one of the descriptors in the GDT or IDT.

The CPU adds the 16-bit or 32-bit offset (from the address portion of the instruction) to the "base" value from the descriptor. The result is a 32-bit "linear" address.

The descriptor also contains a "limit" (length minus one) for the segment. The CPU will generate a fault if you attempt to access beyond the limit of a segment.

Bits 0 and 1 of the selector are known as the "RPL". The current priviledge level is known as the "CPL". Those values are checked against the descriptor priviledge "DPL" to see if the access should be permitted.

### The way it really works

Each segment register is really four registers:

- A selector register
- A base register
- A limit register
- An attribute register

In all modes, every access to memory that uses a segment register uses the base, limit, and attribute portions of the segment register and does not use the selector portion.

Every direct access to a segment register (PUSHing it on the stack, MOVing it to a general register etc.) uses only the selector portion. The base, limit, and attribute portions are either very hard or impossible to read (depending on CPU type). They are often called the "hidden" part of the segment register because they are so hard to read.

Intel documentation refers to the hidden part of the segment register as a "descriptor cache". This name obscures the actual behavior of the "hidden" part.

In real mode (or V86 mode), when you write any 16-bit value to a segment register, the value you write goes into the selector and 16 times that value goes into the base. The limit and attribute are not changed.

In pmode, any write to a segment register causes a descriptor to be fetched from the GDT or LDT and unpacked into the base, limit and attribute portion of the segment register. (Special exception for the NULL Selector).

When the CPU switchs between real mode and pmode, the segment registers do not automatically change. The selectors still contain the exact bit pattern that was loaded into them in the previous mode. The hidden parts still contain the values they contained before, so the segment registers can still be used to access whatever segments they refered to before the switch.

When the CPU switches between V86 mode and pmode it updates all the segment registers to have values which are consistent with the new mode. This is the reason that tricks like flat real mode don't work in V86 mode.

When the CPU switches to "System Manangement Mode" from any other mode it stores the entire contents of the segment register (hidden as well as selectors) in an SMM work area. On some CPU's, this is the only way to read the hidden part of the segment registers.

When the CPU switches from SMM to any other mode it reads the entire contents of the segment registers from that work area. It does not perform many of the consistency checks that would occur under other circumstances, so it is possible to create something like flat real mode for V86, but there are many further problems.

### Examples (why you can't trust the simple version)

In many situations the selectors will be "out of sync" with the base, limit and attribute values.

If you think of the hidden part as a "descriptor cache", you will find it very hard to work with these situations. With a cache, you would have no idea when the values will pop back into sync, or what you can/can't safely do while they are out of sync.

Knowing that they are a hidden portion of the segment registers, means that you know they will change as described above, when you write to the segment registers. Clearly you need to have interrupts disabled whenever the segment registers are out of sync (except for stable cases like flat real mode), because any interrupt routine might save and "restore" a segment register and that "restored" segment register will have only the same selector as was saved, not the same hidden part.

### Switching to pmode

Switching to pmode is real easy. Just

       mov  eax, cr0
       inc  ax
       mov  cr0, eax

It is also real hard, because when you first switch to pmode, the slightest mistake will cause your CPU to "triple fault". That makes your system reboot. It doesn't tell you much about what you did wrong, just that you did SOMETHING wrong.

When you first switch to pmode, your CS register still has the same base, limit and attribute values it had before. Don't worry about the fact that the CS selector is now "incorrect" for pmode. As long as you don't have any interrupts or far CALLs, the CS selector doesn't matter. You can execute lots of code, even do near JMPs and CALLs and RETs, all without loading a pmode compatible CS selector.

Similarly, your data, stack, etc. segment registers retain their base, limit and attributes. This brings me to a great debugging aid for mode switching routines:

I will assume that your early pmode code does not use GS, and that your display is in text mode and that your display regen buffer is at B8000. (Make the obvious change if that last item is wrong). Put the following code well before you switch to pmode:

       mov  ax, 0B800h
       mov  gs, ax
       mov  byte [gs:0], '1'     ;NASM Syntax

Now take lines like these:

       mov  byte [gs:2], '2'
       mov  byte [gs:4], '3'
       mov  byte [gs:6], '4'
       etc.

and distribute them through the following code. DO NOT WORRY about the fact that GS has a selector that is incompatible with pmode. DO NOT load a pmode compatible selector into GS (until you are sure your GDT etc. work well enough that you probably don't need this debugging aid anymore anyway).

When your program triple faults, watch the display carefully. The sequence of characters will be visible for a moment (even after the triple fault), so you can see how far your code got before it died.

Before you can change any segment register within pmode, you need to have a valid GDT. You can do the LGDT instruction before or after switching to pmode. It is safe before switching because the value in the GDT register is ignored in real mode; It doesn't affect the loading of segment registers; It doesn't even need to be valid. Doing the LGDT after switching to pmode is safe because all your segment registers still have valid base, limit and attributes, so memory references (like loading the GDT psuedo-descriptor into the GDT register) are still OK.

Many people misunderstand the extra level of indirection involved in pointing to the GDT. The LGDT instruction encodes an address (usual memory addressing rules). That address is the offset (within the indicated or implicit segment) of the psuedo-descriptor. The psuedo-descriptor in turn contains the address of the GDT. THAT address is a linear address and is not based on the contents of any segment register.

In all my examples I put that psuedo-descriptor in the same location as the GDT itself. This is possible because the CPU never uses the first eight bytes of the GDT. (See NULL Selector).

If your code is a DOS program, or is loaded in some other form in which you cannot know linear addresses until run time, then you must patch the psuedo-descriptor to get the correct linear address.

       xor  eax, eax
       mov  ax, ds
       shl  eax, 4           ;eax = linear address of data segment
       add  [GDT+2], eax     ;patch psuedo-descriptor
       lgdt [GDT]
    . . .
    GDT  dw  GDT_length - 1
         dd  GDT
         dw  0
    ;First descriptor goes here

Before enabling or using any interrupts, remember to set up a pmode compatible Interrupt Descriptor Table. Like LGDT, LIDT may be executed before or after entering pmode; However, unlike the GDT, the IDT does affect the CPU within real mode. Interrupts should remain disabled from before the earlier of the LIDT or the switch to pmode, until after the later of the LIDT or loading pmode compatible selectors into all segment registers touched (including save/ restore) by any interrupt.

WARNING: A 386 needs a very tiny delay (any instruction would be more than enough) after switching to pmode, before it can correctly load a selector into a segment register. In one version of my switch to flat real mode I had the selector value in a general register before switching to pmode, and the very first instruction after switching to pmode was a fast instruction to MOV that selector to a segment register. Depending on instruction alignment, it could corrupt the hidden part of the segment register (on a 386 only). You can safely write to a segment register with the very first instruction after switching to pmode if it is a slow instruction like a POP or a far JMP, but not if it is a fast instruction like "MOV DS,BX". Normally you wouldn't even notice this problem because it is more natural to move the selector to a register right before you move it to the segment register.

### Switching from pmode

In switching from pmode, you usually need to reload your segment registers twice.

Before switching from pmode you must put real mode compatible values in the limit and attribute part of each segment register. You cannot change the limit or attribute while you are in real mode, so you must set the values you want before you exit pmode.

After switching from pmode you must put real mode compatible values in the segment selectors. Immediately after you exit pmode the segment registers still have the selector and base values that were loaded when you fixed the limit and attribute values (as described above). As with the switch TO pmode, you can use the segment registers with those base values to execute code and access data as long as you want after switching. However, you should understand the consequences of having the selector and base values out of sync. As soon as any interrupt service routine saves and restores a segment register, the pmode base will be lost and the base will become 16 times the selector.

There is a default size bit in each of the CS and SS attribute parts. It comes from bit 54 in the descriptor (that is the D_BIG bit, for those using my gdt.inc definitions). In the CS register the default size has a major effect. It sets the default operand and address size to 16 or 32. If you exited to real mode with that bit still set, you would get very strange results (and not very useful, because the BIOS wouldn't be usable and the IVT would still function in 16-bit real mode).

If you exit to real mode with the default size bit still set in SS (a common mistake), you get a much subtler problem. AFAIK, the sole effect of the default size bit in SS is to cause all implicit uses of SP to use ESP instead. Implicit uses of SP are PUSH, POP, CALL, RET, INT, etc.; If the upper half of ESP is cleared, you won't immediately notice that it is being used. However, there are several things that programs might do that will crash as a result of that SS bit. My favorite is:

       xor  sp, sp
       push something

Normally that predecrements sp to FFFE and writes the value there. You may not like the results if the value gets written to ss:FFFFFFFE instead.

### Flat real mode

Flat real mode, AKA "unreal mode" is the mode you get to if you return to real mode from pmode leaving a 4GB limit in some or all of ds, es, fs, gs, ss.

Flat real mode has the major advantage over PMODE, that it is compatible with the BIOS and DOS. In fact, if you are running HIMEM.SYS without EMM386.SYS you are probably already in flat real mode.

Getting into flat real mode is pretty easy:

       cli                ;Don't allow interrupts during this
       push ds            ;Save real mode selectors
       push es
       xor  eax, eax      ;Patch the GDT psuedo-descriptor
       mov  ax, ds        ;  (assuming ds points the segment containing the GDT)
       shl  eax, 4
       add  [GDT+2], eax
       lgdt [GDT]         ;Load the GDT
       mov  eax, cr0      ;Switch to pmode
       inc  ax
       mov  cr0, eax
       mov  bx, flat_data ;Our only pmode selector
       mov  ds, bx        ;Install 4Gb limits (warning)
       mov  es, bx
       dec  ax            ;Switch back to real mode
       mov  cr0, eax
       pop  es            ;Restore real mode selectors
       pop  ds
       sti                ;Notice, I never needed to change the CS while in pmode.
    . . .

               %include "gdtn.inc"
    GDT        start_gdt
    flat_data  desc 0, 0xFFFFF, D_DATA+D_WRITE+D_BIG_LIM
               end_gdt

Most people seem to build their GDT's in hex rather than using macros such as those defined in gdtn.inc. If you really want to do it that way, the four lines above that create the GDT could be replaced by:

    GDT        dw  0xF, GDT, 0, 0, 0xFFFF, 0, 0x9200, 0x8F
    flat_data  equ 8

I think the hex version is unreadable and error prone.

A common fallacy about flat real mode is that it is incompatible with programs that rely on segment wraparound.

On an 8086, offsets within a segment are truely and consistently 16 bits. If you access ds:[bx+si+7000h] and bx=7000h and si=7000h, then you are asking for 7000h*3 which is 15000h; But of course you actually get offset 5000h because offsets are 16 bits.

Similarly, if you access a word at offset [0FFFFh], the low byte of the word will come from offset 0FFFFh and the high byte will come from offset 0, not from the byte that physically follows the low byte.

On a 386+, there is only a slight difference. In 16-bit addressing, 7000h + 7000h + 7000h still equals 5000h; However, accesing the word at offset FFFFh now gives you a protection violation rather than a wraparound.

In flat real mode, there is another slight difference. In 16-bit addressing, 7000h + 7000h + 7000h STILL equals 5000h (despite nonsense to the contrary that I have seen written); However, accesing the word at offset FFFFh now gives you the full word at that location rather than either a wraparound or a protection violation.

In the unlikely event that some program relied on the protection violation, it would be incompatible with flat real mode. If it relied on the wraparound, it wouldn't run correctly on a 386+ anyway.

Flat real mode is incompatible with V86 mode, so if you are running EMM386 etc. which put your DOS session in V86 mode, then you can't switch to flat real mode. Also certain BIOS interrupts and other programs, that temporarily switch in and out of pmode, will trash your extended limits and take you back out of flat real mode.

Another common fallacy is that 32-bit addressing modes are possible in flat real mode but impossible in ordinary real mode. In fact, the modes themselves are always valid; Only the limits change. In real mode you could do:

       mov al, [ecx]

I have even found reasons to do that in ordinary real mode. It allows you to use the cx register for an address (as long as the high bits of ecx are clear). It just doesn't allow that address to be above 64K.

Flat real mode doesn't allow any new addressing modes; It just allows larger offsets (that otherwise would have caused protection violations) in the 32-bit addressing modes that were already available (though rarely useful) in real mode.

### CPU reset

You probably don't care unless you are designing a BIOS or a motherboard, but a 386+ CPU starts up with a strange value in its CS registers.

The EIP is FFF0 and the CS selector is F000, so it would seem that it starts at physical address FFFF0, just like an 8086 (which starts at FFFF:0).

Actually the hidden base portion of the CS is FFFF0000, not F0000, so the actual starting address is FFFFFFF0.

As long as you don't write to the CS register in any way, that base value will remain and you can do near JMP, CALL and RET, to execute code in FFFFxxxx. As soon as you do the first far JMP, CALL, INT, etc. the base value will be back in sync with the selector and you can't access above 10FFEFh again until you switch to pmode.

### Changing the GDT or LDT

While you are in pmode, you may have reason to execute another LGDT instruction, or to change the contents of a descriptor.

These actions do not affect the base, limit or attributes of any segment register. The new descriptor values will only be used when you write to a segment register.

In this sense the "descriptor cache" acts very unlike a cache and very unlike the "TLB" which is the cache for the paging system.

When you change an entry in a page directory or page table, it is very hard to predict how long you can safely use the old "stale" entry in the TLB. In general you can't. When you write a new CR3 value (ananogous to doing another LGDT) the CPU flushes the TLB and immediately gets rid of all the "stale" values. When you do a new LGDT the "descriptor cache" isn't touched.

### Segment register basics

Most memory accesses implicitly or explicitly use a segment register.

- To fetch instructions the CPU unconditionally uses the CS register.
- For most data accesses the CPU uses the DS register by default.
- When implicitly using DI or EDI in a string instruction, the CPU unconditionally uses the ES register.
- When implicitly using SP or ESP as a stack pointer the CPU unconditionally uses the SS register.
- When using BP, EBP, or ESP explicitly as a base register the CPU uses the SS register by default.

In the cases in which DS or SS would be used by default, the choice of segment register may be overridden by a segment prefix.

The CPU first computes an offset. In 16-bit addressing the CPU truncates the offset to the low 16 bits. The CPU then adds the offset to the segment-base provided by the selected segment register. This yields the linear address that will be accessed.

### Writes to a segment register

When I refer to "writing to a segment register", I mean any action that puts a 16-bit value into a segment register.

The obvious example is something like:

      MOV  DS,AX

However the same rules apply to many other situations, including:

- POP to a segment register.
- FAR JMP or CALL puts a value in CS.
- IRET or FAR RET puts a value in CS.
- Both hardware and software interrupts put a value in CS.
- A ring transition puts a value in both SS and CS.
- A task switch loads all the segment registers from a TSS.

All of these writes to segment registers behave as described above according to the current mode of the CPU.

When you are writing code that crosses mode boundaries, you should consider all possible segment register writes. Things like saving and restoring a segment register, that are normally transparent, may change the hidden part of the segment register, when executed after a mode change.

### NULL Selector

In pmode, a selector value of zero gets special handling.

The general rule would say that a selector of zero refers to the zeroth descriptor in the GDT. But this does not occur.

The CPU never refers to the zeroth descriptor in the GDT. The first 8 bytes of the GDT are available for any use you like and their contents will not affect the behavior of the GDT.

When you write a zero to DS, ES, FS or GS in pmode the CPU sets the attributes of that segment register to mark it unusable, but does not generate a fault at that time. If you then use that segment, you will get a fault.

The purpose of the NULL Selector is to cover situations in which you do not have valid contents for a segment register. If you left old invalid contents in the register and simply didn't use that register, you might run into trouble when some routine saves and restores the register. On restoring the register, it might generate a fault.

The NULL selector may be freely saved and restored without causing a fault.

Trying to put the NULL selector into the CS register will generate a fault on the instruction that attempts that. It will NOT load the NULL selector and generate the fault on the next attempt to fetch an instruction. I am not sure whether putting NULL in SS generates an immediate fault or waits until the next use of SS.

### NASM Syntax

All the example here use NASM Syntax.

If you want to use MASM or TASM etc. I hope the meaning of the NASM instructions is clear enough that you can translate to the assembler you choose.

NASM is free, so you may be better off downloading a copy of NASM and using that, rather than trying to translate syntax.

I have made several enhancements to NASM. You can find both my enhanced version of NASM and links to standard NASM on my web page:

http://www.erols.com/johnfine/

Web Space for this page is provided by

The CubeSoft Network | Codezone