You people have left me no choice but to research my own solution. The command to use is "xmodmap".
"xmodmap -pm" Shows the mappings in place. Here's an example:
shift Shift_L (0x32), Shift_R (0x3e)
lock Caps_Lock (0x42)
control Control_L (0x25), Control_R (0x6d)
mod1 Alt_L (0x40), Alt_R (0x71), Meta_L (0x9c)
mod2 Num_Lock (0x4d)
mod4 Super_L (0x7f), Hyper_L (0x80)
mod5 Mode_switch (0x5d), ISO_Level3_Shift (0x7c)
So we need to pull Alt_R out of mod1 and put it into mod4. We can do this with two commands:
xmodmap -e "remove mod1 = Alt_R"
xmodmap -e "add mod4 = Alt_R"
And that's it. The right alt key is a windows key for the rest of the session. I found the solution at this site: