Crazy hacks: Changing Wine key mappings on Mac OS X

One of the biggest quirks of using a Mac is its keyboard — in addition to the usual Control (⌃) and Alt a.k.a. Option (⌥) keys you also have the Command (⌘) key. Unlike on Windows or Linux, the Control key is rarely used at all, the only really important shortcut key being Control-Tab. Instead, most shortcut keys use Command as modifier. So as a Mac user you quickly learn to press Command for pretty much any key combination.

This works well as long as you stick to native OS X applications. As soon as you run Windows applications via Wine however you get into trouble: in order to map Mac keys to Windows keys Wine associates the Control key with Control. Consequently, in Wine you have to for example press Control-C to copy text instead of Command-C. You also have to remember that all other shortcut keys use Control instead of Command which is very annoying.

Given that the best key mapping is a matter of taste, one would expect Wine to be configurable as far as key mappings go. Yet pretty much everything you find on that topic tells you to change Xorg key mappings. This works if you are on Linux or using an old Wine version that requires Xorg. However, current Wine versions have a Mac Driver that allows them to run on OS X natively, without the awkward detour through Xorg. And other than these outdated advises I could only find a question on apple.stackexchange.org — unanswered of course (I answered it in the meantime).

Given the lack of documentation, studying the source code is the only option. The important function here is macdrv_compute_keyboard_layout() that does various things trying to guess the best keyboard mapping — yet none of the steps appear to be configurable (yes, I filed a change request). There is also a hardcoded mapping table called default_map that is uses as a fallback and that’s exactly what is being used for the modifier keys. This means, in order change the mapping of modifier keys one has to modify that table in the compiled binary (that would be /Applications/Wine.app/Contents/Resources/lib/wine/winemac.drv.so on my computer).

Warning: Perform the following steps at your own risk and keep a backup of the original file. Note that you will have to redo this procedure whenever Wine is updated.

The mapping for the two Command keys is defined by the following lines:

{ VK_RMENU,                 0x38 | 0x100,   TRUE },     /* kVK_RightCommand */
{ VK_LMENU,                 0x38,           TRUE },     /* kVK_Command */

For the binary representation of these lines one has to keep in mind that WORD (type of the first two fields) is a two byte integer whereas BOOL (type of the last field) is a four byte integer. You will also need to consult the list of Windows virtual key codes (VK_RMENU has the value A5 and VK_LMENU the value A4). The second field is a key scan code defining the physical key being pressed. So the compiled binary should contain the following byte sequences:

A5 00 38 01 01 00 00 00
A4 00 38 00 01 00 00 00

Any hex editor should be able to open the binary and find these bytes, in my case they were located at hexadecimal offset 62770. In order to change the mapping to VK_RCONTROL and VK_LCONTROL one could change these bytes to:

A3 00 1d 01 01 00 00 00
A2 00 1d 00 01 00 00 00

Now whenever you press the Command key it will be interpreted as Control by the Windows application rather than Alt. But you will no longer have anything mapping to Alt. One option would be to change the mapping of the Control key but I prefer to keep it mapped to Control in order for Control-Tab to work in Windows applications. The other option is to map the Option key to Alt, normally it is unmapped.

All unmapped keys are represented by eight zeroes so you will have to find the corresponding entry by position relative to other keys — three and six entries behind the left Command key for the left and right Option key respectively. You can put the original values for the Command keys there. This will work for combinations like Alt-F4 yet menu shortcuts like Alt-F won’t work — on OS X most combinations involving letters and the Option key are used to enter special characters. Still, you can press Alt to enter the menu and cursor keys to navigate which is a lot more than you can usually do with OS X menus.

Of course, there are more options — e.g. you could leave the left Command key mapped to Alt. Choose your poison, and in time Wine developers will hopefully make this post obsolete.

Comments

  • this just in

    You forgot about the Fn key on OSX

    and on PC (wish game sites would cease using PC as interchangeable for windows) Ubuntu there’s also the option to use the StartKey should your keyboard have one

    more key combos can be more better

    (for humble bundle type no-evil-drm games)

  • this just in

    how would you hackably run windows steam games on osx with wine or playonlinux?

    Wladimir Palant

    I wouldn’t – I rarely run games at all.

  • Edward Mendelson

    Hello Wladimir,

    Thank you for this! I am putting together a Wineskin wrapper for a DOS emulator called vDos, and I had to use the X11 driver, because I was able to map (Windows) Alt to (OS X) Option, and Win to Cmd using an .xmodmap file, but I wasn’t able to map those keys in the Mac driver. However, for many reasons, I wanted to use the Mac driver (which makes it possible to get full-screen mode without a menubar).

    Your instructions made it easy for me to map Alt to the Option keys by patching winemac.drv.so from the current Wine. But I cannot figure out how to map LWin and RWin to LCmd and RCmd. Is there any chance you can help?

    In case it helps anyone else, I’ve put a copy of my patched winemac.drv.so here:

    https://www.dropbox.com/s/e799aofw280gi3a/winemac.drv.so

    I hope someone might find it useful.

    Again, thank you for these clear and valuable instructions.

    P.S. Some years ago I sent a contribution for AdBlock Plus, and it’s good to be able to thank you again in person.

    Wladimir Palant

    In your file the entry for the right command key can be found at hexadecimal offset 63990, left command key is the next entry. I’m not quite sure about the scan codes for the Windows key but it seems that you have to replace these bytes by:

    5C 00 5C 01 01 00 00 00
    5B 00 5B 01 01 00 00 00

    However, I wonder how that is supposed to work – applications rarely handle the Windows key themselves, it is normally being handled by the operating system (which is missing in case of Wine).

  • Edward Mendelson

    That is exactly the answer I was looking for. Thank you again!

    This works because the application I am using (sourceforge.net/projects/vdos) is an SDL application that defines the Win key for its own purposes. It uses Win-F11/F12 to reduce and enlarge the window, Ctrl-Win-V to paste from the Windows clipboard (in Wine, from the OS X clipboard), and Ctrl-Win-click to start drawing a box that is copied to the clipboard when the mouse button is released.

    I am setting up vDos to work with WordPerfect for DOS. Thanks to your patch, all the Alt, Ctrl, and Shift keys work as WordPerfect expects, and the Win keys work perfectly also.

    Now I will try to modify the source code so that it produces a file that doesn’t need to be patched. If you get inspired to post instructions for fixing the source, a lot of people will be grateful. If I figure it out, I will post the details here.

    Also, I’ve replaced the file that I linked in my earlier message with one that has both of your patches.

    Thank you again!

  • Edward Mendelson

    Thanks to your advice, it was fairly easy to change the source code in keyboard.c to produce the desired results.

    I wasn’t able to get the revised code to appear correctly here, so here is a link to the revised file:

    https://www.dropbox.com/s/wa672ngxj6zp9yx/keyboard.c

    The four changed lines have comments that begin “//was” etc., etc.

    Thank you again!

  • marc balentine

    i would love to have basic instructions on finding/editing the source code that Edward Mendelson mentioned/linked to.

    thanks,

    m

    Wladimir Palant

    His link still works. But you'd need to compile Wine yourself of course.

  • mar

    Hi Wlad,

    Yeah, it all looks like quite an undertaking. It would be great to just have a downloadable version of Wine for Mac with this problem corrected.

    Thanks anyhow.

  • Guy

    Could you include the codes I'd need to find and replace to map alt to option? The links you provided are broken, and I may as well be a 5 year old when it comes to hex editing. Thanks!

    Wladimir Palant

    You know, that blog post is five years old. I've stopped using Mac OS X years ago. And it's labeled "crazy hacks" for a reason - this definitely isn't something that 5 year olds should be doing.