Pages

Tuesday, January 22, 2019

Working on Wine Part 6 Sending Your Work Upstream

Andrew Eikum a CodeWeavers employee and Wine developer is writing a series of post to introduce people to Wine usage and development.

About This Guide


This is a series of guides intended to introduce software developers to the Wine ecosystem. It will cover what Wine is, how to use Wine, how to debug Wine, how to fix Wine, and what to do with your fix once you've made it.

The guide will be published throughout January.

  • Part 1 describes what Wine is and provides a short description of various popular forks of Wine.
  • Part 2 describes Wine's build process.
  • Part 3 describes how to use Wine as a developer.
  • Part 4 describes how to debug Wine in general.
  • Part 5 describes Wine's source tree layout and how to edit the source.
  • Part 6 describes how you can send your work upstream.

If you recall from Part 1, there are many forks of Wine. Where your fix belongs can vary depending on what fork of Wine you use, the nature of the bug you fixed, and how you fixed it.

 

 

Choosing the right place to send your fix


The ideal place for your fix is in upstream Wine. This is the origin point of all Wine forks. If you fix it in upstream Wine, then all users of Wine will eventually benefit from your work as the various forks pull in changes from upstream.

Sending your fix upstream should be the default choice for your work. If none of the exceptions below apply, or if you're unsure where your fix belongs, work with upstream first.

 

When upstream is the wrong choice


Your patch may be built on top of an existing patch in wine-staging. In that case, it should be sent to the wine-staging maintainers. If you, or they, think that the patch is ready to be sent upstream, then go ahead and do that instead.

If your patch fixes the issue, but you failed to fix the tests you wrote, or your patch causes some other test failures, it may belong on the bug tracker, or possibly in wine-staging.
If your patch provides a useful feature that upstream Wine is not interested in, it may belong in wine-staging.

If your patch is builds on a feature exclusive to some fork of Wine, like Proton or CrossOver, it may belong in that fork and not upstream. Work with the fork maintainers to determine if it's appropriate for upstreaming.

 

Upstreaming your fix


Hopefully, your patch is going upstream. Wine's patch submission process is done via email. Your patch should be sent in plain text to wine-devel@winehq.org. You should subscribe to this mailing list to avoid being placed into the moderation queue.

It is recommended to use git send-email to send the email through your mail server. You can also use git format-patch and attach the resulting file in an email client. Be careful that your mail client doesn't wrap, or otherwise corrupt, the attachment as if it were a text document.

Patches that are sent upstream should have your Sign-off. This can be applied by Git automatically with the -s switch during git commit and/or git format-patch. You must use your real name in the Author field when submitting a patch.

If you are submitting a series of patches, try to limit yourself to about four patches per submission. Your patches should be self-contained anyway, so there is no harm in submitting them in several batches. Smaller patch series are easier to review, and keep from cluttering up the mailing list if you have to re-send the series with changes.

 

Receiving feedback


Wine has a patch status webpage which will track the status of your patch. Your patch will be tested by the Wine Test Bot to ensure any new tests pass on various Windows versions. If the area of Wine changed by your patch has a maintainer, it will be assigned to that person for review. If not, it will be reviewed by the general community, or by the Wine maintainer.

Be patient, it may take a few days for your patches to receive review. Wine reviewers try to reply to every patch within a week, but if you don't get feedback you may send an email to wine-devel asking for a review. Be sure you are subscribed to wine-devel, as some reviews may be sent to that mailing list instead of directly back to the author.

If you received some suggestions, take those suggestions into account and send a new version of the patch. Feedback from Wine reviewers should not be seen as criticism or an attack. Wine is a very complicated piece of software, and it has a high standard of code quality for contributions. Rejections aren't made lightly—everyone wants Wine to improve. Instead, understand that there are reasons for the rejection, apply the suggestions, and resend the patch. If you do not have the time or interest in making the requested changes, consider sending your patch to wine-staging so some other person may take up the patch in the future and try to get it upstream. Or, attach it to a Bugzilla bug so it is not lost.

If your patch is accepted, then congratulations! You have just made Wine better. It's time to move on to the next bug.

Full Article

Run Microsoft Windows Applications and Games on Mac, Linux or ChromeOS save up to 20% off  CodeWeavers CrossOver+ today.

 

Friday, January 18, 2019

Wine development release 4.0-rc7 is now available for Linux FreeBSD and macOS

Wine development release 4.0-rc7 is now available for Linux FreeBSD and macOS

The Wine development release 4.0-rc7 is now available.
What's new in this release:
  • Bug fixes only, we are in code freeze.
The source is available now. Binary packages are in the process of being built, and will appear soon at their respective download locations.


Bugs fixed in 4.0-rc7 (total 13):

  20728  Multiple video players crash when opening audio or video file (MPC-HC v1.6.5, PotPlayer 1.5.x)(FilterGraph_create releases/destroys controlling IUnknown)
  26369  A.R.E.S. Extinction Agenda 1.x (.NET 2.0, XNA 3.1 game) crashes during intro ('quartz' FilterGraph2_Connect must translate HRESULT of failures more correctly)
  29461  BurnPlot (VB6 app) fails to start, complaining with "Run-time error '438'" (WshShell3 'SpecialFolders' collection 'item' method invocation fails)
  34884  Touhou Danmakufu 0.12m's font becomes distorted
  35573  gdi32:fonts test_stock_fonts() fails on Windows 7 in the Japanese and Hebrew locales
  36082  Cannot Read Text In "Question" Boxes On Microsoft Money 2005 Installation
  36084  Microsoft Money 2005 Window Going "Black" After Certain Menu Operations
  43211  NVIDIA GeForce Experience 3.x installer fails due to 'setupapi.SetupDiDeleteDeviceInfo' stub
  44796  Age of Empires II: The Conquerors is broken when CSMT is enabled
  45874  Secret Files 1-2: hardware mouse cursor corrupted
  46212  Multiple games have performance issues (Project CARS, NFS: Hot Pursuit (2010), Gas Guzzlers: Combat Carnage)
  46459  Secret Files 1-2, Ufo: Extraterrestrials: mouse cursor invisible when anti-aliasing and hardware mouse enabled
  46480  Invalid write of size 2 in ntoskrnl.exe/tests/ntoskrnl.c

Run Microsoft Windows Applications and Games on Mac, Linux or ChromeOS save up to 20% off  CodeWeavers CrossOver+ today.

Thursday, January 17, 2019

Working on Wine Part 5 Fixing Wine

Andrew Eikum a CodeWeavers employee and Wine developer is writing a series of post to introduce people to Wine usage and development.

About This Guide


This is a series of guides intended to introduce software developers to the Wine ecosystem. It will cover what Wine is, how to use Wine, how to debug Wine, how to fix Wine, and what to do with your fix once you've made it.

The guide will be published throughout January.

  • Part 1 describes what Wine is and provides a short description of various popular forks of Wine.
  • Part 2 describes Wine's build process.
  • Part 3 describes how to use Wine as a developer.
  • Part 4 describes how to debug Wine in general.
  • Part 5 describes Wine's source tree layout and how to edit the source.
  • Part 6 describes how you can send your work upstream.

Once you've got some idea of what is causing the problem with your application, it's time to go understand how those APIs are implemented in Wine so you can fix it. You may also need to change Wine's code to debug the application in the first place.

 

Search for existing bugs


Before digging into the source, it's always useful to first search for existing bugs. You can search Wine's Bugzilla for your application, or for the symptoms you're seeing. As you begin to debug your issue, you may find other search terms to try. If you find a bug, you may find some useful analysis or work has already been done for the issue.

You may also want to look through the Wine Staging patches, especially once you have some idea of what the problem is. You may find someone has already written a patch to fix this problem, but it isn't ready to go upstream yet. You can pick up the torch and try to upstream the work.

 

Wine source layout


Wine implements the Windows operating system, which is composed of libraries, programs, and kernel functionality. You can find the code for Wine's implementation of each of those components in dlls, programs, and server, respectively. To give you some idea of how different components of Wine work together, some important, core components are listed here.
  • dlls/ntdll – This is where many of the core OS APIs are implemented, like file handling, thread creation and synchronization, timing, and much more. Applications typically use kernel32 instead of ntdll directly.
  • dlls/kernel32 – This is the application-facing interface for the core ntdll APIs mentioned above.
  • dlls/user32 – This is where much of the Windows GUI handling lives, like window creation, message handling, terminal services, and so on.
  • dlls/winex11.drv and dlls/winemac.drv – These libraries map the Windows GUI interfaces, and some other platform-specific functions, to the native platform. user32 calls into these platform drivers.
  • dlls/d3d* and dlls/wined3d – These libraries implement the Direct3D graphics API on top of the platform's OpenGL implementation.
  • programs/services – This program manages background services, both those created by Wine and those provided by installed applications.
  • programs/wineboot – This program kicks off initial prefix creation and other tasks when a prefix is booted.
  • programs/winecfg – This is Wine's configuration program.
  • server – The Wine server implements all cross-process functionality, including message routing, multi-process synchronization primitives, registry handling, and much more. The Wine server is always running if an application is running.
While debug channels are often named after the component in which they are declared (e.g. the dsound channel is declared in dlls/dsound), this is not always the case. You can use git grep to find the relevant debug trace line if it isn't obvious.

 

Writing tests


In most cases, your work should be driven by tests. Study the documentation for the relevant APIs and write tests first to confirm that Windows's implementation behaves how you expect it to. Then, change Wine's source to pass the tests that you wrote.

Wine has an extensive suite of unit tests. The unit tests comprise well over one million lines of code as of this writing. Any patch you submit must pass all of these tests. Patches should contain more tests proving that the new behavior is correct.

In general, you should focus your work on what a real application actually does. That is, you don't need to implement a bunch of unrelated functionality if the application doesn't actually need it. You should write tests to show how an application is using an API, and how Wine fails to meet its expectations. Then fix Wine to also pass those tests without breaking any existing tests. Any change to Wine has the potential to break other applications. Simply passing tests that you wrote is insufficient reason to change Wine.

You can find the tests for a given component in the tests subdirectory. For example, the tests for dsound live in dlls/dsound/tests. You will find .c files in that directory. The tests start in the START_TEST function and exercise the component being tested. The ok function demonstrates correct behavior. Again, "correct" is defined to mean "like Windows." For example:

    hr = IDirectSound_CreateSoundBuffer(ds, &bufdesc, &primary, NULL);
    ok(hr == S_OK, "CreateSoundBuffer failed: %08x\n", hr);
 
Here you can see the return value from IDirectSound::CreateSoundBuffer is expected to be S_OK. If it is not S_OK, then the test will fail.

Your tests should demonstrate how your application behaves when interacting with that API. Since you are fixing a bug in Wine's implementation of the API, your tests should demonstrate that Wine will fail without your fix. It isn't necessary, or even recommended, to fully exercise all inputs to an API in the Wine tests. Instead, be thoughtful about how your application might use the API given different inputs from the user, and write tests that reflect those possibilities.

Your tests should be self-contained. They should initialize the necessary functions, test the APIs, and then clean up before continuing. If you need to create files on disk, create them in a temporary location and/or clean them up afterwards. If your test may leave changes to the system, expect to have to clean up after a previous run that crashed, before you run your tests.

After you have written some tests, you need to verify that they pass when run on Windows. To do this locally, run make crosstest, which will use your system's mingw-w64 compiler to build a Windows executable. Run this executable on Windows in the command prompt to verify Windows's behavior. If you do not have a Windows system or VM available, you can upload your patch or test binary to the Wine Test Bot.

Once your tests pass on Windows, run them in Wine with make test. If your tests meaningfully demonstrate a Wine bug, they should fail. Now it is time to fix Wine to pass those tests, as well as all existing tests.

 

Working with Wine's source


As you develop your fix, there are some rules you should know if you intend to send your work upstream.

For better or worse, Wine does not have a coding style standard and likely never will. The rule is to try to match the surrounding code as best as possible. Especially in older code, you will find a horrible mish-mash of tabs and spaces and brace styles. Do your best to make the code no worse than it was. However, extraneous changes are discouraged as they make review difficult and looking up the source history in Git more cumbersome. If you are changing a line, or even one line in a short code block, it can be re-formatted to be less ugly than it was. But don't reformat an entire function to fit some style, no matter how ugly it is.

Wine only accepts code that adheres to the C89 standard, because some compilers that Wine cares about don't support anything later. The three most common snags here are that your code must declare its variables at the top of the block, you cannot use //-style comments, and function declarations with no arguments must be declared with void arguments:

    int some_function(void);
 
Resist the urge to gut an entire function or module and re-write it. Small, discrete changes that can be easily understood are the right way to fix Wine. Wine is more than two decades old. There is a lot of hard-won knowledge in much of Wine's source, and throwing out all of that history because it's easier than working in the existing code is not likely to pass review.

Make your changes as obviously correct as possible. Unrelated changes must be placed into their own commits. You should not introduce unused code in an early patch, which begins to be used in a later patch.

Write your patches with the reviewer in mind. It may be unintuitive, but understand that it is more important for your patch to be easy to review than make Wine's code perfect. Reviewer time is one of Wine's most valuable assets. Patches that are easier to review are more likely to pass that review.
Be aware that some tests are "flaky." This is an unfortunate reality of a system as complex as Wine. Ideally all tests would pass on all Windows and Wine environments, but due to bugs, differing platform behavior (especially window managers) and timing differences, they don't. Most modules have well-written tests that should always pass. Some, like the user32 messaging tests, and some audio and graphics tests, fail with some frequency, even on Windows.

Full Article

Run Microsoft Windows Applications and Games on Mac, Linux or ChromeOS save up to 20% off  CodeWeavers CrossOver+ today.

Tuesday, January 15, 2019

Working on Wine Part 4 Debugging Wine

Andrew Eikum a CodeWeavers employee and Wine developer is writing a series of post to introduce people to Wine usage and development.

About This Guide


This is a series of guides intended to introduce software developers to the Wine ecosystem. It will cover what Wine is, how to use Wine, how to debug Wine, how to fix Wine, and what to do with your fix once you've made it.

The guide will be published throughout January.
  • Part 1 describes what Wine is and provides a short description of various popular forks of Wine.
  • Part 2 describes Wine's build process.
  • Part 3 describes how to use Wine as a developer.
  • Part 4 describes how to debug Wine in general.
  • Part 5 describes Wine's source tree layout and how to edit the source.
  • Part 6 describes how you can send your work upstream.
If you've made it this far, then you should have a functioning Wine build and an application installed that has some problem that you want to fix.

Debugging Wine is unlike debugging an application or even most open source libraries and software. As a Wine developer, you typically don't have access to the source of the application. So you have to be able to guess at what the application is trying to do, and then figure out why Wine is failing to meet its requirements.

Applications are often debugged with debuggers, but this is of limited use with Wine. A lot of time is spent in application code, which isn't useful to us, or in OS library or kernel code, which is also often not useful to us. Wine is just the go-between from application code to library or OS code, so debuggers tend to spend a lot more time out of Wine code than in it. Problems in Wine are also often less about bugs than they are missing features. Debuggers don't help here.

 

WINEDEBUG


Instead, most Wine debugging is done with printf-debugging. Wine developers have introduced copious logging into the libraries. When you run an application with these debug traces enabled, it will spit out large quantities of information about how the application is using the library.

You enable these logs with the WINEDEBUG environment variable. This variable takes a comma-separated list of debug channels. While it has a more complicated syntax than this, usually you just enable channels entirely. For example, to debug a networking issue, you might start with:

    $ WINEDEBUG=+seh,+winhttp,+winsock,+wininet ~/src/wine.win64/wine Steam.exe
 
This will emit a ton of output on stderr. Usually you redirect to a file:

    $ rm -f out.txt && WINEDEBUG=+seh,+winhttp,+winsock,+wininet 
~/src/wine.win64/wine Steam.exe &>> out.txt
 
Notice that the redirect is in append mode, to allow for more readable multi-threaded logging. This requires that we delete any old log file first.

This is the basic Wine debugging loop. Run the application with logging enabled, demonstrate the failure, examine the log, add more logging if necessary, and repeat until you understand the problem.

 

Debugging channels


Wine trace logs have four "severities," called TRACE, WARN, FIXME, and ERR. TRACE is just general debugging, and is silenced by default. WARN indicates some suspicious, but non-fatal condition and is also silenced by default. FIXME indicates some missing feature of Wine and will be printed by default. ERR indicates a serious problem and will also be printed by default.

Wine lets developers output logs on specific channels. These are declared in the C source files with WINE_DECLARE_DEBUG_CHANNEL and WINE_DEFAULT_DEBUG_CHANNEL. Grepping the C files for DEBUG_CHANNEL works great.

It's not always easy to determine which log channels are useful to trace. If you know what general area your problem is in, you can add channels from the relevant Wine libraries. If you don't have any idea, then a log with just +seh is a decent starting place to dig for fixmes and errs.
There are some special debugging channels.

tid and pid will prepend each log line with the Thread ID and Process ID that outputted the log line. tid is enabled by default in recent Wine.

relay will output every API entry point that the application, or other parts of Wine, make a call to. This is extremely verbose, but can be very valuable, especially if you don't know where the problem lies.

timestamp will prepend every log line with a millisecond-resolution timer. This is great if you can estimate when in the log the problem occurred, or are working with timing-sensitive code like audio.

 

Working with log files


Most WINEDEBUG logs will grow to hundreds of kB and MB of plaintext. Logs of multiple GB are not uncommon, especially relay logs. Many GUI text editors will not handle files of these sizes very well. An editor like vim is recommended.

You should also get familiar with tools like grep, head, and tail to pare log files down to manageable chunks. In particular, grep -v can get rid of uselessly repetitive log lines.
When encountering a new log, a good first step is to grep for err:, warn:, and fixme:. These can indicate problems. You'll likely get a lot of fixmes and probably some warns. If you're getting messages from areas unrelated to the problem you're examining, you can usually ignore them. For example, fixmes from the d3d channel are probably unrelated to a problem with audio.

 

Log file techniques


One generally useful technique is to find the problem and then search backwards. For example, bad pointer dereferences are a common failure. These can be found in a +seh log by searching for c0000005. Once you've found the failure in the log, you can start going backwards and if you're lucky, find some fixme or err indicating a missing feature. Or in a relay log, you might find some API that is returning a failure, like a bad HRESULT instead of S_OK.

If you don't have an obvious crash point, you can also try to find where in the log things start to go bad. For example, most applications follow a pattern of loading libraries, initializing stuff, normal application processing, then uninitializing stuff and unloading libraries. If you have an application that is quitting unexpectedly, you can search for when shutdown-related code is happening and then start going backwards to try to find the culprit.

If you have an application that fails on some specific input condition (say, clicking on some UI element), you can actually write to the log at the same time that Wine is. While Wine is debugging, run:

    $ echo '@@@@@@@@' >> out.txt
Then perform your action and observe the failure. Then run it again:
    $ echo '########' >> out.txt
Now you know the failure occurred between the at-symbols and the hashes.

Full Article

Run Microsoft Windows Applications and Games on Mac, Linux or ChromeOS save up to 20% off  CodeWeavers CrossOver+ today.