YAMI Home

Concept

Specification

Implementation

What next?

Questions
& Answers

Comments

Implementation

[Downloads] [Internal model] [Connection modes] [Performance]

The following results are not up to date with recent YAMI distributions. In particular, simplex and duplex connection modes significantly vary in performance. New performance test results will be added soon.

Here, I present some of the performance test results with few words of explanation.
This is also a place where you can find how good (or bad) YAMI is when compared to other messaging infrastructures and on different platforms.

Disclaimer

This page is not supposed to advertise nor to mock any of the products and should not be taken as a very detailed and exhaustive description of any technology.
It is possible (and likely) that on different hardware and software configurations or after recompiling the kernels with some specific options the test results would be different than those presented.

The environment

The environment used to test YAMI and compare it to other systems was:

• Pentium III 933 Mhz, 256 MB RAM
  running Windows 2000 or Debian Woody or FreeBSD 4.6.

While the test were running, no other application (apart from standard processes) was active.

What was tested

The tests were written for:

The tests were run on the same machine to measure the actual overhead incurred by each infrastructure and on different operating systems. If the tests had been run across the network, the total cost of sending a message would have been dominated by the overhead introduced by each network hardware component (routers, switches, etc.) and the apparent performance of all the systems would have been almost equal. For example, when sending a message from the North Pole to the South Pole, it really does not matter what particular method was used to accomplish it. Running everything on the same machine shows how fast is the bare mechanics of each messaging infrastructure.

The tests

The test application was a simple client-server system.
In first version, the client had to send a message containing two integer values and the server (a separate process) responded with one integer value (the so-called two-way messaging). The message was sent repeatedly 1 million times and the total time was taken to compute a medium speed of message exchange (in messages per second).
In second version, the client had to send a message containing one integer value and the server did not respond (the so-called one-way messaging).

The YAMI two-way tests were run in three Agent configurations:

• normal, where the Agent is asynchronous
• synchro, where some of the Agent's modules are synchronous
• events, where the applications had to run the message loops by themselves

(please refer to the YAMI Core Library Reference, for Policies documentation to learn about those configurations)

One should expect that synchronous messaging will be faster than normal and that explicit event loops will be faster than synchronous messaging. The tests should show the performance differences.
Note, that the three configurations differ not only in speed but also in flexibility in a way that justifies the existence of all the possibilities. The programmers should consider all the factors (not only the speed presented here) before taking any design decisions.

Raw results

The diagram below presents the results for two-way messaging:

And this diagram presents the results for one-way messaging:

Explanation

CORBA:
It shows that the synchronous messaging model can be really fast - thanks to the notion of object reference, it is possible to set up a channel where the message is sent in the client's thread and the results are retrieved through the same open socket (I can be wrong here, so I welcome any comment on the technical details of CORBA). Moreover, CORBA is statically type-safe and, as a result, the precompiled interface definition allows to prepare tailor-made proxies and stubs for communication. Undoubtedly, they add to the high performance of CORBA systems.

YAMI:
The YAMI Core Library provides a way to exchange messages asynchronously and with no notion of a reference nor interface. These characteristics are very interesting and useful in some contexts (like in scripting languages). However, they influence the design of the YAMI Agent. In particular, it is not possible to send the message and retrieve its results via the same TCP channel. Actually, the response in YAMI does not have to come back from the network address where it was send to, but can arrive from any other Agent after message forwarding. To accomplish this, the YAMI Agent uses separate threads (in the normal configuration) and separate connections for sending and receiving messages. When some fixed set of objects is involved in the communication, the Agent uses the connection pool to speed up the message exchange.

Please note the difference between the YAMI on Windows and on Linux. The library code is exactly the same and the Agent logic is based on the Operating System Layer. On Windows, the API calls are used for thread and network related operations. On Linux, the Posix threads library with API calls are used.

Note that changing the configuration of the Agent (which reduces the number of threads involved in message delivery in the synchro model and in extreme does not use threads at all in events model) results in substantial speed-up in messaging. This is understandable and should be taken into account in performance-sensitive applications.

Note that changes in speed with different configurations are much more visible on Linux and FreeBSD systems than on Windows. This is probably more interesting than the fact that YAMI on Linux is just faster than on Windows. My unconfirmed guess is that socket operations on Windows are a lot more expensive and dominate the total cost of messaging, so that reducing the threading and synchronization overhead has lower influence on the speed.

The thing that requires some comment is also the fact that synchro model had slightly worse performance than the normal model when compiled with MS Visual C++ 7.0. There is probably no other explanation that some other process has influenced the results when that test was run. Anyway - everybody's got equal chances and it would be a dishonesty to repeat the test just to get the expected results.

Final comments

Only YAMI Core Library was taken for tests. This is because the C version of the YAMI library contains the core logic for message exchange and all the other libraries are only thin wrappers that do not introduce any significant overhead on their own. Or at least they should not.