Conflation

The word conflation sounds like a condition that requires you to take a few spoonfuls of Pepto Bizmol. But, it is a term that is used when discussion efficient market data distribution.

The technique has been around for a while. I have always referred to it as aggregation. The term conflation means the combining of data in messages to form one message. You use conflation of messages to overcome slow consumers, limited message queues, and any situation where a message publisher has to throttle the outgoing stream of messages.

Let's say that you are a consumer of stock quotes. The market data distributor receives 1000 quotes for MSFT in one second, but you are only interested in the latest price every second. So, the market data distributor will conflate all of the MSFT quote messages into a single message that gets broadcast to you every second. This means that the server will only send you the last bid/ask price.

You can choose various parameters for the conflation. For instance, as mentioned above, you might want time-based conflation (every 2 seconds, every 30 seconds, etc), you may want conflation based on the size of your message queue, or you might choose your own custom way of doing conflation.


©2006 Marc Adler - All Rights Reserved