Saturday, February 01, 2014

a high-volume, low-latency market data processing system implemented with IBM middleware

Way back in 2003 I worked on a very interesting C++ CORBA project. It was to do with the near real-time distribution of market data to multiple subscribers. Most of the work was contracted out to that well known consultancy, Object Computing Inc. IMHO they did a very good job. One of the things that made it interesting was the way they used an XML configuration file to describe a pipeline of components through which the messages passed. This used the Configurator pattern described in POSA.

Back in January 2013 I came across a system that reminded me strongly of the one I worked on. I found an article in the Wiley Online Library (WOL). Unfortunately the WOL article is behind a paywall. Luckily, the authors also made it available in several other places including one where you get get it as a PDF.

This is a reminder to me that OCI aren't the only ones to develop a system like the one I saw and that there is very good write up about that other system. Here is what is says in the abstract:

A stock market data processing system that can handle high data volumes at low latencies is critical to market makers. Such systems play a critical role in algorithmic trading, risk analysis, market surveillance, and many other related areas. We show that such a system can be built with general-purpose middleware and run on commodity hardware. The middleware we use is IBM System S, which has been augmented with transport technology from IBM WebSphere MQ Low Latency Messaging. Using eight commodity x86 blades connected with Ethernet and Infiniband, this system can achieve 80 μsec average latency at 3 times the February 2008 options market data rate and 206 μsec average latency at 15 times the February 2008 rate.

No comments: