Saturday, October 21, 2006

Concurrency, Joe Duffy and Wall Street

Joe Duffy is soliciting ideas for a new book on concurrency.

Joe's name comes up a lot my talks with Microsoft team that supports me. He and Rico Mariani are two resources from Redmond that I would love to get on an advisory basis. My vision is that they would help us out with optimal .Net architectures for low-latency, high-performance systems.

The elephant in the room that everyone is worried about is the projected Opra feeds of 456,000 messages per second. As I am the Global Architect for Equity Derivatives for one of the largest investment banks in the world, this is something that I am responsible for. As such,the things I need for our stack include high-speed market data feeds with caching and conflation, complex event processing, hardware acceleration, object caches, threading models, efficient GUIs and client-side frameworks, hardware acceleration, super-tight code generation, efficient messaging between components, (and did I mention hardware acceleration?).

From a concurrency point of view, I would like to know what Microsoft considers to be best practices to implement quasi-real-time data processing. Also, what anti-patterns exist and what to avoid in the .Net framework. How to use PerfMon and other third-party performance tools to get the most out of my systems. Coding rules that maximize performance. Things that Microsoft will be releasing (or thinking about) five years down the road.

And, for my own selfish reasons, I would love to see Joe implement a trading system to test out his ideas!

©2006 Marc Adler - All Rights Reserved


chris said...

Are we in the same place or what? I was recently in a room working with some auto-trading folks with melting network cards - TOO MANY INCOMING MESSAGES PER SECOND!!! This is yet - another 'hot topic' for us to deal with. Good luck - I will keep you abreast of how we are doing!

I am amazed that all of your posts this weekend are directly the same stuff I am doing (also in equity derivs even)

Don Demsak said...

Did you listen to Joe Duffy's .Net Rocks episode?