Sunday, October 22, 2006

Separated at Birth

Chris and I came from the same company, and we were both equally dissatisfied with the kind of body shop work that we were doing at our last client. We both struck out simultaneously to find a place where we could do meaningful work. We landed in very similar positions at two of the largest investment banks in the world.

Funnier still is that we share almost the same technology stack, from front to back. I am sure that right after the conference call that we have with our market data infrastructure vendor, Chris is on the same call an hour later.

Craig mentions that there are only about 200 market data specialists in the whole world. Everyone probably knows what eachother is doing. The same thing goes on in Wall Street, and by extension, the City. The after-bonus-shuffle will take place in another 4 months, and it's an opportunity for every company to find out what every other company is doing.

When you come down to it, we all have very similar technology stacks. We all know which vendors are out there, and we all have done the same kind of performance and stressing comparisons between similar vendors. The goal is to squeeze that one extra millisecond of performance so that your order is hit before your competitor's order. It could be the difference of one extra 'lock' in a piece of code. It will be a race to recruit the Joe Duffy's and Rico Mariani's of the world. All of the IBs will need to recognize the need for these kind of people, and adapt themselves so that these people will not feel stiffled within a Wall Street environment.

©2006 Marc Adler - All Rights Reserved


Anonymous said...


Any recommended reading to learn more what "market data specialists" do in practice? Or pointers on your software stack etc. Background: I'm a lowlevel systems programming developing an interest in solving such performance problems - processing opra feeds etc - but I'm all compiler/os/hardware background and no financial domain experience. thanks, sp.

marc said...

I'm trying to get our market data architect to write a guest entry to explain what he does.

For performance issues, read blogs like Joe Duffy and Rico Mariani's from Microsoft. We share a lot of the same latency issues that things like radar and imaging systems have .. how to get async data to a user's display as fast as possible. So, become expert on threadimg.

I have to say that a lot of Wall Street companies are interested in great technologists, and they are willing to train you on the domain. Most companies have tons of classes and web-based training to get you up to speed. Plus, if we can ever get Matt to write his book ....

Anonymous said...

Hi Anon,
I’m not sure I've ever met anyone who set out to specialize in Market Data. Its usually a mix of circumstance, coincidence, and unique character flaws that gravitates individuals into these roles. Market Data Land is a paradoxical place. If you are smart enough to solve the real underlying technical problems of constructing a large scale Market Data System then you are more than smart enough to build far easier things that are more common and most likely pay better such as Order Routing Systems. Better still you can just read a few books on the subject and filter out the noise. There are no good books on Market Data - who would be in a position to write them? why would they? They'd be out of date in 3 months.

Market Data Land - climbing Everest in a suit

There are many reasons I wouldn’t recommend a career in Market Data. 95+% of the easy to find jobs are support roles rather than development. The consultancy positions don’t pay as well as easier roles that are closer to the money (i.e. STP/OMS/etc.) and the dev roles that do exist are usually saddled with designs from incapable architects proud of their square wheeled bicycles cobbled together from inappropriate parts. Add to this the entrenched politics and good-old-boys ready with their curve balls and monkey wrenches and you should ask yourself why do you want to climb this mountain?

Because its there. Because you can.

Even in an industry as obfuscated and unorganized as Investment Banking the field of Market Data is special - It’s the last global unregulated technical playing field where individuals can still have global impact. There is no standardization, few coherent regulations, and technical challenges that would seem to be infeasible with even the latest hardware - its always 1987 in Market Data Land. If you have the technical skill and intestinal fortitude then there is a real underlying need for solutions and necessity could, if the wind was just right, allow a wily individual to navigate the minefields and beaurocratic catch nets and to create something significant. And, like all great feats of engineering, it will be around for sometime. What else would you rather do? I always thought I wanted to write video games and push the technology to the limit. Then I walked into my first trading room and found a worldwide MMORPG where the players use real money and the game never ends. If you find the entrance to the secret garden and you have the stomach for adventure jump in and pwn some nubs! Don’t let the Red Queen stop you - that’s for mortals :-)

Just dont go work for a vendor.


marc said...

> Just don't go work for a vendor