HPC Conference Thoughts

Yesterday's HPC On Wall Street conference could be summed up in one word .... PACKED! I thought that this would be a sleepy little conference, with a few crazy architects from a few IBs and hedge funds walking around. When I got there, I could not even move through the (very narrow) aisles that the vendors were set up in. I would guess that there must have been 700-800 people at this conference, and it is at these times that I am thankful that I am not a lightweight, as I merrily tossed my compatriots all over the Roosevelt Hotel in my quest to get to the conference rooms.

The two panels that I went to were, unfortunately, complete wastes of time. Even some of the panelists that I spoke to afterwards were half-embarassed to be on the folically-challenged panels. (I am still wondering what Gideon Low from Gemstone was doing up there in the session on Multicores and Market Data .. although he DID manage to come up with better answers than the Chief Architect of Fidelity!). It only reinforces my opinion that if you are doing something unique and important, your company will never let you speak about it on a panel!

And, thanks to the people from Bear Stearns, who during our lunch with a vendor, planted themselves at our lunch table and managed to kill our conversations.

I was heartened to see a constant flow of traffic at the Digipede booth. In several square meters, you could talk to Microsoft (CCS), Digipede, and Platform (who were displaying at the IBM booth). I was disappointed to see one CEP vendor there (Aleri).

The most interesting new player was STAC, who purports to be an "independent" kind of lab, where both vendors and customers can go to in order to get evaluations and benchmarking done .... sort of like a Consumer Reports magazine. One interesting comment from them was that, in their early experience, some of the claims about feature sets that various vendors make do not actually hold up when it came time to implement the vendors' products. An independent lab for the securities industry is long overdue, and it is heartening to see vendors contribute hardware to STAC in order to "bootstrap" their operations. The two main guys from STAC are ex-Reuters guys, and so you know that they have good experience in market data systems.

There were some real small players (like RTI in the messaging space) who might be ripe for joint ventures or investment. Seeing these guys work the floor reminded me of my days as a software vendors going exhibiting at shows ... an absolutely exhausting experience.

My colleagues have told me that the HPC show has grown exponentially over the past four years .... if I grows any more, then the Roosevelt Hotel is going to have to abandon the cattle car approach, or else, I am going to have to put on a few more pounds!


©2007 Marc Adler - All Rights Reserved

Another New Position (again)

Last week, I was about to finalize my transfer to the Derivatives Analytics desk when an amazing opportunity was offered to me by my company. I cannot blog too much about this new opportunity other to say that it involves Complex Event Processing and transformational new directions for trading and decision making in Equities. This is a huge initiative for my company, is being sponsored by the top people on the business side in Equities, and goes all the way up to the CEO of the company. I will be leading this new project on the technical end, and I am working with one of the movers-and-shakers on the business end.

This is not the standard algorithmic trading scenario that is the standard use case for vendors like Apama, Skyler, etc. We will be pushing the limits of the CEP and messaging vendors. We have the mandate to have all of our Equities systems interact with our. And, if everything goes right, it will transform the way that systems are written within our company.

The nice thing is that the Business is giving us several months of pure R&D time. Reading papers, attending conferences, meeting vendors, etc. For me, it is coming out of a silo and being given the opportunity to learn about every phase of our systems for trading equities. And, it will be building upon all of the things that I have been investigating for the past year in my Architecture role.


©2007 Marc Adler - All Rights Reserved

CEP and HPC Conferences

I will be stopping by the High Performance Computing on Wall Street conference on Monday, Sept 17, and catch up with the new Digipede release.

Wednesday through Friday, I will be at the Gartner Complex Event Processing conference in Orlando.

I am starting to feel like a vendor. However, in my new role, I am basically playing software entrepreneur again, this time in the context of a multi-billion company.


©2007 Marc Adler - All Rights Reserved

.NET Winforms Expert Wanted

We would like to hire a true .NET/Winforms/C# expert (must be Winforms, not ASP.NET). A substantial part of the job would be to take over the .NET Client-side framework that Equities is using (ie: the thing I developed). You must be proficient in real-time front-ends, multithreading, understanding of messaging systems like Tibco EMS, etc.

You would be sitting in beautiful Warren, New Jersey, which is about 45 minutes west of New York City. However, you would have substantial opportunities to work with the business, which is located both in New York City and in Jersey City, NJ.

We would strongly prefer a full-time employee, although we might make an exception for an indie consultant.

This is an on-site job. So, please don't send me resumes from the Ukraine, asking if you can work remotely. Also, no calls from recruiters and consulting companies for now. I want to pick from people who are truly interested in financial services and .NET ... and these people probably read this blog already.

Since the position is in the Equities Advanced Architecture group, you must be a really strong communicator. If I ask you to get up to a whiteboard and explain something, I should be able to understand you without straining.


©2007 Marc Adler - All Rights Reserved

IBM and Limitless "Vacations"

Catch this while the NY Times still has it for free:

http://www.nytimes.com/2007/08/31/nyregion/31vacation.html

Reminds me a little bit of myself ....

©2007 Marc Adler - All Rights Reserved

Bonus Paralysis

I have heard of a number of financial companies who are having a very difficult time hiring .NET developers, especially ones with WPF skills. The main problem with getting financial developers to move companies at this time of the year is Bonus Paralysis. Bonuses are typically announced in December and paid out in January, and nobody is going to sacrifice their bonuses at this time of the year to change companies .... unless ......

... there were little or no bonuses paid out. Because of the credit crisis, a number of news organizations have stated that bonuses this year will be drastically reduced. Everyone is now suitably warned and prepped to receive a significant decrease in last year's bonus (especially if you are in structured finance and credit derivs). And, at this time of the year, a new company would probably not guarantee your bonus.

If you are not expecting much of a bonus, would you stay around?

©2007 Marc Adler - All Rights Reserved

CEP Conference

There is a good chance that I will be at the first Gartner Summit on Complex Event Processing in Orlando, September 19-21.

I am especially interesting to see what Coral8 has to say. My interests are in using CEP for Business Activity Monitoring and Algorithmic Trading.

Matt and Chris have had some good things to say about Coral8 in the past, and if I had some time to breathe, I would like to download their SDK and start mucking around.


©2007 Marc Adler - All Rights Reserved

Working on the Trading Floor

A few weeks ago, Terry left a comment, asking what it was like to work on the trading floor.

I sit on a very large Equities trading floor that overlooks the Hudson River in lower Manhattan. Our particular area of the floor has the Derivatives Analytics group (my group) and Algorithmic Trading. The next row has the High Frequency Trading guys and some of the guys who control the high-speed market data systems.

Needless to say, the past few weks have been crazy. There is such an energy on the floor that, sometimes, everyone is our row just stops working in order to absorb the electricity. 15 minutes before the opening, things start humming. At the bell, it's chaos for about a half hour. People screaming out orders, groups of traders applauding and whooping when a trade turns out favorable. After 10, things really slow down until about 3PM.

Our group is composed of Quants. We trade our own positions, and we write software which enable us and other groups to trade. Almost all of our software is in C#/.NET 2.0, heavy use of generics and anonymous delegates.

The group is ethnically diverse. 4 Americans, 1 Brit, 3 Far-East Asians, and a couple of Russians for good measure (what's a good Quant group without the Russians!). 3 of the guys have PHDs, the rest have MS's. Actually, the head of the group does not have a Master's degree ... however, he got 3 Bachelor's degrees in 3 years at one of the top business schools in the world. Most of the group is fairly young. If you are in your early 30's, you are ancient. That makes me a pile of ash on the floor.

The group has its own framework. The framework is typical Russian mad scientist code. With this kind of code, it's never "A calls B". It's "A starts 5 threads, waits for completion signals, creates other objects via some dependency injection, does a few somersaults, and then calls B." But the framework is highly performant.

One of the things I am doing is writing some risk hedging apps using my own framework that my team developed for Equities. So, I am eating my own dogfood. I have not found any bugs in my stuff, but I have had to extend it a bit so that it cooperates with some of the stuff that Derivs Analytics have previously written (like Live Greek Servers).

As exciting as the trading floor is, I welcome the one or two days a week that I work from home. If I have to do really major thinking and design work, I cannot do it at work! It is hard to work at work. Luckily, a lot of Wall Street is starting to go to a model of allowing people to work from home part time. The three hours I save on commuting to NYC is time that I put into work. Now that my son is off at college, I don't have to worry about his heavy-metal bands coming into the house at 3:00 every afternoon to thrash around the basement.


©2007 Marc Adler - All Rights Reserved

Random Musings

Yes, I know that I have not been blogging very actively this summer, but at the insistence of some colleagues and vendors who demand continuous entertainment, I will try to embark on another series of stream-of-conciousness postings.

Let's see what's new ....

1) London again. More groups coming aboard with my .NET client framework, so I was asked to pay another visit to London during the week of July 16. That's 3 times in London in less than a year. I don't mind going to London ... the first-class service aboard Virgin Atlantic is stellar, and this time, I was treated to massages on both legs of the trip.

What was exciting about this trip is that the department that I was meeting with are going to be taking four of their separate apps and combining them in a single consolidated front-end, with each particular group's app "running" as a separate DLL, registering services, and communicating with the other "apps" through my internal message bus. Still no separate app domains though.

2) My son is in college now! Yeah! Spent the entire weekend cleaning the house of the carnage that he and his friends wrecked during the past few years.

3) Going to be revisiting Complex Event Processing soon. Coral8 looks like a strong contender in the space. Why doesn't KX Systems realize that Q scares the mere mortal?

4) I am taking a few members of the Derivatives Analytics team up in my plane tomorrow evening. A nice flight over the Catskill Mountains is planned. Usually when I ask people to fly with me, I get all sorts of excuses. However, the DA team is too young and too inquisitive to know what they're getting into! They will even help me pre-flight the plane. As long as they don't start asking me about auronautical mathematics, the flight will go well!


©2007 Marc Adler - All Rights Reserved

Panopticon Rebuttal

A few weeks ago, I made the comment that we have had some good success with the free Microsoft TreeMap control to implement heatmaps. I also mentioned the fact that, for our needs, the Microsoft product seemed to suffice and that I did not see any immediate reason to buy the Panopticon product for heatmaps.

The Panopticon people asked me for some time on my blog to address the difference between the Treemap and their product, so I have reprinted their comments verbatim:

“What we at Panopticon Software offer in the product called Panopticon Developer is an SDK for embedding visual analysis and monitoring functionality into applications by means of integrating components such as interactive visualizations and a powerful data model capable of aggregating real-time data streams. The Microsoft Treemap control seems in to be considerably smaller and restricted in its API and designed to be used as a simple chart library and therefore for example does not include analytical functionality. Collateral posted at (public url at www.panopticon.com) provides more detailed information on how Panopticon Developer is compares in relation to more traditional chart libraries.

With this SDK we’ve built a rapid web deployment toolkit called Panopticon Enterprise which provides very low ownership costs, extremely short time to market while preserving the functionality available in the SDK.

We typically work out significantly cheaper than in-house build due to the lack of maintenance and ease of deployment for new requirements”

(end of Panopticon comment .. now back to Marc)


Like any company, we have a budget that is tightly controlled. If Panopticon, or any third-party product, provides fiunctionality that will cost us more to develop ourselves, then the decision is to go to a third-party product. We like the fact that a third-party will support the product, but on the other hand, custom developing something ourselves gives us tight control over the feature set without having to wait for release cycles from a vendor. But, if we can take an existing free product and, with relatively little effort, modify it to do what we need, then I would be neglecting my fiduciary responsbility as an officer of the company to purchase a third-party product and pay a continuing stream of license and maintanence fees.

We are only at the beginning of exploring what heatmap technology can do for us, and in addition to Panopticon, there are other fish out there like Fractal Edge (makers of what I like to call the Undulating Amoeba heatmap!)

If there are any of you out there who have used or evaluated heatmap products, I would love to get your comments.



©2007 Marc Adler - All Rights Reserved

It's the Models, Stupid

I just can't think that, in the world that we deal with, our efforts spent on low-latency, code optimization, high throughput, best algo containers, etc ... are worthless if the models are not good.

Doesn't matter how fast you get your trades out to the exchanges if you are always on the wrong side of the trade.


©2007 Marc Adler - All Rights Reserved

Trading Floor

I am straddling two positions at the moment. I am still at my current position as SVP of Equity Derivatives at my company, but I am also doing a slow transition to the Derivatives Analytics desk on the business side. So, for the latter role, I have been sitting on the massive Equities trading floor at my company.

The contract in the physical difference in the two roles is stark. In my position in Architecture, I sit in a big office in an old Lucent campus in the middle of North/Central New Jersey, about 35 miles from Allentown, Pa. The place is almost like a morgue ... I could go there, sit in my office, and not see anyone for an entire day.

On the other hand, the trading floor is located in the middle of very hip Tribeca. This is not your father's Tribeca. You have 20 gourmet restaurants within two blocks of my office. You can look out the 20-foot windows that overlook the Hudson River and see the Statue of Liberty. You can see pretty girls walking across the floor all day long.

When I am sitting at the Derivatives Analytics quant desk, there is the amazing energy that accompanies a trading floor. I was lucky enough to be on the floor for these last two trading days, where the Dow dropped 500 points. There was cheering, shouting, tension, madness. It reminded me of the old days, when I started out on Wall Street as a consultant for Goldman Sachs, and then went to consult for Drexel Burnham. In those days, we developing the first cut of Windows-based trading workstations, so we would always get traders who looked over our shoulders, commenting on our work.

It has been no secret that I am fascinated with the quant world. I am sitting with truly brilliant people who talk a different language that I barely understand. Even though the quants at my desk are 50% Russian, they talk in English, throwing around terms like Kurtosis and Skew Risk, terms which have me madly scrambling to Google. Terms which may me wish that I had not shown up drunk to Statistics 101 class every day in college!

It is interesting to have these quants coming up to me, apologizing for their code. Maybe I can teach them a little bit about software and architecture, and they can teach me how to Delta Hedge my portfolios!

©2007 Marc Adler - All Rights Reserved

Next Gen Heatmaps of the Markets?

The next generation of heatmaps?

http://www.ted.com/index.php/talks/view/id/129

Can we also employ tactile feedback using Microsoft Surface? So we can FEEL the market? Make the hot areas really generate heat?

©2007 Marc Adler - All Rights Reserved

The Flavor of the Month is Acropolis

Now, after touting CAB for a few months, Microsoft has announced Acropolis.

A few months ago, the Microsoft DPE team came to me and tried to dissuade us from developing our .NET client framework in favor of using CAB. However, during those conversations, the word Acropolis came up a few times.

Now that Microsoft is touting Acropolis, what has become of CAB? Will this now be an officially supported framework (remember ObjectSpaces?). What about all of the financial service developers that committed to CAB? And, now you have some Microsoft people telling us that CAB was never officially supported by Microsoft, that it was developed by a small group of independent developers in some far off country that nobody ever heard of!

One thing about developing your own framework and making it a standard in the company is that it won't turn into a flavor-of-the-month entry. Goldman lead the way with OCEAN. Wachovia has OneSource, and Morgan Stanley has FidSharp and Concord. And we have ours. And, our companies tell our developers that they should use it.

No doubt that Acropolis will be "better" than CAB, and will have learned lessons and anti-patterns from CAB usage. Only, what will Microsoft come out with in another 6 months?

©2007 Marc Adler - All Rights Reserved

Short Sell Any Time

(Thanks to Wall Street Folly)

The SEC voted unanimously on Wednesday to finally get rid of the pesky and ridiculously outdated tick test. The tick test only allows short sales when the last sale price is higher than the previous price. That means a trader cannot short a stock if the movement prior to the short sale is down

Wonder what this means for all of the models out there? (I mean HFT and Algo models, not Giselle Bunchen-type models... but, I am sure that Heidi Klum is overjoyed too!)


©2007 Marc Adler - All Rights Reserved

Fun with Patents in Financial Services

It's great fun to browse the United States Patent Office to see what your colleagues have come up with in the financial industry. It's even more fun to see who has patented what. In the search page, type your favorite company into the "Assignee" field. Even though you and I can create a great technology, and we *may* be able to be listed as the Inventor, we usually have to Assign the work to the company we work for.

Just take a look at one of the patents. It's from Goldman Sachs. Andy Smith is the creator of Goldman's .NET client framework, OCEAN. In that respect, we are counterparts.

It's also fun to se who was credited as the Inventors of a technology. For example, some Merrill Lynch patents list (the late) Don Trojan and Ed Keenan as the Inventors. I know for a fact that Don and Ed were managers at Merrill, and in my opinion, probably never got anywhere near the technology. (Ed was my wife's first boss at ML, and was a pure manager. Don was my wife's boss at Drexel Burnham Lambert.)

If you want to see the diagrams that are associated with the patent, then click on the Images link. However, you need to download a TIF viewer for your web browser before you can view the images.

As I write this, let's see if some of our favorite characters have patented anything....

John Powers of Digipede : Nothing.... But don't look at this and this.

Geva Perry of GigaSpaces : Nada .... (Maybe they have patents in Israel?) But let's try one of GigaSpaces' competitors, Gemstone. Well, lookee here and here. And here.

You guys should learn the lessons of old Xerox, and make a beeline to your favorite patent attorney! Patents are some of your greatest assets.

©2007 Marc Adler - All Rights Reserved

Google Spam

I have been fairly sparse on this blog lately. I have been incredibly busy at work. But, the main reason is that Google had mistakenly tagged my blog as a spam blog.

Everything is back to normal now.

©2007 Marc Adler - All Rights Reserved

Favorite new blog

http://enhyper.blogspot.com/

A lot of good stuff on implementing HPC within financial services firms.

The author of the blog just took a job with 29West in London, the makers of LBM (Wombat is a big user of LBM). He regularly hosts drinking sessions in London with interesting guest speakers.

(Since I showed Terry the blog, he has already found more reasons to stay late at work. Sorry, mate!)

©2007 Marc Adler - All Rights Reserved

Object Caches for .NET

Reminder to self ---

Now that Tangasol, Gigaspaces and Gemstone have native .NET APIs for their caches, I need to re-evaluate the three. The advantages of distributed object caching is being picked up by some other groups in my company.

©2007 Marc Adler - All Rights Reserved

Heatmaps

I decided to integrate the Treemap control (ie: Heatmap) from Microsoft Research into my .NET client-side framework that my company now uses. As a demo, I emulated the heatmap found on the Smart Money site. The free Microsoft heatmap was fairly easy to develop for, and in almost no time, I had real-time trades streaming from KDB+ into the framework, with the trade data displayed in the heatmap.

Given what we have, it may be more difficult to rationalize paying that big yearly license fee to Panopticon.

The Bubble Chart control is worthwhile too.


©2007 Marc Adler - All Rights Reserved

WPF and Vol Surface Editors

Microsoft came in the other day and showed some Silverlight and WPF demos. Infusion Development wrote a demo of a vol surface viewer (not an editor). According to the guys from MSFT, you can actually embed the WPF control within a .NET 2.0 Winforms app.

Our main col surface editor was written upteen years ago, using Open GL. It would be interesting to see how a WPF vol surface/curve editor would compare, especially when the CPU is busy handlign feeds and calculating greeks.


©2007 Marc Adler - All Rights Reserved

Anyone going to SIA?

I will be at the SIA convention on Tuesday, June 19. Big Microsoft/Intel party afterwards.

Anyone going?

©2007 Marc Adler - All Rights Reserved

Looking at ACE

Looking at high-performance frameworks for writing distributed systems, especially for high-frequency trading. ACE has been on the radar screen for a while.

http://www.cs.wustl.edu/~schmidt/ACE.html

I also found out that Bloomberg is using it in their PhatPipe product.

©2007 Marc Adler - All Rights Reserved

Performance Architect Wanted

Haven't posted in a while ... I have been super busy. My .NET client-side framework is taking off in a big way. I am off architecting other interesting things that bring me in touch with messaging acceleration, KDB+, visualizations, and more.

My new intern starts on Tuesday. Every year, we get a bunch of interns from the top universities who have just completed their Masters or PHD in comp sci or physics. Mine will be devoted to expanding and maintaining my framework, but he has a lot of interests in hardware accel too.

Ted, our performance architect, has left the firm for a great new job at AB. So, we are looking for a great performance architect to work in Warren, New Jersey. Let me know if you are interested.

©2007 Marc Adler - All Rights Reserved

Somasegar, who I gather is the head of the Visual Studio product line, blogs that Microsoft will be giving full support to multicore processing in their dev tools.

When I was out in Seattle a few weeks ago, I met with the Microsoft Accelerator team about the future of their technology. Also their stuff is wed to GPU's right now, the Accelerator team claims that it can be used as an abstraction for multicore systems. The guys from the Accelerator team also predict that the world will eventually be moving to massively multi-core machines (ie: Intel's 80 core processor)

Word is that both Credit Suisse and Goldman have teams devoted to exploring FPGA technology. The problem with FPGA is the amount of effort that goes into porting code, and the entire develop/build/debug cycle is very slow.

It will be nice to get first-class support from Microsoft as far as development tools go.

©2007 Marc Adler - All Rights Reserved

CASTrader Blog

Came upon this while following blog links:

http://www.castrader.com

(Chris, you probably know about this one already...)


©2007 Marc Adler - All Rights Reserved

Any impressions of the Microsoft FinDev Conference?

I was on vacation with the family this past week, hiking in Moab (Utah), Ouray (Colorado) and the Great Sand Dunes (Alamosa, Co). Therefore, I sadly missed the Microsoft Financial Developer's Conference.

Anyone care to leave their impressions of the conference?

©2007 Marc Adler - All Rights Reserved

Airplane Boarding and Messaging

Interesting discussion on Marc Jacobs' blog on the airplane boarding process and some parallels to distributed programming. Also, one of the commenters on the blog gave this link:

http://www.math.duke.edu/news/awards/MCM2007lmw.pdf

At the airport, I always fantasize myself with a Jedi light sabre in hand, lopping off the heads of those people who block the aisles for ten minutes, trying to shove some ridiculously-sized carry-on into the overhead bin.

Of course, my solution is to equip everyone with their own private jets, ala The Jetsons. However, the New York Times threw cold water on this idea.


©2007 Marc Adler - All Rights Reserved

The Boss and Blogging

©2007 Marc Adler - All Rights Reserved

KDB+ Architect Needed

I need a KDB+ architect to work in my group. Please email me at magmasystems at yahoo dot com if yiu are interested.

©2007 Marc Adler - All Rights Reserved

Finetix and Sungard - It's Official

As pointed out by Matt:
http://biz.yahoo.com/bw/070425/20070425005824.html?.v=1

Funny thing that no comments from the owners of the acquired company was made in the press release, going against the tradition of each party stroking eachother's backs.

This acquisition harkens back to the 2001 buyout of PriceWaterhouse Consulting (PWC) by IBM Business Consulting Services (BCS). The consultants of PWC were paid about 40% more than the equivalent consultants at BCS. Immediately, many of the PWC consultants experienced "downwards salary adjustments". For several years after that, the BCS consultants got no merit increases, nor did they get anything that resembled a bonus. Many of the best PWC consultants left.

I know for a fact that SunGard does not pay the same excellent salaries that Finetix paid. (I have been contacted many times by recruiters for product and project management positions at Sungard, and the total comp was about 2/3 of the Finetix comp.... maybe the technical consulting positions pay better?) It will be an interesting next few years for the Finetix employees that decided to stay with the new SunGard organization. I know that, when I was hiring for Finetix, I received resumes from a number of people who were employed by SunGard. I cannot help but see how this cycle will not repeat itself.

Best of luck to all of my old colleagues from Finetix! You know where to reach me. My group has a number of openings.....

©2007 Marc Adler - All Rights Reserved

Resharper Suggestions from a Colleague

From a colleague...

(from Tim)

- A way to split all the classes in a file into their own files (e.g. I’ve refactored one class into several, I’m happy with my new class hierarchy, and I want to now make sure I have one class per file)

- A way to copy XML documentation from the base class or interface (it will do this if you ask it to implement interface members, but after that you have to copy & paste)

©2007 Marc Adler - All Rights Reserved

TopCoder.com

A Thoughtworker who is with my group was saddened to find out that, for some reason, Google Code is blocked by our corporate firewall. I suggested that, in addition to the usual suspects, he check out the software at TopCoder.com.

There are more components and apps written for Java, but the .NEt world is well represented on that site.

I have also heard that some of the financial companies are starting to recruit from TopCoder. It's a good way of examining a candidates coding technique and architectural insight. It's a better way of screening a candidate, rather than going through the same old "Describe Polymorphism to Me" questions.

©2007 Marc Adler - All Rights Reserved

Help Improve Resharper

Jetbrains is soliciting feedback and feature requests for their next version of Resharper. Fill out the questionaire here

Off the bat, I would like:

1) A way to tell Resharper to fix an entire file in one fell swoop, without having to navigate to each error or warning.... especially when resolving Redundant Qualifier warnings.

2) A way to Encapsulate all local variables in one fell swoop without having to navigate to each variable, right-clicking, and choosing Encapsulate Field.




©2007 Marc Adler - All Rights Reserved

Sunguard Consulting Services

People have been coming to me with news of a new consulting company ... Sunguard Consulting Company (I thought that Sunguard already had a consulting company).

Looks like Chris Conte saw some value in my old employer, and made an offer. But, according to my various sources, the only catch is that 50% of the consultants have to stay in order for the deal to go through.

Are Lab49 and Infusion next? They would certainly make a tasty snack for Larry Ellison.


©2007 Marc Adler - All Rights Reserved

Street#Grid

Anyone going to the Street#Grid conference on April 16 at the W Hotel in New York City?

Maybe meet up for a drink afterwards?

©2007 Marc Adler - All Rights Reserved

QuickFix and Parsing

I am amazed that, in the .NET API that QuickFix offers, there is no method or MessageFactory that will take an arbitrary FIX string and return a FIX message object of the proper type. Am I missing something?

I believe that FIX4NET has this functionality, but they do not support FIX 4.4.

We have to petition Mike Roberts to get a .NET version of the TTConnect FIX Engine.

©2007 Marc Adler - All Rights Reserved

Solution: QuickFix to Enums

To convert a QuickFix type that has const int's or const char's that represent values into a enum, you can do this:

        static public Type EnumOrdType = new ConstCharsToEnumCreator<OrdType, char>().Convert();

        static public Type EnumProduct = new ConstCharsToEnumCreator<Product, int>().Convert();

Now, if we want to fill a combo box with the values of these objects,

            this.cmbOrdType.Items.AddRange(Enum.GetNames(OrderBO.EnumOrdType));

            this.cmbProduct.Items.AddRange(Enum.GetNames(OrderBO.EnumProduct));

The definition of the helper class is:

    public class ConstCharsToEnumCreator<TQuickFixType, TEnumBaseType>

    {

        static private AssemblyBuilder m_assemblyBuilder;

        static private ModuleBuilder m_moduleBuilder;

        private EnumBuilder m_enumBuilder;

        private Type m_createdEnum;

 

        private Type m_typeThatHasConstChars;

 

        public ConstCharsToEnumCreator()

        {

            this.m_typeThatHasConstChars = typeof(TQuickFixType);

 

            if (m_moduleBuilder == null)

            {

                CreateCallee(Thread.GetDomain());

            }

        }

 

        public Type Convert()

        {

            CreateEnumBuilder();

 

            FieldInfo[] fields = this.m_typeThatHasConstChars.GetFields();

            foreach (FieldInfo field in fields)

            {

                if (field.IsStatic && field.IsLiteral && field.IsPublic && field.FieldType == typeof(TEnumBaseType))

                    m_enumBuilder.DefineLiteral(field.Name, field.GetRawConstantValue());

            }

 

            m_createdEnum = m_enumBuilder.CreateType();

            return m_createdEnum;

        }

 

        private void CreateCallee(AppDomain myAppDomain)

        {

            // Create a name for the assembly.

            AssemblyName myAssemblyName = new AssemblyName();

            myAssemblyName.Name = "EmittedAssembly";

 

            // Create the dynamic assembly.

            m_assemblyBuilder = myAppDomain.DefineDynamicAssembly(myAssemblyName, AssemblyBuilderAccess.Save);

 

            // Create a dynamic module.

            m_moduleBuilder = m_assemblyBuilder.DefineDynamicModule("EmittedModule", "EmittedModule.mod");

        }

 

        private void CreateEnumBuilder()

        {

            // Create a dynamic Enum.

            string enumTypeName = string.Format("{0}.{1}Enum", this.GetType().Namespace, this.m_typeThatHasConstChars.Name);

            m_enumBuilder = m_moduleBuilder.DefineEnum(enumTypeName, TypeAttributes.Public, typeof(TEnumBaseType));

        }

    }

©2007 Marc Adler - All Rights Reserved

Eclipse

I had to do an in-depth examination of a Java AppServer that our group has written in order to do some prelimiary integration of my .NET client-side framework. I finally have had a chance to do some in-depth work with Eclipse, and there is definitely a lot of goodness in it.

I have to admit that there are a lot of positives about being in the Java world. I like the Java Community Process, and the fact that major functionality is discussed in the open before the spec is drafted. There is a lot that Microsoft can learn from this.

©2007 Marc Adler - All Rights Reserved

QuickFix and Enums

It seems that the latest version of QuickFix has a .NET assembly and FIX 4.4 support, so I am going to be using it in an OMS demo that I am putting together to show off my .NET client-side framework.

One thing that I wish that the QuickFix guys did was to use enums instead of static const chars. For example, here is the definition of the OrdType class (this class is written in Managed C++, but a lot of the other .NET classes are in C#).



public __gc class OrdType : public CharField
{
public:
static const int FIELD = 40;
static const __wchar_t MARKET = '1';
static const __wchar_t LIMIT = '2';
static const __wchar_t STOP = '3';
static const __wchar_t STOP_LIMIT = '4';
.............

OrdType() : CharField(40) {}
OrdType(__wchar_t data) : CharField(40, data) {}
};


I completely understand why they used const chars, but for data-binding purposes, it is easier for me to use enums.

I wonder if I should reflect over a few classes and translate these classes to corresponding enumerated types.

I'm supposed to be a manager .. I have to find other people to do this kind of stuff for me. However, in my group, I am the only .NET person in a vast sea of Java-ness.


©2007 Marc Adler - All Rights Reserved

ITSDoc.org Redux

ITSdoc.org is still very much alive. It recently passed its millionth page view and it gets hits from all over the world every day. It strives to be helpful to coders (hopefully open source) of Investment and Trading Systems by assembling into one place a great set of documentation.

Personally, I have been drawn away from driving ITSdoc forward for about a year now volunteering actively at FIX on a standard that will cover the next generation Algorithmic order types. I’ve got to say, volunteering at FIX and mixing it up with the fantastic people in the various technical Work Groups and Global Technical Committee has been wonderful! They have a bunch of brilliant people who all selflessly volunteer tons of time to the betterment of the industry through a FREE standard. Plus, it’s been a real great eye opener on exactly how the largest industry players cross connect. Mind you these same people go back to their firms and compete vigorously.

Last summer our Algorithmic Order Standard Work Group conduced in-depth interviews with five MAJOR order management system (OMS) vendors as well as considerable secondary research into the adoption trends in algorithmic orders types. Much of that learning was incorporated into the proposed standard, which is in the final stages of official adoption at FIX. In just a few weeks its expected to enter the pilot testing phase. See http://www.fixprotocol.org/working_groups/algowg/documents

FIX is a bit more complex than the average open source coder of Investment and Trading Systems might like to struggle with, starting with the printed protocol specification alone. There are many tags and enumerated values, the various messages can be complex and there is a substantial learning curve. There is one outstanding open source FIX engine out there QuickFix, available in C++ and Java, but due to the complexity of the FIX standard it’s quite complex to implement. In the future ITSdoc hopes to help lower the bar to adopting FIX, and help get around using all those nasty, expensive, dedicated IP circuits.

Dealing with emerging ISO standards this past year has also been challenging. If you are designing an open source Investment and Trading System your internal data models need to comply with various standards (including various versions of standards, service packs, extensions, customizations, variations in usage, etc). Needless to say all these different “flavors” and “dialects” of the various standards often don’t mesh well with each other. However, the situation is slowly improving. XML is now everywhere and XSLT can ease (but not eliminate) the pain of making translations.

The ITSdoc UML project is on hold for the moment. I went to write up a meta Use-Case on processing text news during investment decision-making and got mired down with various sub topics such as bias in news coverage. My background includes some work in cognitive psychology and I was perhaps too far down in a “deep dive” on the subject. When I caught myself with a new book on psycholinguistics I thought to myself, you’re in pretty deep, perhaps its time for some decompression. Still the material is fascinating and to a degree exceedingly relevant to investment decision-making. Getting a handle on it however has proven very time consuming. I have a ton of great material but getting it all into publishable format has been constrained by the “hours in the day” limit.

So, plenty going on behind the scenes at ITSdoc. Coders of open source Investment and Trading Systems are very much encouraged to register at the site. We shoot for quality over quantity. Plus, we can often hook people up with other “birds of a feather” who are interested in similar things.

Rick

Http://ITSdoc.org

©2007 Marc Adler - All Rights Reserved

FIX serializer for .NET

Anyone know of a FIX (and FAST and FiXML) serializer for .NET ... something that will reflect over an arbitrary object and produce a FIX string? I imagine that you will be able to tag an object with custom attributes that let you specify the FIX tags.

©2007 Marc Adler - All Rights Reserved

News from the Front

Wall Street Tech writes on the hardware arms race going on now in Wall Street. According to the article, names evaluating hardware acceleration include Merrill Lynch (IBM Cell processor), Citigroup, and Mellon Financial.

Oracle just bought out Tangasol. I predict a Microsoft/Gemstone deal in order to make the Windows CCS stack a bit more attractive. You heard it here first. Gemstone just came out with a native .NET implementation of Gemfire, so it probably makes much sense for Microsoft to gobble them up before Oracle uses the .NET version of Tangasol as a lever to sell more Oracle DBs into Wall Street.


©2007 Marc Adler - All Rights Reserved

Windows Workflow and Auto-Execution?

In preparation for Joe's lecture next week on Windows Workflow Foundation (WF), I am reading a book by Kenn Scribner that just came out on Microsoft Press.

WF has a rule-based workflow engine. I am wondering how application this would be for auto-execution engines.

Also, the middle-office guys have asked me if WF can be used to replace Fuego. I don't know anything about Fuego, but hopefully, Joe can answer this.

I am most excited to see how I can integrate WF with the .NET client framework that my team is writing. I know that Merrill Lynch is doing a lot of things with WF, so it looks like there are some good use cases in the capital markets sector. OMS's? EMS's?


©2007 Marc Adler - All Rights Reserved

My Son's Band at The Starland Ballroom

Cartesia, my son's band, played at the Starland Ballroom on March 17, 2007. Starland is the local mecca of hard rock, punk, and metal in Northern New Jersey. My son is the skinny thing playing drums and singing.

©2007 Marc Adler - All Rights Reserved

New York XTC Club

Mike Roberts hosts the monthly meeting of the New York XTC club, held on Tuesdays at some place that serves an awful lot of Belgian beers. This is a good excuse to make the trek into NYC on a Tuesday.

By the way, XTC stands for "eXTreme Chugging" ... but seeing what kind of state Mike was in, it probably stands for "eXTreme Chundering".

Congrats to Mike for leaving Finetix and taking a new position. A long time coming...


©2007 Marc Adler - All Rights Reserved

I Got a Tech Writer !!!

Read it and weep, people ....

I got myself a real live technical writer for a few months. She has even done Visual Studio and MSDN documentation for Microsoft.

In all of the major projects that I have done for IBs, I have never seen the budget for a tech writer. We usually have to do it ourselves, after we are totally exhausted from delivering the technical aspect of a project. We might be lucky to get a BA to do some of the functional docs, but they will never dive into code.

This young lady who I now own actually installed Visual Studio on her laptop, and started stepping through the code!

©2007 Marc Adler - All Rights Reserved

Grids and Scavenging

During our meetings with the Windows Compute Cluster Server team last week, it came out that CCS does not do any sort of scavenging, nor or their immediate plans to implement it.

Scavenging is an important way to lower the costs of the grid and to gain wider acceptance by letting groups make some money by charging back the use of their spare CPU cycles to the organization. Imagine all of those hundreds of idle CPUs gathering dust in a large Disaster Recovery center at an investment bank. I think back to Morgan Stanley's DR center, with hundreds of cubicles of untouched CPUs.

Would the absence of a scavenging feature eliminate a grid for you?

Does DataSynapse or Digipede have scavenging?

©2007 Marc Adler - All Rights Reserved

New Database TPC-E test Stock Market Scenarios

http://news.com.com/Database+servers+get+a+new+yardstick/2100-1010_3-6168201.html?tag=nefd.top

(Thanks to Jeremy for the link)

©2007 Marc Adler - All Rights Reserved

Microsoft - You're Almost There

I went out to the Microsoft MVP Summit as one of the 20 or so specially invited "Industry Influencers"... which means two things. First, we are supposed to be able to influence Microsoft's future directions. Second, we are supposed to be shown the love by Microsoft so that we can buy lots more products.

What I came away with was that

1) Microsoft is stil very much focused on the desktop and the world of two-tier apps.

2) There are a lot of parts that can be assembled into a killer stack for cap markets, but that there is no overall strategy to congeal the parts.

3) Microsoft needs to look beyond its borders to what is happening in the industry, and to integrate better with third parties.

I get the feeling that we are on the cusp of an incredible stack from Microsoft, but the parts needs to be put together in a cohensive manner. Mike Walker is charged with providing reference architectures for the financial vertical, but Mike needs some assistence from us in understanding the trading stack.

©2007 Marc Adler - All Rights Reserved

The canned presentations of LINQ/PLINQ at the MVP Summit were very interesting, but more questions were raised than were answered. It also was apparent to us Capital Markets people that Microsoft still does not treat our environment as a first class citizen.

Here are some of the questions I had:

No support for async operations (Dinesh from the LINQ group told us that teh query operations are blocking)

No Support for notifications. In many scenarios, if records on the back-end database change, our apps need to be notified asynchrounously. This is what Service Broker is designed for. I would like to see first-class language or API support for receiving async db notifications to a callback function or delegate, or maybe have something integrated with ObservableSet.

Support through WCF or other mechanism for 3-tier apps using transports like Tibco RV/EMS and MQ. LINQ really seems designed for 2-tier apps.

Integration of PLINQ with efforts like Compute Cluster Server.

Support for security and integration within C#

Comparisons with (N)Hiberate and IBattis. These are used extensively in CapMarkets.

Integration with object caches like Gemfire and Gigaspaces.

Possible extensions for time-series (ie: like Streambase)


©2007 Marc Adler - All Rights Reserved

A Comment from Gemstone

Ocassionally, someone will make an important comment to a post I made several months ago. Such is the case now ... Jags from Gemstone replies:

You have hit the nail on the head. At GemStone, we recognized early on that a distributed cache that supports notifications has no choice but to ensure that object change notifications can reliably reach consumers. Else, it pretty much becomes useless.

There are many considerations we took into account:-

Applications are sharing data through a p2p cache and the cache supports dynamic membership - i.e members can come and go at any time. Great from a scalability perspective, but what happens when you have apps connected to the peer cache expecting notifications and a peer node generating the events suddenly departs? In GemFire, we introduced a declarative means to ensure that messages are received by all critical members - each producer or consumer can play one or more roles; A publisher only succeeds if the roles that are required are available. If this interest you, checkout section 3.2 here

http://developer.gemstone.com/display/gfedev/Topologies+and+Architectures+for+Middle-Tier+Caching-

In the scenario you mention - clients connected to a logical cache - we also support asynchronous reliable notifications through in-memory replication of the queues on the cache servers. The challenge here is to ensure in order delivery of events, detecting the failure conditions, automatic client failover to the server managing the secondary queue and avoiding duplicate notifications. And, given that there might be a local cache on the client, the server side can further optimize transmission to the client by merely propagating a object "delta".

One important reason why financial institutions like the whole "caching + messaging" story is because now you enable applications to connect to a distributed object repository that gives pub-sub semantics, but all the contextual data required by the application to make instantenous decisions is available in memory.


©2007 Marc Adler - All Rights Reserved

Off to the MVP Summit

See some of you at the Microsoft MVP Summit starting on Monday evening?

What's important to me is to see Microsoft's directions on Capital Markets. I would liek to see a cohensive story on:

- Grid computing (Windows CCS and possible synergies with Digipede)

- Quasi-realtime Analysis Services, with associated WPF visualizations

- Support for parallel constructs in C#/NET (Microsoft Research Accelerator for GPUs)

- Any forward movement for Complex Event Processing


©2007 Marc Adler - All Rights Reserved

StreamBile

We love bile here! Here is a little bile from Mark Palmer on Apama:

http://apama.typepad.com/my_weblog/2007/03/streambase_algo.html

while giving some kudos to Kaskad. According to this page, Mark Palmer is the VP of Complex Event Processing at Progress Software, a competitor to Streambase.

Judging by some recent comments on Matt's blog, it seems like the CEP vendors relish in sniping at eachother. More bile! We love it!

Meanwhile, we hear from Terry that Gemstone has been impressive with their recent release of a native .NET client. Coming right on the heels of similar announcements from Gigaspaces and Tangasol.

©2007 Marc Adler - All Rights Reserved

Bonus Satisfaction Results

http://news.hereisthecity.com/news/business_news/6552.cntns

Results of a survey about the 2007 bonuses

©2007 Marc Adler - All Rights Reserved

Thanks to James

.. for pointing out something something simple and forcing me to spend a half hour simplifying my code ....

Sometimes people justg get tunneled into thinking about a piece of code in a certain way .. that's why we have code reviews...

©2007 Marc Adler - All Rights Reserved

Antelope Natural Language Processor for .Net

http://www.proxem.com/

Free NaturalLanguage Processor for .Net 2.0.

Worthwhile if you are going to construct a semantic analysis engine from scratch.

I have been reading up on NLP. Take the sentence "Time flies like an arrow.". How many different semantic meanings does this statement have? Now imagine getting a news feed of financial information. How would you parse the news into an ontology that equity traders can work with? And how would traders use this information?

Matt is exploring the CEP engines right now (Streambase, Coral8, etc). I would be interested to see if and how these CEP engines can handle news feed. What kind of info would they need?

©2007 Marc Adler - All Rights Reserved

Flying with Rich

I went flying this past weekend with Rich. It was very windy. It was so windy that, when we pointed the nose directly into the wind and put down the flaps, we were able to make the plane go backwards in the air.

Rich flew co-pilot in my Mooney 20G. Rich owns a very big mutual fund. Rich has a TBM 700. I hope I fly Rich's plane some day.

My wife always asks "Why don't you work for Rich?" I say that friends and business never mix, and friendship is always more important. Silly Marc.

©2007 Marc Adler - All Rights Reserved

One of my Son's New Bands

They are playing at the Starland Ballroom (the local mecca for punk, hardcore and metal bands) in Sayreville, New Jersey on March 17.

©2007 Marc Adler - All Rights Reserved

Barcap Boom

Word has it that the developers at Barcap are jumping for joy at their bonuses this year. After several years of sub-par bonuses, and difficulty last year in bringing new developers on, Barcap finally saw the light and rewarded their hard-working devs with a decent chunk of change.

Companies should realize that Wall Street is an extremely small, tight-knit community, and that word of good and bad bonuses spread like wildfire. In order to retain good talent and hire new talent from other companies, a firm has to have a good reputation with regards to bonuses in the IT department. It doesn't matter if your traders and sakesmen are getting 300% bonuses ... it's the IT department that has to write and maintain the systems that drive trading operations.

Consider the cost of losing the only person on a team who knows the code of a complex system. Think of the time you have to spend hiring a replacement (taking time from employees to do the interviews, headhunter fees, sign-on bonus, etc), doing the knowledge transfer, and waiting for the perosn to get acclamated to the system. How much does a company spend when it loses an experienced hire and tries to bring on a new one? Would that amount of money, if applied to a bonus, suffice to retain a talented individual?

Often times, it is not the fault of the IT managers. It's a matter of convincing the business that IT really matters.

©2007 Marc Adler - All Rights Reserved

IBM's Patterns for Enterprise Grids

An IBM Redbook on patterns for grid computing.

Separated at Birth

Don Demsak (DonXml) and Kent Tegels (Enjoy Every Sandwich).

Kent is the SQL Server curriculum developer at DevelopMentor. I took his most excellent course this week. I was surprised to see that I was the only person with a manager title in there. Surprisingly, lots of Visual FoxPro people were there. I didn't know that anyone used VFP in a corporate environment.

Don, Kent and I hoisted a few beers and claws at Ulysses Pub on Stone St, right in back of the Goldman Sachs building. Both will be at the MVP Summit in March, and I hope to attend sessions that are being hosted by their respective groups. I also hope that a lot of the "shortcomings" in SQL Server 2005 that Kent pointed out will be addressed by Microsoft in future releases.

©2007 Marc Adler - All Rights Reserved

Spring.Net supports JMS

True to his word, Mark Pollack has delivered JMS message integration in the latest version of Spring.Net.

I also need to check out ActiveMQ. They have integration with C#/.NET through NMS. It would be interesting to see if there will eventually be some integration with WCF. I will certainly bring this up at the MVP Summit in March.

©2007 Marc Adler - All Rights Reserved

Syncfusion for Hierarchies?

According to Chris, the Syncfusion Grouping Grid is giving his developers heartburn when it comes to all of the hierarchical and grouping related functions. Syncfusion certainly has its place in a simple, one-dimensional world, and will probably be continued to be used by Wall Street companies for real-time blotters. But this opens up the door for companies like Infragistics and DevExpress to come up with a really fast, bug-free, grouping grid for the kinds of blotters that we know and love so well.

©2007 Marc Adler - All Rights Reserved

Farewell to The Scotsman

Our token Scot has moved on to make Market Data Magic at another firm. One of the truly great market data experts in the world ... I am sure that we will be meeting at the Brandy Library for a snifter or two, if you can manage to drag yourself downtown (take the S or 7 to Times Square, then take the 1 downtown to Franklin Street).

I might even get Mike Roberts to join us, since he seems to be a frequenter of that establishment. (Mike, take the 2 uptown to Chambers St, transfer for the 1.) Maybe we can convince him to move his Tuesday Night XTC club to the Brandy Lib. Wouldn't it be fun to try to convince Mike (the author of CruiseControl.Net, when he was at Thoughtworks) to join our little group as the CI/Agile expert.... We can even get CD down so I can steal some dispatch patterns from him.

©2007 Marc Adler - All Rights Reserved

HPC on Wall Street - Hardware Acceleration

Sam Brown of the University of Utah just came out with some slides on his initial forays into hardware acceleration. He contrasts the use of Cell Processors, FPGAs, and Multi-core processors. He was trying to speed up 2-Way Wave Equation Modeling, which is not a financial application, but the lessons that he learned can be applied to those of use who are doing (or exploring) HPC in Wall Street.

The big effort is in unrolling loops and parallelizing your code. Using the POSIX PThread API and multi-core cpus is the easiest thing to do. With 4 threads (2 dual-core 2.2 Ghz Opterons, 4 GB RAM), he got a 11x improvement on one set of benchmarks, and a 2x improvement on the big application.

The FPGA system used the Altera Stratix II FPGA. He got about a 14x improvement on the initial set of benchmarks, but the development effort was a lot harder. It took 12 hours to get the design onto the FPGA chip.

The Cell systems were a Sony Playstation 3 (!!!) and a IBM Cell Blade with 2 Cell Processors with 8 SPEs each (a Cell processor has 8 SPEs, which function as coprocessors.) The Cell results were generally disappointing, with the author wondering if he programmed the Cell correctly. One thing that I have heard about the Cell is that it is very difficult to program. Perhaps companies like PeakStream can help you with this effort.

(For instant gratification, the money shots are slides 28, 29, and 30)

What can hardware acceleration do for you? Faster pricing of derivatives. Faster generation of risk values, especially for exotics. Better overnight stressing.

The problem is in acquiring the skill set to parallelize algorithms, deal with the development environments (especially debugging), learning new variants of C, and getting buy-in from the business.



©2007 Marc Adler - All Rights Reserved

My New Favorite Place

The Brandy Library is in the TriBeCa section of New York City. Plush furniture, wood-paneled walls that contain top-shelf liquor, amazing scotches and brandies, interesting tapas to knosh on.

My scotches for the evening were Ardbeg Uigeadail and Caol Ila (18 years). Jerry was sipping their Hot Apple Cider all night. To top it off, a great "hot chocolate".

(I am not affliliated with the Brandy Library in any way ... just a satisfied customer.)









©2007 Marc Adler - All Rights Reserved

Sharing XSDs between .NET and C++/Win32

Free products for generating C++ code for XSD's :

http://www.codesynthesis.com/products/xsd/
http://freshmeat.net/projects/xsd/

A further list of products can be found at http://www.w3.org/XML/Schema

I am in the market for a FREE XSD->C++ code generation tool that I can use to share schemas between .NET GUIs and C++/Win32 servers. Matt pointed me to these two.

What is the best JaxB/Castor equivalent for the C++/Win32 world? I'll let you know when I have finished my evaluation.

(Morgan Stanley makes it easy with CPS and their SOAP libs... every firm should have one of these)


©2007 Marc Adler - All Rights Reserved

Just returned from London, where 2 inches of snow managed to shut down most of the city. Even the newscasters on Sky News were lamenting the fact that London (with one week of advance warning) cannot seem to deal with white powder.

Got together with some ex-colleagues, who were marvelling at the Lodnon consulting market.
The hot areas are Grid Computing, with the prevelant stack being DataSynapse and Tangasol. Also demand is picking up for WPF, with Morgan Stanley leading the way. The daily rates for qualified individuals are about 1000 pounds per day, which at the current exchange rates, is about $2000.

There is a well-known financial consulting firm who shut down its London offices last year, telling employees that they were losing money and that there was no money to be made in the London market. Hmmmmmm.....

It will be interesting to see if Microsoft's Compute Cluster Server and Digipede can make any inroads intot his market. There seems to be a very strong bias against using .NET for a grid infrastructure, something which I hope to see turned around in 2007.

I got a chance to walk around Canary Wharf a bit more this time. I hope that JL gets to go to the Gaucho Grill (Argentinian Steaks) and Nobu (Japanese) on his upcoming trip. But I need to have TS take JL out for a night of karaoke, and take some pictures to prove it! (Our karaoke outing got cancelled because of the snow .... to the relief of the collective ears of the London Equities Architecture group!)

©2007 Marc Adler - All Rights Reserved

Welcome Back Crispin

Crispin has a new blog ... but you'll have to find it for yourselves!

©2007 Marc Adler - All Rights Reserved

Syncfusion GridControl vs GroupingGrid

I am starting to write the grid abstraction for my client-side framework. Developers will code to a "virtual grid" API, and there will be adapters for the actual implementations of the grids (Syncfusion, Infragistics, WinForms, DevExpress, etc).

Because most of Wall Street uses Syncfusion grids for trading apps, this is the first adapter that I need to write.

Syncfusion has 3 different grids: the regular old GridControl, the DataBoundGrid, and the GroupingGrid. The grouping grid is used a lot in my place to provides rollups of data. You would think that the GroupingGrid would inherit from the GridControl, but unbelievably, it does not. The API sets are not even consistent.

I have spent my time using Lutz Roeder's very excellent Reflector to try to hunt and peck through the Syncfusion class hierarchy in search of the various classes and functions in the GroupingGrid that implement the same functionality as the GridControl. The fact that I am dealing with two incompatible API sets has increased my work by 100%.

C'mon Syncfusion ... you should know better than that!


©2007 Marc Adler - All Rights Reserved

New York Code Camp

DonXml is organizing a Code Camp on Saturday, March 3, in New York City. Last year's camp sold out in 24 hours and they had to max out at 150 attendees. This year, they expanded the number to 400, so there should be plenty of room.

Don is looking for speakers, especially from the financial sector. So, all of you people from Goldman, Morgan, JPMC, Citi, DB, Barcap, Lehman, hedge funds, etc .... get ready to spill the beans! Come tell us all how you are using .NET at your jobs.

Don is also trying to organize a Java/.NET Code Camp in New Jersey, and is looking for some company to donate space. Get in touch with Don if you are a North/NorthCentral-based organization that can donate some conference facility for one Saturday.

©2007 Marc Adler - All Rights Reserved

Microsoft MVP Summit

Although I am not a Microsoft MVP, I have been invited to attend this year's MVP Summit in Seattle as one of the 20 "Influencers" of .NET technology. I hope to see some of the readers of this blog over there and share in some good Washington State microbrews. The summit runs the week of March 12. (Big shout out to Joe Rubino for this event...)

Coming from the capital markets sector, I will be most interested in the following:

Windows Compute Cluster
- how can Microsoft compete with establish grid players like Platform
- how do they cooperate with other grid vendors (ie: Digipede) that we might want to use for some lighter projects
- tie-ins with Excel Services

Analysis Services
- how can we use Analysis Services to perform risk analysis
- how can we use AS in a quasi real-time mode to perform real-time risk analysis and performance attribution

WPF
- what new compelling visualizations for trading systems can WPF give us
- performance aspects of WPF for trading systems

WCF
- would love to see WCF expanded to Tibco RV and EMS. That wouold give it more bang in the Capital Markets area.

SQL Server
- what is coming down the pike for SS that would be most applicable to our systems (time-series, complex event processing, etc). Will they be doing anything to get into Vhayu's and Streambase's space?

Capital Markets Strategy
- I have been very vocal with Microsoft about their (lack of) coherent strategy for Capital Markets. I want to see what plans they have for the entire stack. Let them demonstrate to me that they are think about coming out with best-of-breed components along the entire stack, and that people will take Microsoft seriously for server-side components.


©2007 Marc Adler - All Rights Reserved

Off to London Again

I'll be in London again on the week of February 5, so I will be moderating the comments sporadically during that week. A week of hard work, and a kareoke session on Thursday night. The last time I was there, I found that the Brits could not handle their tequila shots very well (The Aussies and Kiwis were only slightly better). They have promised to practice while I was away .... we'll see if their hard work paid off. Maybe Terry will take a few photos...

©2007 Marc Adler - All Rights Reserved

Dissecting a Wall Street Job Posting

A lot of you have emailed me privately with questions about working on Wall Street, and how to land a job there. (I assume that the questions are coming by email because everyone is afraid of their bosses tracing the IP addresses.)

I would like to take a recent job description that was on the 96Bosses.com site and dissect it.

Here it is:

=======================================================

Job Description
Our client, a top-tier global brokerage firm, is seeking multiple C++/Java Developers to work on an a real-time algo trading platform for their US Equitites trading area. This is a fulltime opportunity located in New York City, NY.

Qualifications
Experience in designing and implementing high speed trading system that can be used for algorithmic automated trading and smart order routing.

Prior experience in US equities trading environment is essential. Prior experience in US options/futures trading environment a plus. Prior experience in UK/Asia markets a plus. Knowledge of building automated trading systems and/or exchange connectivity is a big plus.

• Strong technical programming skills is a key requirement. The Object Oriented Design and Programming is essential. Knowledge of design patterns is also key. Hands on expertise in programming in C++ and Java (server side programming) is necessary. In depth knowledge of multithreading, exceptions, template etc.

• Needs to have strong expertise in inter process communication and event driven programming
Projects will include:
• Algorithmic Trading
• Automated Market Making
• Smart Order Routing
• Trade Analysis for Equities, Options and Futures in US Domestic and International Markets

Experience with the following products:
• TIBCO Messaging Services
• Ticker Plant
• Times Ten
• GemFire
• Transact Tools (FIX Gateway)
• Web servers (Tomcat/Weblogic)

Compensation
$150 -$300k

========================================================

The work is for an Algorithmic Trading platform. This is a good thing. Wall Street is becoming more and more automated, with the NYSE reducing floor space and a number of firms paring their trading staffs.

Algo trading is a very hot area. Firms want to trade large blocks of stocks without affecting the marketplace. The trading has to be broken up into a lot of small blocks, according to different strategies. The firm also has to be able to switch strategies in mid-stream. This requires monitoring the marketplace in real time, and keeping an intraday history of the stocks that you are interested in.

The firm is looking for experience in high-speed, low latency trading. Latency is everything. You want to get yourselves out there faster than all of your competitors. Latency can be affected by many factors, from speed of the networks to inefficient locking algorithms in your code. There is also a need for experience in smart order routing, which means that you might get to work with some kind of rules engine, like ILog Rules.

Let's look at the technology stack.

- Tibco is being used (either Rendevous or EMS), probably to get the spot prices from the ticker plants into the algo trading server. (Watch out for those multicast storms...)

- "Ticker Plant" probably means something like Reuters or Wombat. If the job is for Merrill, Bear Stearns or RBC, then most likely that Wombat is being used,otherwise most likely Reuters.

- Times Ten is an in-memory database (now owned by Oracle) that is probably being used to hold tick history. Other solutions in this area include KDB+, Vhayu, StreamBase, and some of the newer players that I have talked about on this blog in the past. These newer time-series databases can be used to perform complex event processing on the incoming tick stream.

- Gemfire (by Gemstone) is a distributed cache. This is probably being used to cache queries to the reference data. The use of Times Ten may mean that you will have an Oracle database.

- Transact Tools TTConnect will be used to send and receive FIX messages between your Algo server and the various exchanges. FIX is the standard message format that is used to send equity orders to exchanges and to receive updates about the status of your orders. Latency is extremely important here too. How fast can you get your messages to and from the exchanges.

- Web Servers. Perhaps for some web apps that will be used to monitor status. I would expect to see a .NET front-end here. The fact that .NET is not on the stack makes me believe that the job is not for Goldman nor Morgan Stanley, where .NET front-ends are "de riguer".

Now, onto the compensation. 150-300K ... That's a pretty wide range. The 300K might lead me to believe that this position was for a hedge fund, but the first line tells me that it is a top-tier global brokerage firm. If you take the word "Brokerage" literally, then this would eliminate Lehman, Goldman, and Bear Stearns. Base salaries go from about a low of 130K to about 160K, with 150K being the norm. So, if the total comp was 150K, then you are not getting much of a bonus, are you? Let's take 150K as being the base. Lehman had about 60-80% bonuses this year and Morgan Stanley had about 45-55%. Except for Goldman, you are probably not going to get 100% bonus for a Java/C++ developer role from one of the other firms. So, I would take the 300K figure with a HUGE grain of salt, and look more at 210-225K as total comp.

With all of this said, my firm is also looking for top talent. If you have experience in the above areas, and if you want to explore some opportunities with us, please send your resume to me.


©2007 Marc Adler - All Rights Reserved

96Bosses

While searching for some good interviewing questions for some candidates we have coming in (you are rolling your eyes at that one, but I swear that it's true!), I came upon this site, presumably created by the guy who wrote So You Want To Be a Wall Street Programmer .

Will all people who are applying to my group please *not* read this site, as I don't want you to see the questions that I am going to be asking you....

©2007 Marc Adler - All Rights Reserved

Python and FP on Wall Street???

One of the heads of a group that I deal with asked me about the use of Python on Wall Street. This group is strictly Windows.

Anyone reading this blog using IronPython? Any opinions? What are you doing with it?

Looks like Jane Street is using OCaml. Perhaps we should also be keeping our eyes on Microsoft's F#? Credit Suisse is investigating FP in one of their derivatives groups (FP through Excel).

©2007 Marc Adler - All Rights Reserved

How was your bonus this year?

Bonus season is upon us, and for the first time in my life, I have been part of it as a full-time employee.

When you are independent consultant, you view bonus season as some sort of other-worldly event, a day where the only people who are expected to work are the consultants. Now, when you are part of that world, you have to have to act as father, brother, priest, rabbi, drinking buddy, etc to fellow employees at various times throughout the day.

When I went into work that day, everyone was just standing around eachother's cubicles, nobody able to concentrate on anything. One division actually cancelled all meetings for the entire day. And, despite the exhortations to the employees to NOT discuss their compensation, you know that everyone knows what everyone else got.

Now comes the post-bonus season where the shuffling of chairs begins. I have already gotten calls from ex-colleagues inquiring into positions in my company. Let me give you some advice:

1) The grass always seems greener on the other side, but often isn't.

2) Did you deserve your bonus? Be your harshest critic.

3) Calm down for a day or two, and don't panic the second that you leave your manager's office.

4) Do you really want to work for Goldman? At this time of the year, there is a black hole at GS that resumes flow into. Look at quality of life. Do you want a job where your wife has to remind your kids of your name?

5) The Street is very very very small. Do not burn bridges!


So, how was your bonus?

©2007 Marc Adler - All Rights Reserved

Blood on the Streets

The New York Post reports layoffs on the trading floor.

©2007 Marc Adler - All Rights Reserved

New Visualizations for Trading Systems

I am doing some thinking about new kinds of visual interfaces for trading systems. Whenever you walk through the trading floor, you see rows and rows of traders, each one having 6 monitors, each monitor containing a multiple array of grids of cells flashing in different colors. It takes a special kind of mentality to be able to navigate this extremely hypnotic array of lights 8 hours a day, 5 days a week, 52 weeks a year.

I came upon this thesis, written by one Pasha Roberts of MIT. This is probably the best summary of different visualization techniques being used for financial industries. Pasha also formed his own company, LinePlot, to take advantage of his thesis work.

The use of WPF for new kinds of visualizations is upon us. I have heard that several of the big investment banks are dipping their toes cautiously into the waters. The trick will be to come up with compelling visualizations without overpowering the user. Word is also that several IBs are recruiting game developers. In fact, the other day, there was a job posting on CraigsList for a hedge fund seeking to recruit a game developer.

I wonder if any of the CEP companies like StreamBase, Vhayu, Kaskad, etc have considered visualization schemes as output for the data it processes. It would be interesting for one of these engines to do some CEP and eventual transformation into a stream of XAML. I bet that Matt is thinking along these lines, maybe as an offshoot of his patented work at JPMC.


©2007 Marc Adler - All Rights Reserved

New Book on Building Automated Trading Systems

I'm looking forward to this one ....


©2007 Marc Adler - All Rights Reserved

This is Patently Mad

Financial Services firms rushing to the patent office.

Details of a Goldman patent for reducing volatility. And this is the patent that Goldman just got for diddling.

What does this mean for small companies developing financial software? I know a few people developing new trading packages which they hope to sell. Hope you guys are engaging a patent attorney.

Whenever I came out with a new product through Magma Systems, the first thing I would do was to copyright the code. At that time, you needed to send the "first 100 and last 100 pages" of the souce code to the copyright office.

(thanks to Terry for the first link)


©2007 Marc Adler - All Rights Reserved

Coffee

Random thoughts ....

I have always wondered if there is any correlation between the coffee service that a company provides to the IT staff and the perceived quality of the company.

Barclays had the best coffee service. Being a British bank, it paid a lot of attention to the variety of teas it made available. It also had one of those big machines that made cappucino, expresso, or hot chocolate. I was at that machine five times a day.

BusinessEdge and Morgan Stanley have the little cups of Flavia coffee and hot chocolate ... totally free.

Wachovia in Charlotte ... never tried their coffee service ...in Charlotte, there are three Starbucks in every office building ... I never saw so many Starbucks per square meter.

My current company has free Martinson's coffee, but charges 50 or 75 cents for Flavia. After years of drinking Starbucks, Martinson's seems very bitter and flavorless. I guess that I am too used to Starbucks.

Do you bring in your own coffee machine? Watch it... many companies are cracking down on that practice.

How does your company's coffee culture stack up?


©2007 Marc Adler - All Rights Reserved

I Have Been BlogTagged

The newest fad in the blogsphere is "tag, you're it". DonXml tagged me, and now I have to tell you all five things about myself that you did not already know. I also have to tag five other people.

Ok, let me think ....

1) In high school, I played in a Progressive Rock copy band called Heresy. We performed songs by Gentle Giant, King Crimson, ELP, and others. The keyboard player was a rich kid from Baldwin, Long Island, who had every keyboard ever made.

2) I was a huge pro wrestling fan when I was a kid. I used to get ringside passes to the old Sunnyside Gardens in Queens, and take photos of wrestlers like Pedro Morales, Chief Jay Strongbow, Haystacks Calhoun, etc. I was also a writer for a little wrestling newsletter put out by a guy named David Meltzer ... now Meltzer is probably the best known journalist in pro wrestling.

3) In college, I was the winner of the annual concerto competition, and I played the Mayazumi Conceto for Marimba and Orchestra with the SUNY Albany Symphony Orchestra.

4) My father was good friends with Dave Kamansky. Dave was the ex-CEO of Merrill Lynch, and presided over Merrill in the go-go 90's. My father and Dave worked together in the Forest Hills branch of Merrill Lynch in the 70's, and Dave was part of my dad's pinochle group. So, every few weeks, Dave would come over to our apartment for the big pinochle game.

5) I was classmates with the actor, Tim Robbins. A better of acquaintance of mine was Stacey Nelkin, who was the female lead in Halloween III, and was a former girlfriend of Woody Allen.

Now, I need to tag 5 people. They will be:

Francis Shanahan

Chris Donnan

Matt Davey

John O'Sullivan

David Lattimore-Gay


©2007 Marc Adler - All Rights Reserved