Eclipse

I had to do an in-depth examination of a Java AppServer that our group has written in order to do some prelimiary integration of my .NET client-side framework. I finally have had a chance to do some in-depth work with Eclipse, and there is definitely a lot of goodness in it.

I have to admit that there are a lot of positives about being in the Java world. I like the Java Community Process, and the fact that major functionality is discussed in the open before the spec is drafted. There is a lot that Microsoft can learn from this.

©2007 Marc Adler - All Rights Reserved

QuickFix and Enums

It seems that the latest version of QuickFix has a .NET assembly and FIX 4.4 support, so I am going to be using it in an OMS demo that I am putting together to show off my .NET client-side framework.

One thing that I wish that the QuickFix guys did was to use enums instead of static const chars. For example, here is the definition of the OrdType class (this class is written in Managed C++, but a lot of the other .NET classes are in C#).



public __gc class OrdType : public CharField
{
public:
static const int FIELD = 40;
static const __wchar_t MARKET = '1';
static const __wchar_t LIMIT = '2';
static const __wchar_t STOP = '3';
static const __wchar_t STOP_LIMIT = '4';
.............

OrdType() : CharField(40) {}
OrdType(__wchar_t data) : CharField(40, data) {}
};


I completely understand why they used const chars, but for data-binding purposes, it is easier for me to use enums.

I wonder if I should reflect over a few classes and translate these classes to corresponding enumerated types.

I'm supposed to be a manager .. I have to find other people to do this kind of stuff for me. However, in my group, I am the only .NET person in a vast sea of Java-ness.


©2007 Marc Adler - All Rights Reserved

ITSDoc.org Redux

ITSdoc.org is still very much alive. It recently passed its millionth page view and it gets hits from all over the world every day. It strives to be helpful to coders (hopefully open source) of Investment and Trading Systems by assembling into one place a great set of documentation.

Personally, I have been drawn away from driving ITSdoc forward for about a year now volunteering actively at FIX on a standard that will cover the next generation Algorithmic order types. I’ve got to say, volunteering at FIX and mixing it up with the fantastic people in the various technical Work Groups and Global Technical Committee has been wonderful! They have a bunch of brilliant people who all selflessly volunteer tons of time to the betterment of the industry through a FREE standard. Plus, it’s been a real great eye opener on exactly how the largest industry players cross connect. Mind you these same people go back to their firms and compete vigorously.

Last summer our Algorithmic Order Standard Work Group conduced in-depth interviews with five MAJOR order management system (OMS) vendors as well as considerable secondary research into the adoption trends in algorithmic orders types. Much of that learning was incorporated into the proposed standard, which is in the final stages of official adoption at FIX. In just a few weeks its expected to enter the pilot testing phase. See http://www.fixprotocol.org/working_groups/algowg/documents

FIX is a bit more complex than the average open source coder of Investment and Trading Systems might like to struggle with, starting with the printed protocol specification alone. There are many tags and enumerated values, the various messages can be complex and there is a substantial learning curve. There is one outstanding open source FIX engine out there QuickFix, available in C++ and Java, but due to the complexity of the FIX standard it’s quite complex to implement. In the future ITSdoc hopes to help lower the bar to adopting FIX, and help get around using all those nasty, expensive, dedicated IP circuits.

Dealing with emerging ISO standards this past year has also been challenging. If you are designing an open source Investment and Trading System your internal data models need to comply with various standards (including various versions of standards, service packs, extensions, customizations, variations in usage, etc). Needless to say all these different “flavors” and “dialects” of the various standards often don’t mesh well with each other. However, the situation is slowly improving. XML is now everywhere and XSLT can ease (but not eliminate) the pain of making translations.

The ITSdoc UML project is on hold for the moment. I went to write up a meta Use-Case on processing text news during investment decision-making and got mired down with various sub topics such as bias in news coverage. My background includes some work in cognitive psychology and I was perhaps too far down in a “deep dive” on the subject. When I caught myself with a new book on psycholinguistics I thought to myself, you’re in pretty deep, perhaps its time for some decompression. Still the material is fascinating and to a degree exceedingly relevant to investment decision-making. Getting a handle on it however has proven very time consuming. I have a ton of great material but getting it all into publishable format has been constrained by the “hours in the day” limit.

So, plenty going on behind the scenes at ITSdoc. Coders of open source Investment and Trading Systems are very much encouraged to register at the site. We shoot for quality over quantity. Plus, we can often hook people up with other “birds of a feather” who are interested in similar things.

Rick

Http://ITSdoc.org

©2007 Marc Adler - All Rights Reserved

FIX serializer for .NET

Anyone know of a FIX (and FAST and FiXML) serializer for .NET ... something that will reflect over an arbitrary object and produce a FIX string? I imagine that you will be able to tag an object with custom attributes that let you specify the FIX tags.

©2007 Marc Adler - All Rights Reserved

News from the Front

Wall Street Tech writes on the hardware arms race going on now in Wall Street. According to the article, names evaluating hardware acceleration include Merrill Lynch (IBM Cell processor), Citigroup, and Mellon Financial.

Oracle just bought out Tangasol. I predict a Microsoft/Gemstone deal in order to make the Windows CCS stack a bit more attractive. You heard it here first. Gemstone just came out with a native .NET implementation of Gemfire, so it probably makes much sense for Microsoft to gobble them up before Oracle uses the .NET version of Tangasol as a lever to sell more Oracle DBs into Wall Street.


©2007 Marc Adler - All Rights Reserved

Windows Workflow and Auto-Execution?

In preparation for Joe's lecture next week on Windows Workflow Foundation (WF), I am reading a book by Kenn Scribner that just came out on Microsoft Press.

WF has a rule-based workflow engine. I am wondering how application this would be for auto-execution engines.

Also, the middle-office guys have asked me if WF can be used to replace Fuego. I don't know anything about Fuego, but hopefully, Joe can answer this.

I am most excited to see how I can integrate WF with the .NET client framework that my team is writing. I know that Merrill Lynch is doing a lot of things with WF, so it looks like there are some good use cases in the capital markets sector. OMS's? EMS's?


©2007 Marc Adler - All Rights Reserved

My Son's Band at The Starland Ballroom

Cartesia, my son's band, played at the Starland Ballroom on March 17, 2007. Starland is the local mecca of hard rock, punk, and metal in Northern New Jersey. My son is the skinny thing playing drums and singing.

©2007 Marc Adler - All Rights Reserved

New York XTC Club

Mike Roberts hosts the monthly meeting of the New York XTC club, held on Tuesdays at some place that serves an awful lot of Belgian beers. This is a good excuse to make the trek into NYC on a Tuesday.

By the way, XTC stands for "eXTreme Chugging" ... but seeing what kind of state Mike was in, it probably stands for "eXTreme Chundering".

Congrats to Mike for leaving Finetix and taking a new position. A long time coming...


©2007 Marc Adler - All Rights Reserved

I Got a Tech Writer !!!

Read it and weep, people ....

I got myself a real live technical writer for a few months. She has even done Visual Studio and MSDN documentation for Microsoft.

In all of the major projects that I have done for IBs, I have never seen the budget for a tech writer. We usually have to do it ourselves, after we are totally exhausted from delivering the technical aspect of a project. We might be lucky to get a BA to do some of the functional docs, but they will never dive into code.

This young lady who I now own actually installed Visual Studio on her laptop, and started stepping through the code!

©2007 Marc Adler - All Rights Reserved

Grids and Scavenging

During our meetings with the Windows Compute Cluster Server team last week, it came out that CCS does not do any sort of scavenging, nor or their immediate plans to implement it.

Scavenging is an important way to lower the costs of the grid and to gain wider acceptance by letting groups make some money by charging back the use of their spare CPU cycles to the organization. Imagine all of those hundreds of idle CPUs gathering dust in a large Disaster Recovery center at an investment bank. I think back to Morgan Stanley's DR center, with hundreds of cubicles of untouched CPUs.

Would the absence of a scavenging feature eliminate a grid for you?

Does DataSynapse or Digipede have scavenging?

©2007 Marc Adler - All Rights Reserved

New Database TPC-E test Stock Market Scenarios

http://news.com.com/Database+servers+get+a+new+yardstick/2100-1010_3-6168201.html?tag=nefd.top

(Thanks to Jeremy for the link)

©2007 Marc Adler - All Rights Reserved

Microsoft - You're Almost There

I went out to the Microsoft MVP Summit as one of the 20 or so specially invited "Industry Influencers"... which means two things. First, we are supposed to be able to influence Microsoft's future directions. Second, we are supposed to be shown the love by Microsoft so that we can buy lots more products.

What I came away with was that

1) Microsoft is stil very much focused on the desktop and the world of two-tier apps.

2) There are a lot of parts that can be assembled into a killer stack for cap markets, but that there is no overall strategy to congeal the parts.

3) Microsoft needs to look beyond its borders to what is happening in the industry, and to integrate better with third parties.

I get the feeling that we are on the cusp of an incredible stack from Microsoft, but the parts needs to be put together in a cohensive manner. Mike Walker is charged with providing reference architectures for the financial vertical, but Mike needs some assistence from us in understanding the trading stack.

©2007 Marc Adler - All Rights Reserved

The canned presentations of LINQ/PLINQ at the MVP Summit were very interesting, but more questions were raised than were answered. It also was apparent to us Capital Markets people that Microsoft still does not treat our environment as a first class citizen.

Here are some of the questions I had:

No support for async operations (Dinesh from the LINQ group told us that teh query operations are blocking)

No Support for notifications. In many scenarios, if records on the back-end database change, our apps need to be notified asynchrounously. This is what Service Broker is designed for. I would like to see first-class language or API support for receiving async db notifications to a callback function or delegate, or maybe have something integrated with ObservableSet.

Support through WCF or other mechanism for 3-tier apps using transports like Tibco RV/EMS and MQ. LINQ really seems designed for 2-tier apps.

Integration of PLINQ with efforts like Compute Cluster Server.

Support for security and integration within C#

Comparisons with (N)Hiberate and IBattis. These are used extensively in CapMarkets.

Integration with object caches like Gemfire and Gigaspaces.

Possible extensions for time-series (ie: like Streambase)


©2007 Marc Adler - All Rights Reserved

A Comment from Gemstone

Ocassionally, someone will make an important comment to a post I made several months ago. Such is the case now ... Jags from Gemstone replies:

You have hit the nail on the head. At GemStone, we recognized early on that a distributed cache that supports notifications has no choice but to ensure that object change notifications can reliably reach consumers. Else, it pretty much becomes useless.

There are many considerations we took into account:-

Applications are sharing data through a p2p cache and the cache supports dynamic membership - i.e members can come and go at any time. Great from a scalability perspective, but what happens when you have apps connected to the peer cache expecting notifications and a peer node generating the events suddenly departs? In GemFire, we introduced a declarative means to ensure that messages are received by all critical members - each producer or consumer can play one or more roles; A publisher only succeeds if the roles that are required are available. If this interest you, checkout section 3.2 here

http://developer.gemstone.com/display/gfedev/Topologies+and+Architectures+for+Middle-Tier+Caching-

In the scenario you mention - clients connected to a logical cache - we also support asynchronous reliable notifications through in-memory replication of the queues on the cache servers. The challenge here is to ensure in order delivery of events, detecting the failure conditions, automatic client failover to the server managing the secondary queue and avoiding duplicate notifications. And, given that there might be a local cache on the client, the server side can further optimize transmission to the client by merely propagating a object "delta".

One important reason why financial institutions like the whole "caching + messaging" story is because now you enable applications to connect to a distributed object repository that gives pub-sub semantics, but all the contextual data required by the application to make instantenous decisions is available in memory.


©2007 Marc Adler - All Rights Reserved

Off to the MVP Summit

See some of you at the Microsoft MVP Summit starting on Monday evening?

What's important to me is to see Microsoft's directions on Capital Markets. I would liek to see a cohensive story on:

- Grid computing (Windows CCS and possible synergies with Digipede)

- Quasi-realtime Analysis Services, with associated WPF visualizations

- Support for parallel constructs in C#/NET (Microsoft Research Accelerator for GPUs)

- Any forward movement for Complex Event Processing


©2007 Marc Adler - All Rights Reserved

StreamBile

We love bile here! Here is a little bile from Mark Palmer on Apama:

http://apama.typepad.com/my_weblog/2007/03/streambase_algo.html

while giving some kudos to Kaskad. According to this page, Mark Palmer is the VP of Complex Event Processing at Progress Software, a competitor to Streambase.

Judging by some recent comments on Matt's blog, it seems like the CEP vendors relish in sniping at eachother. More bile! We love it!

Meanwhile, we hear from Terry that Gemstone has been impressive with their recent release of a native .NET client. Coming right on the heels of similar announcements from Gigaspaces and Tangasol.

©2007 Marc Adler - All Rights Reserved

Bonus Satisfaction Results

http://news.hereisthecity.com/news/business_news/6552.cntns

Results of a survey about the 2007 bonuses

©2007 Marc Adler - All Rights Reserved

Thanks to James

.. for pointing out something something simple and forcing me to spend a half hour simplifying my code ....

Sometimes people justg get tunneled into thinking about a piece of code in a certain way .. that's why we have code reviews...

©2007 Marc Adler - All Rights Reserved

Antelope Natural Language Processor for .Net

http://www.proxem.com/

Free NaturalLanguage Processor for .Net 2.0.

Worthwhile if you are going to construct a semantic analysis engine from scratch.

I have been reading up on NLP. Take the sentence "Time flies like an arrow.". How many different semantic meanings does this statement have? Now imagine getting a news feed of financial information. How would you parse the news into an ontology that equity traders can work with? And how would traders use this information?

Matt is exploring the CEP engines right now (Streambase, Coral8, etc). I would be interested to see if and how these CEP engines can handle news feed. What kind of info would they need?

©2007 Marc Adler - All Rights Reserved