It Has Begun !!!!!!

I'm the First !

:-)

Watch the news over the next few days and you will see what I mean.


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Job Openings with my Group

I have one full-time opening in my group, and possibly two consulting positions. The full-time job will be located on our Equities trading floor in the TriBeCa section of New York City. The consulting positions will be either in NY or in Jersey City.

For the full-time position, I am looking for a great .NET developer. The ideal candidate should be familiar with Team Foundation Server and issues around build and deployment. The candidate should also be familiar with WPF and WCF; we would ideally welcome someone who is proficient with one of the two and would be willing to become expert in the other.

For the consulting positions, I am looking for great .NET developers. WPF and WCF are highly desirable, as there will be definite UI work.

The consulting positions are on-site. Do not email me with requests to work remotely. They will be routed directly to /dev/null.

Because of the current fiscal environment, I cannot use external headhunters. Sorry.

You can send email to me at magmasystems at yahoo dot com.



©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Magma in Paris, February 12, 13, 14 2009 - A Recap

-->

I arrived in Paris at 11:00 AM on Wednesday, February 11 2009 for 3 concerts given by my favorite group of all time, Magma. The concerts were in celebration of their 40^th anniversary as a group (even though they had their 35^th anniversary concerts in 2005). The concerts were to take place at Le Casino de Paris, a theater in the 9^th Arrondissement, which is a section of Paris more known for its sex shops than anything else.

In addition to Magma, the opening act for all three nights was Jannick Top’s Infernal Machina. Top was the bassist for Magma during their glory years of 1973 and 1974 (and also, 1976), and is a personal hero of mine. His new album, also called Infernal Machina, is a throwback to the glory days of Magma, and to tell you the truth, I wanted to see Jannick just as much as I wanted to see Magma.

After taking the RER B train from the airport to Gare de Nord, and taking the number 2 train to the Pigalle stop, I managed to find my hotel. I used my time-honored technique of sleeping for a few hours immediately after arriving in Europe so that I would have energy for the night’s festivities. I picked a 3-star hotel that I found on Expedia that was only about a ten minute walk from Le Casino. Typical 3-star Paris hotel … clean, sparse (not even a clock in the room), and cable TV.

That night, there was a pre-festivities dinner at a Southwestern French restaurant that was organized by Yann Richard (yannzeuhlwortz). Besides Yann and me, in attendance that night were photographer Fabrice Journo (Riah), Fanny Layani (A-Fiieh) and Muriel Huster (a photographer who is not on the Magma forum). Later, we were joined by Marcello and Tathiane, two Brazilian Magma fans who live in Los Angeles, and whom I met in 2007 in Nancy, France.

Thankfully, between the courses of escargot and Magret de Canard, everyone mostly spoke English to me, and thus spared me the pain of inflicting my self-taught French upon the fair people of Paris.

Yann had a friend who had seen Magma two weeks before in Toulouse. Yann relayed to me a mini-review that his friend had given. The review consisted of the single sentence “Magma is Dead, Long Live Magma.” This review turned out to be very prophetic.

Then I was told that Magma was supposed to play the same exact set every night, and they had no special guests scheduled. Fuck!!!! I thought that, for their 40^th anniversary, they would trot out some of their masterpieces. But, this was going to be 3 nights of new or unrecorded compositions.

Everyone had planned to meet up at the brasserie across the street from Le Casino at 5:00 PM the next day. I spent most of Thursday sleeping, as I went to sleep around 4 in the morning. At 5:00, a small contingent of Magma fans had started to gather at the brasserie. I met my old pal Robert Guillerault from Bordeaux, who runs the Magma Web Pressbook. Robert speaks no English and I cannot understand his spoken French, so we spent a half hour gesturing wildly in the air. We were soon joined by some of the other Zeuhl Kombattahnts that I only knew from the French Magma forums. Fabrice, Yann, Fanny, and Muriel soon arrived. Marino (hpl) came from Venice, and brought me some DVDs of some live shows. Maxence (Zwenskaia) from Montpelier came by, as well as photographer-supreme Madi, Francis Lecointe (who has graciously bought the tickets for me), old friend Fabrice Bellearbe (Barfix), ace photographer Marco Tchamp, Roman (Peaudhorse), and more. Magma themselves came by for their pre-concert drinks. Robert introduced me to Magma bassist Philippe Bussonet, who seemed thoroughly unimpressed that I came from New York to see them. I also got to shake hands with Stella Vander and Herve Aknin. I usually try to avoid contact with bands that I like; I am a firm believer in the old adage “Never meet your heroes because you will inevitably be disappointed.”

At 7:30, we all staggered across the street to the venue. Inside, I met old friend Gaeten Villard (Zess), and I finally met British magma fanatics Steve Ashworth and Steve Davis. Steve Davis has one of the few English websites devoted to Magma. Steve is well-known in England for his exploits in the world of Snookers, which is much like the American game of Pool. In fact, he was 6 or 7-time world snookers champion, and now, devotes a substantial amount of his time playing professional poker. Steve also brought the Magma offshoot group, Offering, to the UK in the mid 1980’s. Steve and I have been corresponding at infrequent intervals over the years, so it was great to put a face to the name. Both Steves had brought their girlfriends to the Magma show, which I thought was a very dicey proposition.

The seats in Le Casino were uncomfortable, to say the least. The amount of space between the rows of seats was designed for little Frenchmen, not for big Americans.

At about 8:15, the first group, Jannick Top’s Infernal Machina, came on. The group consisted of two drummers (one playing a large Japanese Koto drum), bass, guitar, electric piano, and two vocalists. One of the vocalists was Klaus Blasquiz, who was Magma’s vocalist during the glory years. The other vocalist was a tall, blond woman who was the subject of much conversation over the next three days.

After a soft opening section of cello and percussion, Top’s band ripped into the full force of the composition. Most of the hour-long piece consisted of the group jamming out on various old Magma themes, including parts of MDK, Kohntarkosz (the ending section), and De Futura. I loved it.

There were various issues with the sound system on the first night. From where we were sitting in the third row of the mezzanine, the sound was too muddy. Luckily, over the next two nights, the sound improved a great deal.

In a portent of things to come, Magma also had technical issues at the start. They opened the concert with a brand new piece, only to have Bussonet’s amplifier shut off after the first few chords. Vander signaled for the band to stop, and Bussonet took a few minutes to rectify the situation.

The first piece is unnamed as of now. To my ears, it sounds as if Christian Vander has been listening to a lot of Steve Reich. Two main themes play off of each other in this composition. First, there is a Reich-like ostinato pattern that is played by the vibes, guitar, and electric piano. This is overlaid by a more Magma-like vocal line that is sung by Herve Aknin, the new vocalist who recently replaced Antoine Paganotti. As these two themes move in and out of phase, the drums and bass punctuate the music with a series of intermittent crashes. This goes on for about ten minutes, building up to a climax.

The second piece was a real snoozer for me. It is a 25 minute piece named Felicite Thosz, and is reminiscent of the vocal-intensive Magma offshoot called Offering. To my ears, it sounded like a combination of Magma playing Christmas songs with Magma doing a mouthwash commercial. There were lots of vocal solos and harmonies and sleigh bells … and for me, a reason to catch up on my sleep. A bit of promise in the last section with a Russian-dance theme, ala Wurdah Itah, but no real punch. Polite applause from the audience at the end.

The third piece was Emehnteht-Re, which is third piece of the Magma trilogy that includes Kohntarkosz and Theusz Hamtaahk. It is an hour-long piece that unites several of Magma’s previously-recorded themes. I first saw Magma perform this piece in Brooklyn at Club Europa in June of 2007. Since then, Vander has reworked the piece in various minor ways. The piece now seems more coherent to me. The piece starts out with Rinde from the Attahk album, transitioning into Hhai, transitioning into Zombies, and ending with a dirge that represents the funeral of an Egyptian king.

The Zombies section is supposed to be the atom bomb in the piece. About 10 minutes of unadulterated Zeuhl madness, and is supposed to be the centerpiece for Vander’s remarkable drumming. For the first night, Vander seemed rather subdued. I am not sure if it is his age catching up to him or the tendonitis that has been bothering him for the past few years.

E-R was followed by the cacophonous De Futura. This is a Jannick Top composition that appeared on the Magma album, Udu Wudu, in 1976. When it was recorded, it was a sparse but rhythmically driven piece that only included bass, drums, occasional synthesizer, and growling vocals. However, for the 3 concerts, it was played by the total combined forces of Magma and Infernal Machina. 15 people on the stage, including 3 drummers, two guitars, two basses, two pianists, and 5 vocalists. All I can say is “What a mess!” Remember the blond vocalist from Infernal Machina? The running joke during the 3 nights of the show was “What will she be wearing during De Futura?” Each night, she seemed to come out with less and less clothes on, and during the afternoon, she must have been shopping for her next costume at the shops next to the Moulon Rouge. The performance of De Futura was extremely sloppy, as confirmed by a lot of the opinions I heard the next day at lunch.

As an encore, Magma finished with Ballade, which is Vander’s moving tribute to his idol, John Coltrane. A beautiful piece with lots of rich harmonies, and featuring Vander doing his famous scat-singing in Kobaian.

After the concert, a lot of the Magma fans gravitated to another brasserie that was a few blocks up from the concert hall. This was to be a nightly occurrence that kept me out until 2:00 AM. I spent most of the time with the only other native English speakers, Steve Davis and his girlfriend, Jeannie, talking about Magma, snookers, and poker. You can actually watch videos on YouTube of Steve playing snookers at the various stages of his career. Steve was also able to satisfy some of my curiosity about the Women’s Pro Pool circuit in the US.

The next day, we had a wonderful Zeuhl lunch at Le Triton, a jazz club in Paris that Magma has played in many times. The talk around the table was about the collective disappointment of the previous night’s concert. There was consensus about the lack of “punch” in the songs and about the poor sound quality. If you ever find yourself in Paris to see a show at Le Triton, I hardily recommend eating there before the show … the food does not disappoint.

After a brief siesta, it was back to the brasserie for some beers, and onto the show. Friday night’s concert was noticeably better. The word must have gotten to the bands about the poor performance on Thursday, and there were great improvements in the sound and in the quality of playing. It was the same set list, but it seemed that Infernal Machina really jammed out on the De Futura-sounding section, and Magma did the same on the Zombies section of E-R. And, the blond vocalist did not disappoint, as she seemed to outdo her previous night’s antics. Would someone tell Jannick that her presence on stage seriously detracts from his music!

I also met some other fans from the French Magma forum. People like Eric (RI Kobaia), Maahnt (name unknown), Bruno (Nohb), Fatcat (name unknown), JC Alluin, Allain Julliac (Chicago), Bernard (Wurd), and others. I was surprised to see Steve Ashworth’s girlfriend return for the last two nights, as she did not seem happy after the first night. Marcello and Tathiane were so bummed out by Thursday night’s performance that they did not attend on Friday.

The next day, Guy and Bernadette Mas organized a nice lunch across the street from Les Halles. Lots of wine, laughter, broken French and broken English. Fanny and Fab Journo could not attend the show on Saturday, so we said goodbye to them.

At the pre-concert meet-up, I had the chance to meet and talk to Klaus Blasquiz, who is friends with Marcello and Tathianne. Very nice guy and speaks impeccable English. I did not have the nerve to ask Klaus about all of the old Magma tapes that he has stored in his basement. (He was Magma’s archivist during the early years, and I cannot imagine the riches that he has hidden away.)

Saturday night’s show might have been the best. Even the super-group that played De Futura seemed to be in sync. I congratulate Jon Grandcamp, the drummer for Infernal Machina, for having so much restraint and letting Vander do most of the drumming.

After the show on Saturday night, we tried to go to the usual brasserie again, but for some reason, it was closed, as were some of the other brasseries around. New York City still retains the title of the “City that Never Sleeps”. I said my goodbyes to Madi and Maxence, and even said goodbye to Jean-Claude Buire.

Why did I go to Paris for these three days? Christian Vander is 61 years old, and Magma won’t be around forever. I need to be able to see Magma while Vander is still able to play. For me, it’s like being able to see the last days of Michael Jordan or Babe Ruth. But the biggest reason for me to have gone to Paris was to see all of the great friends that I have made through Magma, and those are the memories that I like to keep.

©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

I am in Paris in Magma Heaven

Leaving in an hour to go to Paris for the 40th anniversary series of concerts by Magma at Le Casino de Paris. I will be back on Sunday night.

Since the comments are moderated here, if you post a comment, I will not be able to approve it until Sunday evening. So please don't keep resubmitting the same comment over and over ... the approval pixies are also in Paris.

©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Request: Streaming R or R-like Functions in CEP Systems

I have been looking a bit into R lately. I have written here before about being able to transform R into something that can take real-time streams of data, or integrating R (or SPlus) into a CEP system. Here is a small example. Let's say that you want to take a sample of a "window" of streaming data every few seconds and compute some function on that sample.

Let's see how we can do this in R using static data.

In R, we can define a range of stock prices that range from $10.00 to $100.00, in increments of 25 cents.

> StockPrices<-seq(10.0, 100.0, 0.25)
> StockPrices
[1] 10.00 10.25 10.50 10.75 11.00 11.25 11.50 11.75 12.00 12.25 12.50 12.75 13.00 13.25 13.50 13.75 14.00
......
[358] 99.25 99.50 99.75 100.00


We can take a sample of 10 random stock prices like this:

> sample(StockPrices, 10, rep=F)
[1] 20.50 19.00 78.75 44.75 62.75 45.25 66.25 70.00 71.00 91.00

We can compute the average price of a sample of 10 random prices like this:

> mean(sample(StockPrices, 10, rep=F))
[1] 52.775


Using the Coral8 term of "window", let's say that we have a window called StockPrices that contains one minute's worth of prices. Every 5 seconds, we want to take a random sampling of 10 prices in that window and compute the average price of that random sample.

How can we do it in your CEP system? I would welcome examples in Coral8, Apama, Streambase, Aleri, Esper, StreamSQL, and any other CEP system.


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

IBM MQ Series, XMS, and .NET

We have to be able to read in some information that another system is putting out over IBM MQ. There is a messaging implementation from IBM called XMS that seems like it's JMS over MQ. This is great for us because this means that we can (hopefully) clone our existing Tibco EMS adapters, and with a few little changes, be able to read off of MQ.

I am not sure if the original publisher of the data also has to use XMS ... maybe someone here can tell me. I know for a fact that the publishing system does not use XMS. If the publisher does have to use XMS in order for us to consume through XMS, then we will need to come up with an input adapter that uses the native MQ API.

The information about the IBM Messaging Client for .NET is here.


Update: A great comment by Richard Brown of IBM WebSphere pre-sales tells me that there is nothing to worry about if we are just a consumer ... which we are.


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Follow-ups to Some Comments

Marco says:
>> What do you mean by "Entitlements on output events"?

I have blogged about this here before. When our CEP engine generates alerts, we do not want everyone to see them. There might be very few people who can see alerts on orders. Prop traders should not see many of the the alerts that sales traders see. One trading desk might not be able to receive the alerts that another trading desk should see.

We have the concept of coarse-grained and fine-grained alerts. A coarse-grained alert might be filtered out by role. For example, a particular alert should only be seen by a North American Cash Trader, and not by derivative traders. In the case of fine-grained entitlements, we might have to filter by certain data that is within the alert's payload. For example, someone might not be able to see an alert if the symbol is IBM and the volume is greater that 1000 shares.

There is also the matter of entitlements on real-time aggregations, but that is another long topic.


Charlie Skelton from KX says:
>> "don't make us code in Q" Please can you elaborate?

Charlie, I have written ad nauseum here about Kdb and Q. I actually started to learn Q a few months ago in order to be able to read some of the code that was thrown our way. I have a greater appreciation of Q than I did a few months ago, but I would not want to ask all of my developers to learn Q.

My opinion is that, if you have a facade (input adapter) to a database, then you should be able to code in some database-agnostic way and leave the db-specific language access to the drivers. Look what you can do with O/R frameworks like Hibernate and iBatis.


Hans asked about Unit Testing

When we first started to use Coral8, we actually wrote a C# application that did the exact same thing as our Coral8 application and we did reconciliations between the two. We had found an issue or two with Coral8, which they were able to fix fairly rapidly. We also found some issues with our Coral8 coding, which helped us learn Coral8 better.

Unit testing is still something that is not mature in our Coral8 processing, although we do have unit tests for our internal framework. We have fabricated data in flat files that we can pump into our C8 engine, and we can do regression testing against a known set of outputs. However, we know that we can do a better job.

I would like to hear more about Streambase's unit testing capabilities.


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Streambase Again!!!! Arghhhhh !

It seems that my posting on Streambase's recent customer win has gotten some people pretty ticked off. I have received some private messages from a few of the CEP vendors who are crying foul. Nobody wants to appear that they have sour grapes against the Streambase win, but of course by contacting me, they know that I will pursue the issue in the interest of fairness and transparency.

The issue revolves around the joint claims by Mark Palmer and Penny Crossman that Phase Capital had evaluated all of the major CEP vendors. Penny had written the following:

PhaseCapital evaluated a number of CEP vendors in the trading space, including Progress Apama, Coral8, Aleri and StreamBase.

What some of the CEP vendors are claiming is that Phase Capital had never downloaded copies of their products for evaluation, nor contacted the CEP vendor with any questions.

In my experience, before you can download a copy of a CEP product for evaluation, you need to fill out a registration form. Some of the CEP vendors won't even let you download a copy from their website ... they want to pre-certify you by paying you a personal visit.

So, if Phase Capital did not download the evaluation copies, how did they evaluate some of the products. Did they do part of their evaluation from going over a publically available list of features? Did they get information from blogs such as this one? Did they download the product under another name? Did they have a third-party consultant do the evaluations?

Update: They could have gotten the copies from a reseller. Sybase resells a number of CEP products.

In the interest of transparency, I would ask Greg McSweeny from Wall Street and Technology to ask his writer, Penny Crossman, to substantiate what she wrote in her article concerning the competitive evaluation that Phase Capital did between the various CEP products. Penny is the designated beat reporter for the CEP industry, so I would like to see a bit of investigative journalism from her on this matter. Otherwise, serious doubts would still remain as to Phase Capital's evaluation process.

This is why organizations like Stac Research are so important to our industry.


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

What is Important to us in a CEP System

I have posted this information before in previous blog entries, but I think that, since I have some new readers here, I should post a list of some of the things that are important to me in a CEP system.


Input Adapters
-------------------
Market Data - Reuters, Bloomberg, and to a lesser extent, Wombat and Activ
FIX - 4.2, 4.4. Predefined schemas for the most popular FIX messages would be great.
SWIFT
Message Buses - Tibco EMS, RTI, LBM
Tick Databases - KDB+ (read and write capabilities, don't make us code in Q), Vhayu
Object Caches - Velocity, Gemfire, Coherence


Output Adapters
---------------------
Various chat/messenger mechanisms
Message Buses - Tibco EMS, RTI, LBM
KDB+, Vhayu
Object Caches - Velocity, Gemfire, Coherence

Development Environment
----------------------------------
Strong support for debugging
Profiling and performance monitoring
Hooks into Performance Counters
Ability to easily add C#-based custom functions


Analysis
-----------
Integration with tools like R, SPlus, Matlab
Real-time OLAP
Strong support for real-time dashboards


Miscellaneous
------------------
Workflow
Entitlements on output events
Integration with Active Directory
Priority input queues into the CEP engine


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Streambase On The Move?

Mark Palmer seems to be turning Streambase in a positive direction since he joined .... either that, or the Streambase marketing machine is in full throttle.

Remarkably, Mark has managed to get testimonials from a number of financial services customers. Usually we are a secretive bunch, and many of the larger financial institutions do not like to have themselves used as a reference customer. Which makes it all the more interesting that Streambase has been able to persuade these customers to go public. Not only that, but Streambase has managed to convince Penny Crossman of Wall Street and Technology to rewrite their press release into an article.

Say what you will about Streambase's sales and marketing staff .... they have proven themselves to be amazingly effective. They have been able to trot out customer references, whereas Coral8 and Aleri have not. This gives the perception that Streambase is eating the competition, whether that perception is true or not.

Mark posted an interesting paragraph:

As Dr. Goodell described, Apama didn't even make the first cut in their evaluation. On the other hand, Aleri, the other final CEP product in PhaseCapital's evaluation, fared well with its high-performance, multi-threaded server architecture.

And, Penny follows it up with:

PhaseCapital evaluated a number of CEP vendors in the trading space, including Progress Apama, Coral8, Aleri and StreamBase.

It's very interesting that Phase Capital found that Aleri was the highest performing CEP engine. This has confirmed something that I have suspected. Aleri has always been touting themselves as the most performant of the CEP engines, and has welcomed the other CEP vendors to take the Aleri Challenge. However, it may have been the lack of spit-and-polish in Aleri's development environment that may have sunk it. I have previously blogged here that I thought that Streambase had the best development experience and documentation out of all of the other products.

I am wondering what factors caused Phase Capital to not choose Coral8. One thing that Streambase has that most of the others do not is built-in adapters for trading companies. As Penny writes:

StreamBase already had an adaptor for the QuickFIX engine as well as adaptors for the data streams provide by Lime Brokerage, PhaseCapital's execution broker. "The building blocks were very complementary to what we're ultimately trying to deploy," Pritchett says.


I have been saying this for a long long time on this blog --- the quality of the CEP engine does not matter as much as the entire ecosystem surrounding the CEP engine. You can have the best, fastest, most expressive CEP engine and language, but if you don't surround it with a compelling ecosystem, then you are sunk. And, Mark Palmer understands this.



©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Real-time Messaging in Sybase 15

If your organization is migrating from Sybase ASE 12.5.4 to Sybase 15, then you might want to take a look at these slides.

One slide caught my attention. It was a slide on the migration effort done by eSpeed. One of the things that eSpeed did in their Sybase migration was:

Replace "Polling-Broadcasting”legacy solution with integrated real-time messaging (RTDS 4.0) in ASE 15

I need to find out more about this. Sybase has some slides here.

Many financial companies who are using Sybase have applications that actually do database polling every 'n' minutes in order to find out what rows in a table have changed. If eSpeed did what I think they did, then this presents itself as a compelling reason to move to Sybase 15. (Of course, this assumes that you are not goign to be moving eventually to SQL Server 2008/2010.)

©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Back from Seattle

I just returned home from a two day trip to Microsoft. I cannot say anything specific about this trip because of NDAs and my promises to the Microsoft team. But, there is GOODNESS coming out of Redmond soon, and I am totally pumped !!! And, the people there "get it". And, the people there are listening.

©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

I Have Succumbed to the Facebook Phenomenon

I always said that I would never join a social network where my kids were active. The kids have a word called "teenile", which refers to an adult who tried to act like a teenager. I vowed that nobody woul dever describe me a teenile.

I establish a token account on Facebook a few years ago in order to try to see what my kids were up to. However, since I had to "friend" my kids in order to see what they were doing, and since any sensible kid will never accept a friend request from his folks, then I just let my Facebook account lie dormant. Over the years, I have received friend requests from various tangential business associates, but I have always ignored these.

Over the past few weeks, the pressure just became too much to resist. First, my high school graduating class (and the class directly before and after mine) established Facebook groups, complete with photos. Then, a few old girlfriends managed to track me down and sent me Facebook links to photo albums of me in somewhat embarassing situations. Then, a few of my college buddies. who I still see, got themselves up on Facebook.

So, with a beet-red face, I admit that I am now a Facebooker.


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Some travel plans

I'll be at Microsoft in Redmond on January 25, 26 and 27. If any of the Microsofties who read this blog would like to talk to me about capital markets, please send me an email.

From February 11 to 15th, I will be in Paris to see the 3-nights of concerts by Magma to celebrate their 40th anniversary. I don't even want to think about work and capital markets during those 5 days ... I will be searching for a bar that serves Absynthe, so if you have any recommendations, please let me know.


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Followup on Horizon Graphs

(From Stephen Few)

This paper confirms the usefulness of Horizon Graphs.

Congrats to Panopticon and Stefan Odelik.

©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Reuters Feeds and Market Cap (Grrrr....)

We need to find out the market cap of a company that we are receiving quotes on from Reuters using RFA 6.x. We see a field called "MKT_CAP" coming through on the initial quotes. Looking at the Reuters documentation, this should contain the market capitalization of the company.

However, every MKT_CAP field has the value of zero.

Reuters support has confirmed that the MKT_CAP field will ALWAYS be 0 for every quote.

So, Reuters ... why are you sending this field with every quote if you are never populating it? And, how many other fields that come with each quote are not being populated with valid values? Don't you realize that low-latency means that you should be sending the minimum of data for a quote .... and we expect each and every field to contain something useful?

Brian ... please clean this up if you can.


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Blogs Influence Purchasing

References here to a study by Jupiter Research

I get a lot of private email with questions like this: "I read your blog. I want a buy system X. what do you think of it?" So, I can personally attest that people who want to purchase trading/CEP/algo systems do a lot of research on blogs. I think that it would be irresponsible to make a major purchase without researching every piece of information that is publicly available, and that include what people say on blogs.

I think that a blog, if written responsibly, can positively influence the development and quality of a product. If I was running a software company, I would certainly try to keep any negative publicity off of the blogosphere, and if that means doing a little bit of extra QA, then I would do it ... and that benefits all customers.


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Two interesting posts from Scott

Scott is the main UI and build/deployment guy on the CEP team. He is also the one who has pushed us into the world of WCF and LINQ.

He just came out with two interesting posts that have ramifications for our development team.

We communicate with many of our external systems using Tibco EMS. However, between our CEP GUI and our one of our .NET-based CEP servers, we use WCF. We were experiencing some performance issues when a lot of users started to connect to our servers, and the issues were so bad that I had asked Scott to start to consider abandoning WCF and going to Tibco for communication between the GUI and server. Not even the Microsoft PSS guy that Microsoft sent us was able to help.

In his blog post, Scott documents all of the steps he took in order to solve the performance issues. I advance the notion that some of these problems could have been avoided by documentation and patterns from Microsoft, either from the WCF group or from the Patterns and Practices group. Also, I would like tools like Resharper and Coderush to be able to identify possible performance problems in code like this:

[OperationContract]
MyDataTable GetData(...);

In another interesting post, Scott shows what it would take to use LINQ with KDB, and validates the notion that we can probably replace CodeRush templates with T4. (In this economic environment, if we can get away with a free templating tool that is just as good as CodeRush, then it is our fiduciary duty to use it.) Even though several people on my CEP team have started to learn Q on their own (including myself), it is more natural if we use LINQ to query our KDB database. (This is the approach that we would have liked for Coral8 to take when they wrote a KDB adapter ... why query KDB in Q when you could have queried it in CCL?)




©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

CEP 2009 Predictions

David Luckham has asked the CEP crowd to weigh in with their predictions for the CEP marketplace for 2009. While this is an interesting effort in mental masturbation, I think that you can only look foolish if you set yourself up as a prognosticator in an economic environment like the world is experiencing right now.

Instead, I would point you to certain facts that I have observed in 2008, and you can draw your own trendlines from these data points.

1) There has been a massive consolidation in the bulge-bracket financial firms. I know for a fact that most of the CEP companies had spent a lot of time and resources on POCs at some of the now-deceased firms. If a CEP vendor spent most of their time pursuing these IBs, then you can be assured that they lost some amount of money on pursuing these deals. Most of the acquirers of the struggling IBs (Wells Fargo, Bank of America) are not known for their pursuit of advanced technologies.

2) There has been a consolidation in hedge funds. Some are no longer with us. Most of the others have lost money and will have to start consolidating expenses.

3) The bulge-bracket IBs that are still around are consolidating expenses, and taking a very hard look at discretionary technology spending. In other words, it will be very difficult to convince the business to spend $250,000 on CEP technology because it looks "interesting". Technologies that allow machines to do what humans do now will be looked at favorable. Green technologies and data center consolodation (ie: expense reduction) are also looked at favorably.

4) The Gartner CEP conference was poorly attended by customers.

5) Cloud Computing is now the buzzword of the day. People do not go into a cold faint anymore when you mention that you are involved with CEP. Blog traffic is down in the CEP space. The bloom is off the rose.

6) Venture Capitalists are re-examining their investments. There is pressure on the CEP vendors to generate revenue. Several of these vendors had approached me with very interesting and peculiar financing deals in 2008. Some of the CEP vendors have had reorganizations and layoffs.

7) Several of the horizontal CEP vendors are now hotly developing for the financial vertical. Will these new products be used for alpha generation, risk reduction, or cost savings?

8) I get email frequently from people in the financial industry who have heard the term CEP, but have no idea how to apply it to their business. The interest is there. The ready-made solutions are not.


What does all of this mean in 2009? Although these are not predictions, things that I would not be surprised to see in 2009 are:

1) Consolidation of the CEP marketplace. It would be difficult to merge different CEP product lines. However, a larger company would be able to buy a smaller CEP vendor for a cheap price in order to get rid of some competition and to pick up some very good talent. (Hans and Tim predicted this.)

2) A push towards delivering realtime OLAP and integrated analytical tools. The CEP vendors will have to make it more difficult for IT staff to justify the development of their own CEP technologies. Ready-made stacks of CEP processing need to pushed out there. (This is analagous to Hans Gilde's "patterns".)

3) A greater push towards using LINQ as a streaming query tool.

4) The emergence of a new major player in the CEP marketplace, perhaps with CEP technology as a free add-on to their existing technology stack. This would make stream-based processing more mainstream, but would set a new (very low) price point that other CEP vendors would feel compelled to match.


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Are You All Getting Ready for the New Options Symbology?

http://www.theocc.com/initiatives/symbology/default.jsp

We are getting ready. Are you?

The new symbology (21 bytes per symbol)
Symbol – 6 bytes
Year – 2 bytes
Month – 2 bytes
Day – 2 bytes
Call/Put Indicator – 1 byte
Strike Dollar – 5 bytes
Strike Decimal – 3 bytes


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.