The Streambase Amnesty Program

Every since this morning, I have have gotten a number of emails telling me of a new Streambase press release and asking me what I thought of it. It seems that Streambase is trying to take advantage of the Coraleri merger in their "Streambase Amnesty Program".

And Coraleri may be making a mistake by publicizing Streambase's program.

When I first was alerted to Streambase's program by various people, I thought that it was yet one more instance of Streambase's questionable marketing practices.

But then I thought more and more about it, and now, I am not entire sure if I would categorize it as something totally sleazy.

When Don and Terry told me of their plans to have two engines, the first thing that I told them was that it would result in confusion, and that in order to avoid confusion, NEW customers and prospects might shy away from Coraleri and go to another vendor like Streambase or Apama. They might even want to avoid smaller companies totally, and go with something safe like IBM or Oracle. And, I even stated this fact in my earlier blog posting on the Coraleri merger.

I know that Streambase are big readers of my blog, and they certainly picked up on this fact right away. And, they are using this to take advantage of the period of haziness that surrounds any merger.

I question the use of the word "Amnesty". Amnesty usually implies forgiveness for someone who has committed a transgression. What transgression has a Coral8 or Aleri customer committed except not use Streambase? I'm sorry Streambase, but I don't that anyone needs your forgiveness. But it is very gracious of you to offer your beneficence to the Coraleri customers.

Streambase has detected a weakness here in the marketplace, and is positioning itself to exploit it. Is this sleazy or is this simply an aggressive but honest business practice? Competitive upgrades happen all of the time in the business world. Companies are constantly trying to poach customers from each other by offering various kinds of incentives.

Let's examine the Streambase press release.

(Bile On)

The Streambase press release says:

“StreamBase received several enquiries on Monday from concerned customers."

Really? The combined Coraleri only has about 80 customers. There were actually customers who called you on Monday, the day of the merger? Everyone was that disturbed by the merger?

"StreamBase has designed a Best Practices Workshop for those investigating the migration process."

Wow! An entire workshop was designed and completed in the 2 days since the merger was announced. Pretty quick work there!

“Migration Best Practices: Moving from Aleri SPLASH or Coral8 CCL to StreamBase StreamSQL” is a workshop designed by Richard Tibbetts, StreamBase’s Chief Technology Officer and led by StreamBase’s senior field architects."

All of your senior field architects became experts in CCL and Splash in two days? Pretty impressive!

(Bile Off)


I am sure that Streambase has performed competitive analyses and gap analyses on both Aleri and Coral8 over the years. Most vendors know what their competitors are capable of. So, I am sure, at some level, there are people at Streambase who know what CCL and Splash do. I am sure that there are customers who evaluate Streambase, Aleri and Coral8 side by side, just like we did. A good field engineer like Streambase's Steve Barber probably gets questions from prospects all of the time like "I did this in Coral8. How do I do the same in Streambase?" I would question whether Streambase could handle the load of even 10 Coraleri customers asking for conversions at once. One migration would probably take Steve Barber several weeks to do.

As much as I dislike Streambase's sales and marketing organization, and as much as I personally think that they sometimes play fast and easy with the facts, I think that Streambase is probably doing right by their VC's by taking advantage of some of the uncertainty that surrounds a merger of complimentary products. And, I am sure that most majors companies would do the same.

I would like to think that Don and Terry and John and Jeff had completely thought this out, but until I see a complete roadmap from Coraleri, I will continue to think that there will be some degree of risk that arises from the integration. And, I cannot fault Mark Palmer from looking at this situation and thinking to himself that there's money to be made from this. I might do the same....


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Streambase and Quant Fund Corrections in WS&T

Thanks to Penny Crossman and her editor, Greg MacSweeny, for doing some due diligence and subsequent corrections on their article about PhaseCapital's evaluation process in which they chose Streambase. The link is here.

Here is the important paragraph from the revised article:

Last summer, the quant fund began evaluating major complex event processing venders by attending trade shows, checking general market acceptance and investigating research on CEP. "We don't claim we've seen every CEP vendor," Goodell notes. During the fall, the firm drilled down on CEP vendors by meeting with representatives, checking product references (blind and vendor provided), downloading publicly available products, and reading white papers. "Some products appeared to be more flexible than others, and that was a large part of our evaluation process," Goodell notes. "While we wanted to make sure the product would integrate well with our environment and that it was rich enough to allow us the control we needed to implement our algorithms, we also wanted to make sure it was enough of a high-level platform that we did not need to worry about data management issues."

And, the money shot is in the sentence: "We don't claim we've seen every CEP vendor"

This is quite a difference from the assertion of the Streambase press release in which they say:
"PhaseCapital chose StreamBase after a comprehensive evaluation of the leading CEP products. "

Kudos to Penny and Greg for having the integrity to drill down into the original questionable assertions from PhaseCapital and Streambase in their original article.


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Streambase On The Move?

Mark Palmer seems to be turning Streambase in a positive direction since he joined .... either that, or the Streambase marketing machine is in full throttle.

Remarkably, Mark has managed to get testimonials from a number of financial services customers. Usually we are a secretive bunch, and many of the larger financial institutions do not like to have themselves used as a reference customer. Which makes it all the more interesting that Streambase has been able to persuade these customers to go public. Not only that, but Streambase has managed to convince Penny Crossman of Wall Street and Technology to rewrite their press release into an article.

Say what you will about Streambase's sales and marketing staff .... they have proven themselves to be amazingly effective. They have been able to trot out customer references, whereas Coral8 and Aleri have not. This gives the perception that Streambase is eating the competition, whether that perception is true or not.

Mark posted an interesting paragraph:

As Dr. Goodell described, Apama didn't even make the first cut in their evaluation. On the other hand, Aleri, the other final CEP product in PhaseCapital's evaluation, fared well with its high-performance, multi-threaded server architecture.

And, Penny follows it up with:

PhaseCapital evaluated a number of CEP vendors in the trading space, including Progress Apama, Coral8, Aleri and StreamBase.

It's very interesting that Phase Capital found that Aleri was the highest performing CEP engine. This has confirmed something that I have suspected. Aleri has always been touting themselves as the most performant of the CEP engines, and has welcomed the other CEP vendors to take the Aleri Challenge. However, it may have been the lack of spit-and-polish in Aleri's development environment that may have sunk it. I have previously blogged here that I thought that Streambase had the best development experience and documentation out of all of the other products.

I am wondering what factors caused Phase Capital to not choose Coral8. One thing that Streambase has that most of the others do not is built-in adapters for trading companies. As Penny writes:

StreamBase already had an adaptor for the QuickFIX engine as well as adaptors for the data streams provide by Lime Brokerage, PhaseCapital's execution broker. "The building blocks were very complementary to what we're ultimately trying to deploy," Pritchett says.


I have been saying this for a long long time on this blog --- the quality of the CEP engine does not matter as much as the entire ecosystem surrounding the CEP engine. You can have the best, fastest, most expressive CEP engine and language, but if you don't surround it with a compelling ecosystem, then you are sunk. And, Mark Palmer understands this.



©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Brief Thoughts on Standardized Streaming SQL

I haven't blogged for a few weeks because I am been on vacation for two weeks (San Francisco, Vancouver, Whistler, Banff) and in between that, I have been bogged down with all of the paperwork at my job, including doing budget planning for 2009. Let me tell you that, in this current economic environment, budget planning is a very interesting and challenging exercise. (The best strategy for dealing with budget time is to take your favorite managing director on the business side out to The Brandy Library, make sure he gets all liquored-up, and make him sign off on your budget after his 5th Highland Cooler.)

Also, I have been a bit turned off by all of the recent in-fighting that has been occurring on the CEP-related blogs. It seems that one person posts an opinion, then two or three of the well-known pundits start a counter-argument on their own blogs, which leads to a spiral of (sometimes good-natured) venom. Just like the recent Democratic and Republican conventions in the United States, I am hoping that the Gartner CEP Summit in two weeks will be a big love fest, and that the pundits will make peace with each other for a week or two.

I read with great amusement the recent announcement of Oracle and Streambase getting together and attempting to define a new standard for Streaming SQL. Some things that immediately crossed my mind were:

1) Streambase and Oracle are presenting their paper at the VLDB conference. The list of presentations is very impressive. Notice the number of papers that Microsoft is presenting, which leads me to hope that some very interesting stuff will be coming down the pike from MSFT. I wish that I could attend the tutorial on "Detecting Clusters in Moderate-to-High Dimensional Data: Subspace Clustering, Pattern-based Clustering, and Correlation Clustering".

2) Mark Palmer, who railed against Streaming SQL for such a long time in favor of Apama's more procedural language, now has to publically support the effort to standardize Streaming SQL. I wonder how much of Apama's language will appear in the standardized language.

3) There are other efforts at standardization underway. Opher Etzion is involved in one of them. I am not sure if Opher's efforts are aimed at solely defining a meta-language for events, or if what he is working on is in direct competition to the Streambase/oracle effort.

4) How willing will Aleri, Apama, and Coral8 be in adopting this effort? In particular, Aleri has a richer programming environment because of their procedural FlexStream language that gives developers a procedural "out" from the SQL-based language. Apama prides themselves on their Java-like language.

5) What happens if Microsoft ever weighs in with something of their own? You know that Microsoft will tie any effort in this area in with LINQ. Aleri is also moving to a more LINQ-like way of doing things. Of course, one can write a LINQ provider for Streaming SQL, but would anyone be motivated to do so?

(Update: Thanks to a reader who does not wish to be identified... the Streambase/Oracle paper is located here)

----------------------

On another note, good luck to Colin Clark, who has left Streambase as quickly as he joined them. Colin looks like he is striking out on his own as an indie consultant. Colin just had his first child, so this must be an especially interesting time for him. It reminds me a bit of my own story when , in 1988, I quit consulting for Goldman Sachs while my mortgage application was pending in order to strike out on my own. When you get the entrepreneurial fever, nothing can stop you ....




©2008 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Abstracting the CEP Engine

Here are two comments that I received yesterday, and my answers:

1) You bought an CEP Engine that doesn't support event clouds?

We feel that Coral8 does support event clouds, but we are looking for the best pattern to implement it. Mark, who is the CTO of Coral8, doesn't quite agree with the term "event cloud". His posting here highlights his argument. According to Mark, an "event cloud" can be represented as multiple event streams, something that Coral8 supports.


2) How and why did you abstract the CEP engine in your system?


First, the why. We want to insulate ourselves from any uncertainties concern the CEP engine, both in terms of the product itself and of the company. In this economic environment, we are concerned that some of these smallish CEP companies might be strained. Ones who are backed by Venture Capital might find their VC's getting worried and thinking that we are reliving those inglorious times from 2002 to 2003. Ones who are privately financed might find that the backers want to move into other areas. It is no secret that most firms are cutting back or delaying their software purchases, and the ones who get impacted first are the smaller niche companies.

We also want to have some flexibility in case the CEP engine itself does not function as advertised. Coral8 has given us great support, but we have not stressed it yet. We know other companies who have evaluated Coral8 who have foudn some shortcomings, things that the Coral8 staff have addressed. However, it is perfectly within the realm of possibility that we may need to consider another CEP engine should Coral8 fall on its face.

Now, the how ....

We are not using Coral8's native input and output adapters. We are not even reading databases using Coral8's PollFromDatabase and ReadFromDatabase adapters. We have an input server that is used to read static and real-time data and marshall that data into coral8 tuples. On the other side, we have an output server that takes the derived event tuples from Coral8, marshalls them into a common format, and does various kinds of alerting and visualizations.
From the days that we did evaluations of other CEP vendors, we have layers in our input and output servers that deal with Aleri and Streambase. In other words, we have our own adapters! Changing from Coral8 to Streambase or Aleri involves a simple edit to our Spring-like configuration files.

In addition, we have the ability to farm out work to other engines, such as KDB+. We can then read the derived events that are generated by other systems (as long as they are in our common format) and put them into the CEP engine's "event cloud".

In our architecture, we have introduced extra hops in order to abstract the CEP engine. But, we are not that concerned, since we are dealing with analysis and alerting rather than trading.

©2008 Marc Adler - All Rights Reserved

Coral8 is Our Choice or “How the hell did we get here?”

When we went down to Orlando last fall to attend the Gartner Summit on Complex Event Processing, we went with eyes wide open. We were new to the domain of CEP, and one our missions was to try to pick a vendor for the CEP engine that would drive our efforts to produce a major CEP system for our Equities business.

There were a bunch of event-processing systems that were not under consideration because it seemed that they had moved into the strictly vertical area of Algo Trading. These CEP systems included Truviso and Skylar. We needed a general-purpose CEP system, and we wanted to only consider systems that still had a generalist product. An exception to this rule was Aleri, a company who had just come out with a Liquidity Management System as a separate product. We thought that Aleri would still keep its focus on the core CEP engine, so it warranted inclusion of our evaluation.

Apama fell into the Algo trading vertical, but Apama still has a general purpose CEP engine. However, when we tried to evaluate Apama, we were told that we had to go through the dog-and-pony marketing show, something that we did not want to do. I am not sure if this requirement was brought on by the purchase of Apama by Progress Software, a company who I think of as being Computer Associates Lite. I was also told about some interesting experiences between Apama and a major bank by a former colleague of mine whose opinion I trust, and this also influenced by decision to evaluate Apama. This was unfortunate, as I happen to side more with Apama on the whole EPL vs Stream SQL debate.

This left four systems: Streambase, Coral8, Esper, and Aleri.

Coral8 had always been the front-runner, mostly due to recommendations by some former colleagues who were at Merrill Lynch. I had always liked the “vibe” surrounding Coral8, and their openness at giving out eval copies of their software.

Readers of my blog know that I had strong negative opinions about Streambase because of the aggressiveness of their marketing department, an opinion which was shared by a lot of people out there. Nevertheless, their new CEO, Chris Risley, contacted me personally and told me that he had addressed my concerns. After Chris and Richard Tibbetts came down to NYC to meet with me, we decided to include Streambase in the evaluation.

I really wanted to try Esper, but there were a few things that worked against them. The primary factor was that the .NET version, NEsper, was something that was developed by the hard-working Aaron Crackajaxx for his business needs, and did not seem to be part of the mainline Esper product line. We are a .NET shop here, and we needed a product that supported .NET as a first-class citizen. If Aaron decided to become disinterested in Nesper, or if he moved on to another company, then where would we be? We also preferred a product that had an entire ecosystem built around it. So, we passed on Esper. However, Esper is still very much on my radar screen, and I am interested to see how Thomas continues to develop the company and the product.

We spent a good deal of time evaluating Aleri, and most of these experiences were detailed in past entries in this blog. We really wanted to see Aleri succeed, as they were a local company, staffed with a lot of very smart and gentile ex Bell Lab-ers. However, we felt that their product was not ready for us, mostly because of what I called the “spit and polish” issues. I won’t rehash the details now, but if you are interested, please go back and read the old entries in this blog. The areas that needed improvement in the Aleri product were the Aleri Studio, the documentation, and the integration of external data sources.

I met a good deal with Don DeLoach, the CEO of Aleri, and the one positive that will come from my rejection of Aleri is a renewed focus by Aleri on the aesthetics of their product. You can already see these efforts by reading the new Aleri Blog. From what Don had told me a few months ago, their 3.0 product will start to focus on easier integration of external data sources, and will have much improved documentation. I look forward to seeing their efforts come into fruition.

Coral8 was always the front-runner in our evaluation. Their engine is written in C++. They had a decent .NET SDK that let you build out-of-process adapters in C#, and also let you interact with the internals of the Coral8 engine. Their documentation was good, although a bit obtuse at times, and the documentation was backed up by a ton of whitepapers that are available on their website. Their Coral8 Studio gives you a source code view of development, and the GUI part of the Studio is updated after every compile of the source. The CEO of the company was the person who created Crystal Reports, and knows what it takes to build a software company. But, most of all, their CTO and President interacted with us all of the time, and was extremely receptive to our ideas on improving his product. I like when a CTO and the pre-sales engineer send me mail on a Sunday morning!

Streambase was a strong contender. They had some great features in the product that Coral8 has only just come out with (ie: windows that are bucketed by column value). Their GUI is very strong, and their documentation and tutorials are first class. However, I have to say that the interest that Coral8 has shown in our success, and the availability of their CTO was what tipped the odds in Coral8’s favor.

By now, you must be saying to yourself “Where’s the meat?” Didn’t we try to soak and stress the various engines? Didn’t we have an OPRA feed running into the engines in an effort to break them? Didn’t we monitor the use of the CPU and other computing resources? Well … no. To tell you the truth, we were relying on STAC Research to try to do that job for us. STAC is only now just starting to get up to speed in the CEP world, and we will be monitoring their efforts in this space. The general feeling is that most of these CEP engines perform in roughly the same manner, and if one of the CEP engines is 5% faster than Coral8, it is not going to sway our decision, since we are not that concerned right now with super low latency. We are more concerned with the intangibles; responsiveness of the support organization, evolution of the product (and our input into the roadmap), support for .NET as a first-class citizen, stability of the company, etc.

Despite choosing Coral8, we have been careful in our architecture to abstract the specific CEP engine, and no external system will know that Coral8 is driving our CEP system. In the same way that CEP engines have abstracted datasources by using pluggable adapters, we have abstracted the CEP engine. Yes, we have chosen Coral8, and so far, we are satisfied by our choice. But, we are also keeping our eyes open for how the other products evolve in the world of Complex Event Processing.



©2008 Marc Adler - All Rights Reserved

Colin Clark is with Streambase - Where's my Finder's Fee?

A few week ago, this blog mentioned the news that Colin disbanded Kaskad, and was looking for new opportunities. I also proposed that Streambase hire Colin and some of his gang, as they were all in the Boston area together.

Lo and behold, Colin is now with Streambase.

An example of the wonders of the blogging network in action!

Also interesting that Bill Harts is on the Streambase Advisory Board. When I was consulting with Goldman in the late 1980's, I was friendly with Bill, as our groups interacted. I actually interviewed with Bill and Mike Dubnow for a job in Fischer Black's group, but mercifully, they spared me the pain of having to try to understand the world of the Quants (leaving me the opportunity to go full-time into Magma Systems after I left Goldman). Emmanuel Derman has written extensively about the goings-on in Black's little world.

Mike Dubnow eventually became the first Managing Director from the tech side at Goldman, and has since retired. I think that Mike was one of the creators of the SLANG language at Goldman.

On Wall Street, we have an incestuous little circle.

When will they ever learn?

http://apama.typepad.com/my_weblog/2008/01/the-power-of-th.html


Sigh .... When will the guys at Streambase learn that dubious marketing may overshadow what seems to be a really good product .....


©2008 Marc Adler - All Rights Reserved

Kudos to Streambase Support

Thanks to Steve Barber and some of the other members of the Streambase tech support team. I could not get an out-of-process, .NET-based adapter to communicate with the Streambase engine on my laptop, although it worked fine on my office machine.

The Streambase people worked during the slow holiday season to diagnose the problem. It was caused by a DLL called BMNET.DLL that was installed by my Cingular/ATT Communications Manager. I have a Cingular broadband card that I use to connect to the internet when I am on the road and don't have a free wireless connection to tap into. BMNET.DLL provides data acceleration to the Internet.

Microsoft references this problem here: http://support.microsoft.com/kb/910435



©2007 Marc Adler - All Rights Reserved

CEP Vendor Thoughts

Recently, I came across an article on Streambase in Windows in Financial Services magazine. One of the questions to the head of Streambase went like this:

WFS: Does StreamBase have any competitors?

BM: The major players have not yet delivered anything in this space. IBM, for example, does not have a project to build a technology like this. We are IBM’s solution in this space.

In my opinion, this answer totally evades the question. What happened to companies like Aleri, Coral8, Esper, Apama, Skyler, Truviso, Kaskad, etc? How about the IBM offering that Opher is working on? Alll of these companies freely acknowledge Streambase as a worthy competitor, and rightly so. It would be nice to see Streambase acknowledge the same. Brown University certainly was not the only university doing CEP research and not the only one to commercialize their offerings.

And shame on Microsoft and Windows in Financial Services magazine for letting this slip by. Are you a journalistic effort or a fluff rag?

In our evaluation of CEP vendors, we chose not to evaluate Streambase for various reasons. Streambase might have the best technology of all of the CEP vendors (for example, look at Tibbets comment from a few weeks ago on a question about cancelling events), but we will never get to find out. The people who I feel badly for at Streambase are the dedicated development and support staff who have probably come up with a really good product.

(In the interest of fairness, Bill from Streambase told me recently that they had reduced the price of their offering, which was one of our concerns.)

And, if anybody from Streambase reads this blog ---- doing an end-run around me and trying to market directly to the business will not earn you any points. The business people rely on me to make the right decision, and all of your email to the business side (as is any email from information technology vendors to the business side) gets forwarded directly to me. And, I guess that we will end up paying real dollars to your imaginary competitors.

Meanwhile, let's take the attitudes of Coral8 and Aleri. One of these companies JUST hired its first salesperson. Their mantra was that the product should be the best that it can be before it was pushed by a salesforce. The other company has a low-key sales approach too. They have gone beyond the call of duty to incorporate our suggestions into their product and to come up with a POC that really impresses us.

Both vendors have come up with FIX input adapters at our behest. Aleri has incorporated some of our suggestions into their FlexStreams, and has cleaned up some of their visual development studio. (With FlexStreams, you can use a procedural programming language to create custom processing for streams). I am impressed in what these companies have done to earn our business. I feel that, in exchange for these companies doing some of what we want, they get to expand their offerings for the capital markets communities, and bring themselves out of the narrow focus of algorithmic trading and pricing engines.

Kudos to Mark, John, Henry and Gary of Coral8, and to Don, John, Jerry, Jon, David, etc of Aleri. All very nice people, and all trying compete honestly for a piece of the pie.

In my opinion, the Coral8 and Aleri offerings are so close that we will eventually be choosing one vendor as primary and the other as hot backup. What needs to be done is performance evaluation. Pushing multiple streams of fast moving data into the CEP engine and seeing their performance under heavy load. Let's see if they can handle the data rates that come at 2:15 PM on a Fed decision day.

One message that we have been hearing from the CEP and messaging vendors is that they perform better under Linux than Windows Server 2003. This is probably not a surprise to most people on Wall Street. But, I wonder what Windows Server 2008 has to offer in comparison to Linux. The November 8, 2007 article at Enhyper has some interesting things to say about Microsoft's marketing of the London Stock Exchange deal. We will most likely be running our CEP engine on Linux unless Microsoft comes up with a real compelling reason to the contrary.


©2007 Marc Adler - All Rights Reserved

Coral8 vs Streambase

http://www.dbms2.com/2007/08/10/coral8-versus-streambase/

Interesting reply by Bill of Streambase .....

However, I think that the only way that this will be solved publically is by STAC and their independent benchmarks.



©2007 Marc Adler - All Rights Reserved