How I helped build the bomb that blew up Wall Street

The subprime mess, this time from a Wall Street programmer's point of view:

http://nymag.com/news/business/55687/

©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Yet Another Wall Street Tax

Yet another attempt to kill off the talent on Wall Street .... and this one isn't limited to banks who accepted TARP.

http://www.nytimes.com/2009/03/29/nyregion/29tax.html

New York's temporary 3-year tax kicks in for people earning over $300,000, with the highest rate imposed on people earning $500,000.

I am watching an entire industry being decimated in slow motion .... much like the melting glaciers due to global warming.


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Some More Thoughts on Sybase RAP and Coraleri

First, some brief tidbits....

Congrats to Terry Cunningham, the former CEO of Coral8. It looks like he is now a Senior VP over at i365, which is a Seagate company. I guess that Terry will be wearing a dual hat, as he is also the Chairman of Aleri. One wonders if Seagate will eventually go into the CEP business :-)

Also, good luck to Mark Tsimelzon. A few people at Coral8 have remarked that Mark is no longer with the combined entity. With Mark's track record, I am sure that he will be involved with something new and interesting.

It looks like Coraleri has chosen Coral8's CCL to be the SQL-based language of choice for developing Coraleri applications. This is the first piece that is falling into place, and for me, it's good news. Now, we have to wait to for the Splash integration to happen, and then the second piece of the puzzle will be solved.

The latest Coral8 5.6.1 has lived up to its claims of major performance enhancements. My CEP team reports that CPU usage is down significantly in our tests on our most complex queries.

With Coral8 making big strides in performance, the case to keep the Aleri CEP engine around is weakening. Don DeLoach is adamant that none of their customers will need to rewrite a single line of code, and that Aleri has even put it in writing. I know that Don believes this with all of his heart, but I am still a bit skeptical. But I am willing to cut them a big piece of slack, as long it does not affect what we are doing with Coral8. We are continuing on as if nothing happened, and I am trusting in Don and the Coraleri team to make the transition completely transparent to us.


Which brings me to my next topic ... Sybase RAP and Coral8.

For this topic, I am putting on the other hat that I wear, which is Chief Architect of Equities. One of the jobs of the Architect is to keep abreast of the current technologies that we use to see if any of them will approach an "end-of-life" situation. Chief Architects needs to make sure that the company does not invest in any technology that might be risky in any way.

Almost every large investment bank has one of every technology. I am not giving away any of our secret sauce when I say that we have a number of applications that use Sybase ASE for the database. So, when Sybase comes knocking on my door, asking me if I am interested in their new CEP product, what do I think?

First, I think it is a bit curious that Sybase did not publicly mention the use of Coral8 as their CEP engine. I can now understand why. I can speculate that Sybase knew of the pending acquisition of Coral8 by Aleri, and wanted to avoid some of the confusion that goes along with a merger like this. If I knew that a company's product was based on another vendor's product, and that other vendor had just gotten acquired by another company, I would need to be absolutely sure that all of the moving parts were going to have a long life, and that we would not be faced with an end-of-life situation. I could imagine more that a few sweaty brows at Sybase when Coral8 said that they were selling to Aleri. I am sure that Sybase did not want this consternation to leak out to their potential customers. So, better not to mention the association between Coral8 and Sybase RAP. If this was the primary reason for the silence, then I can certainly appreciate that.

But, as the Architect, I would also want to know what Sybase's plans are for the future of their Coral8 engine. Sybase bought the source code to Coral8. Does Sybase plan on getting continuous enhancements from Coral8 as the Coral8 engine and language evolves? Does Sybase plan a separate fork of the source code so that, eventually, the Coral8 engine and language and the Sybase RAP engine and language are two different things? Will Sybase RAP eventually incorporate Aleri's Splash language? These are things that are very important to us. We would need to see Sybase's roadmap as well as Coraleri's roadmap.

In a posting on Marco's blog, Era asks some very important questions:

Are there other details on the deal? Is this a one time source code tree fork? Will Sybase get new versions from Aleri? It would be interesting to hear if Sybase they will develop it itself further or if bugfixes, patches and new features will come from Aleri’s developers?

John Morrell from Coraleri responds with the very mysterious:

Sorry, we are not releasing more details on the terms of the deal. Those are business terms between Sybase and Aleri.

That's a bit scary. I can understand the financial terms not being released (although it would be interesting to see if Coraleri gets a piece of every Sybase RAP sale, as that would help with Coraleri's financials). But not giving transparency into Sybase's plans is not good news for Sybase, although it is a positive for Coraleri. I have no transparency into whether Sybase will be upgrading their engine based on the upcoming work of Coraleri, and that does not sit well with me.

Sybase RAP can be used without the CEP portion. However, I do not know what other parts of RAP are dependent on other third-party vendors, and whether these other parts are subject to any risk. And, it looks like it has some overlap with KDB+ and with Vhayu. (I wonder if there is a third-party comparison between the three.)

I will not be recommending Sybase RAP for consideration until some of these underlying questions are answered. The current situation just leaves too many questions unanswered. I would welcome a bit more transparency from Sybase.


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

90% Bonus Tax Article

This is the most lucid assessment of the madness that has engulfed the US Congress and the unintended consequences of the 90% bonus tax:

http://news.hereisthecity.com/news/business_news/8876.cntns



©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Sybase and Coral8

A very curious situation between Sybase RAP and Coral8.....

I ran into Ivy Schmerken the other day. She has been an editor and writer at Wall Street and Technology for over 20 years, and I used to read here articles when I started on Wall Street back in 1986.

She told me that she had just written an article on the new Sybase RAP "CEP" product, and asked me what I thought of it. I said, "I know all about it. They licensed the Coral8 engine, and that's how they are doing the CEP part of things." I thought it was public knowledge on Wall Street that Sybase RAP was using Coral8, but I guess that Sybase didn't think so, as they never disclosed this fact to Ivy in her interview.

Ivy followed up on this fact with the people from Sybase, and then it came out that Sybase was using Coral8 under the hood. Ivy's news story was then picked up by other news sources like Finextra, and all of a sudden, it was out that Sybase was using Coral8.

This prompted an interesting post from John Morrell of Coraleri that was posted on the Coraleri blog.

I am frankly puzzled why Sybase did not disclose the Coral8 partnership up front. Surely they realize that most of Wall Street (at least the people who care about CEP) knew about the partnership. I was told about this by Sybase themselves. Wombat used Coral8 under the hood in their Acumen product, and in fact, it was Wombat's strong endorsement of Coral8 that helped us in our choice of CEP engines.

I would have thought that it would be to Sybase's benefit that they tout the fact that a tried and tested CEP engine was at the heart of their new offering. Can anyone shed some insight on the thought process here?


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Ovum article on Coraleri

http://www.ovum.com/news/euronews.asp?id=7745

(Basically parroting what was said here before)


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

The Streambase Amnesty Program

Every since this morning, I have have gotten a number of emails telling me of a new Streambase press release and asking me what I thought of it. It seems that Streambase is trying to take advantage of the Coraleri merger in their "Streambase Amnesty Program".

And Coraleri may be making a mistake by publicizing Streambase's program.

When I first was alerted to Streambase's program by various people, I thought that it was yet one more instance of Streambase's questionable marketing practices.

But then I thought more and more about it, and now, I am not entire sure if I would categorize it as something totally sleazy.

When Don and Terry told me of their plans to have two engines, the first thing that I told them was that it would result in confusion, and that in order to avoid confusion, NEW customers and prospects might shy away from Coraleri and go to another vendor like Streambase or Apama. They might even want to avoid smaller companies totally, and go with something safe like IBM or Oracle. And, I even stated this fact in my earlier blog posting on the Coraleri merger.

I know that Streambase are big readers of my blog, and they certainly picked up on this fact right away. And, they are using this to take advantage of the period of haziness that surrounds any merger.

I question the use of the word "Amnesty". Amnesty usually implies forgiveness for someone who has committed a transgression. What transgression has a Coral8 or Aleri customer committed except not use Streambase? I'm sorry Streambase, but I don't that anyone needs your forgiveness. But it is very gracious of you to offer your beneficence to the Coraleri customers.

Streambase has detected a weakness here in the marketplace, and is positioning itself to exploit it. Is this sleazy or is this simply an aggressive but honest business practice? Competitive upgrades happen all of the time in the business world. Companies are constantly trying to poach customers from each other by offering various kinds of incentives.

Let's examine the Streambase press release.

(Bile On)

The Streambase press release says:

“StreamBase received several enquiries on Monday from concerned customers."

Really? The combined Coraleri only has about 80 customers. There were actually customers who called you on Monday, the day of the merger? Everyone was that disturbed by the merger?

"StreamBase has designed a Best Practices Workshop for those investigating the migration process."

Wow! An entire workshop was designed and completed in the 2 days since the merger was announced. Pretty quick work there!

“Migration Best Practices: Moving from Aleri SPLASH or Coral8 CCL to StreamBase StreamSQL” is a workshop designed by Richard Tibbetts, StreamBase’s Chief Technology Officer and led by StreamBase’s senior field architects."

All of your senior field architects became experts in CCL and Splash in two days? Pretty impressive!

(Bile Off)


I am sure that Streambase has performed competitive analyses and gap analyses on both Aleri and Coral8 over the years. Most vendors know what their competitors are capable of. So, I am sure, at some level, there are people at Streambase who know what CCL and Splash do. I am sure that there are customers who evaluate Streambase, Aleri and Coral8 side by side, just like we did. A good field engineer like Streambase's Steve Barber probably gets questions from prospects all of the time like "I did this in Coral8. How do I do the same in Streambase?" I would question whether Streambase could handle the load of even 10 Coraleri customers asking for conversions at once. One migration would probably take Steve Barber several weeks to do.

As much as I dislike Streambase's sales and marketing organization, and as much as I personally think that they sometimes play fast and easy with the facts, I think that Streambase is probably doing right by their VC's by taking advantage of some of the uncertainty that surrounds a merger of complimentary products. And, I am sure that most majors companies would do the same.

I would like to think that Don and Terry and John and Jeff had completely thought this out, but until I see a complete roadmap from Coraleri, I will continue to think that there will be some degree of risk that arises from the integration. And, I cannot fault Mark Palmer from looking at this situation and thinking to himself that there's money to be made from this. I might do the same....


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

Streambase and Quant Fund Corrections in WS&T

Thanks to Penny Crossman and her editor, Greg MacSweeny, for doing some due diligence and subsequent corrections on their article about PhaseCapital's evaluation process in which they chose Streambase. The link is here.

Here is the important paragraph from the revised article:

Last summer, the quant fund began evaluating major complex event processing venders by attending trade shows, checking general market acceptance and investigating research on CEP. "We don't claim we've seen every CEP vendor," Goodell notes. During the fall, the firm drilled down on CEP vendors by meeting with representatives, checking product references (blind and vendor provided), downloading publicly available products, and reading white papers. "Some products appeared to be more flexible than others, and that was a large part of our evaluation process," Goodell notes. "While we wanted to make sure the product would integrate well with our environment and that it was rich enough to allow us the control we needed to implement our algorithms, we also wanted to make sure it was enough of a high-level platform that we did not need to worry about data management issues."

And, the money shot is in the sentence: "We don't claim we've seen every CEP vendor"

This is quite a difference from the assertion of the Streambase press release in which they say:
"PhaseCapital chose StreamBase after a comprehensive evaluation of the leading CEP products. "

Kudos to Penny and Greg for having the integrity to drill down into the original questionable assertions from PhaseCapital and Streambase in their original article.


©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.

The Birth of CorAleri

It is now March 9^th, 2009 somewhere in the world. And, the atomic bomb news that I have been mysteriously referring to for the past two weeks is ….

ALERI BUYS CORAL8

Not quaking in your boots? Well, this IS earth-shattering news in the small community of Complex Event Processing. But, you guys were good guessers. No, I am not leaving my company (yet). No, I did not take the CTO position at a Complex Event Processing vendor (none has been offered yet). No, I did not get escorted out off of the trading floor by a gaggle of security guards (yet). No, I didn’t run off to play marimba with Magma.

However, it confirms the predictions of many people in the blogosphere that the consolidation amongst CEP vendors is starting to happen. And, I am sure that this consolidation is not over, both in the CEP space and in the infrastructure space. Already, I am hearing of financial troubles among several of the infrastructure players, and it would be really wise of everyone out there to re-engage your vendors in a little bit of financial due diligence. Make sure that their source code is in escrow …. not that you would ever do anything with it.

I had been predicting that Coral8 would be acquired for quite some time. Terry Cunningham had been personally bank-rolling the company since its inception, and unless Terry went short on the market, I am sure that his portfolio has suffered like the rest of ours. I can only speculate whether financial considerations inspired Terry to sell the company, or whether Terry just got the itch to move on to his next adventure (although he did take the role of Chairman of the new company). To be honest, when I heard that a large database company had purchased a source code license for Coral8, I thought that the end was near for our intrepid band of CEP’ers from Mountain View.

My curiosity was aroused when I received a dinner invitation from Terry about two weeks ago. Since I have been burning the midnight oil due to my new role and responsibilities, I politely declined, saying that I was just two exhausted to go out at night and play. However, Terry knew how to appeal to the closet “foodie” in me, and picked a very nice restaurant at Columbus Circle that I have only been able to dream about. Imagine my surprise when I got to the restaurant, and found Terry locked arm-in-arm with my other favorite CEO, Don DeLoach of Aleri. I thought that they had united to jointly serve me with the lawsuit that I deserved for publicly taking their products to task on my blog.

(I am taking a bit of literary license here … I actually found out about the merger the night before from another source, so I was not too surprised to see Don there.)

Why did they single me out for such attention? I guess that having a big mouth and a title pays off sometimes. But as a customer of the newly combined company, they wanted to know how a typical customer would react, and what effect the merger might have on us. Would we continue to buy licenses, or would we go running to a competitor? Or, did they want me to give them love and kisses on my blog?

So, in between lot of glasses of wine and a lot of arm-waving on my part, I expressed a number of concerns. But, also, there is lots of goodness in this merger.

There have always been some issues with Coral8. First and foremost, they had no financial domain expertise. They had attempted to rectify that recently by hiring Colin Clark, but I had my doubts whether that strategy would result in anything concrete. Second, they had no true presence in New York, and if you are going to market to the financial services industry, you have to at least have an office in New York City. Third, they had no European penetration, and I could imagine that Apama, Streambase and Aleri were cleaning Coral8’s clock in EMEA. Fourth, they were one of the only CEP vendors who did not have a vertical product for trading or risk, and they really had no intentions of writing one.

Here was this nice 40-person company, sitting in their nice, sunny offices in Mountain View, 3000 miles away from their main customer base, without a clue of what the capital markets companies really needed. However, Coral8 is an extremely nice company, very friendly to developers, writing lots of nice white papers, and putting out a good product that was less expensive than all of the other products (except Esper).

Then, there was Aleri. Lots of financial domain expertise, with a few products that were geared towards trading firms. Good European presence. Ex-Bell Labs rocket scientists working in a basement office in Mountainside, right across the street from their old Bell Labs haunts. Not a very developer-friendly company. Very hard to configure the product. Documentation that was not as good as Coral8's (although my last experience with their docs was the old 2.4 version). A better IDE than Coral8, but that’s not saying much as Coral8 has a minimalist IDE. Not very good at testing and QA (based on 2.4 experiences, and some recent feedback from some customers in London). Some sort of real-time OLAP strategy. More adapters geared towards financial services companies.

Now, Aleri has bought all of the assets of Coral8. What does the combined company have to offer us? It has a European presence, it has financial expertise, it has a good documentation writer, it has a person who knows .NET, it has people who are concerned about compelling visualizations, it has a procedural language called Splash, and it has two engines and two SQL-ish languages. And it has two pricing models, with Coral8 coming in at $15K per core and Aleri costing about $27K per core.

What did you say? Did you say two engines and two languages? What’s a customer to think?

First, let’s tackle the issues of the engineers. You have two very smart teams located on different sides of the USA. Each team is justifiably proud of the CEP engines that they wrote. One engine came out of a Stanford University research group, and the other engine came out of a commercial effort from Bell Labs. (Correction: the engine is not the same one that they developed at Bell Labs.) Eventually, the combined CorAleri is going to have to choose one language and one engine. So, you will have two sets of development teams positioning themselves as the winner of the language and engine competition.

Every customer of Coral8 MUST program the engine in CCL, which is Coral8’s SQL-like language. However, even though Aleri supports an SQL language, almost nobody uses it, as the Aleri IDE does not support it. (The Aleri IDE likes to generate XML-based code.) Let’s assume that Coral8 wins the language and engine competition. However, in my vision, Aleri gets their licks in when CCL is supplemented with the Splash language.

So, in my ideal world, the language of CorAleri would be CCL with some Splash. And, since the language and the engine are linked tightly, Coral8 keeps the engine as well. However, Aleri would help the Coral8 engine team to make their engine even faster.

At first, CorAleri will have to support two engines and two languages. But, the longer you postpone the single-engine strategy, the more time you give the Coral8 and Aleri engineers to improve their individual engines in order to win the engine competition. Then, it becomes more difficult to have a single-engine strategy.

Don and Terry talked about a single product that includes BOTH engines. The Aleri engine would be the “performance engine” and the Coral8 engine would be the “deterministic engine”. If you check a certain checkbox, “stuff” would be sent to the Coral8 engine and if you check another checkbox, “stuff” would be sent to the Aleri engine.

You cannot have two engines running simultaneously on a single server. They would be competing for resources. We have to worry about administering another process on our servers. We wouldn’t be sure which engine was handling what. It makes monitoring more difficult.

If you give a customer too many choices, then the customer might get confused. Customers want transparency and simplicity. If you confuse a customer, then that customer might change over to a CEP vendor who offers a simpler solution. This means that a confused customer might run over to Apama or Streambase.

CorAleri must come up with a solid roadmap about the merging of the languages and engines, and they must make this roadmap available to customers.

We do not want to take 25% of each sprint to play catch-up on a moving target. The worst thing for us is to be surprised. With each release of the new CorAleri engine and language, we want the migration effort to be painless. We want to change as little of our existing CCL scripts as possible, and we don’t want to find that queries that previously worked do not work anymore.

To me, this is the greatest challenge to CorAleri and the greatest risk to us.

I would like to see the engine and language and the documentation stay with the Mountain View folks. What would the Aleri folks do? They could concentrate on writing more adapters, work on integration with other partners (ie: Netezza, Gemfire, RTI, etc), come up with a better IDE with debugging a monitoring support, come up with more Jack Rusher-type visualizations, real-time OLAP and analytics, and develop some more financial apps. This would give a clear line of separation of duties between the two development teams.

In summation, there is a lot to be excited about, but there is also risk that is associated with confusion.

I wish CorAleri good luck. They have some really good people. They have a sense of integrity. They have a new sense of purpose. And, if they could come up with a unified roadmap that makes sense, then it would be a company to be reckoned with in the CEP marketplace.

A Response from Penny Crossman

Hi Marc,

I just came across this blog today — I guess I should check your blog more often! Thank you for reading my article, here's the self-serving link to it: Quant Fund Uses CEP for Smart Order Routing, Data Feeds

I've placed calls to set up a follow-up interview with Phase Capital and StreamBase to clarify this, but my article was based on an interview with Phase Capital's CEO Eric Pritchett and director of technology Geoff Goodell.

Goodell said, "We did a comprehensive evaluation of different OMS/EMS solution providers in the CEP space. Our goal was first to discern if a CEP solution was appropriate for our firm, then also determine which of these CEP solution providers were most well suited to our development paradigm and the kinds of analysis we’d need to be able to do to execute our strategies in the market. We looked at a number of providers, including Progress Apama, Coral8, Aleri and StreamBase, and we drilled down on these and tried to figure out what aspects of these different solutions were interesting to us. We’re a very heavily technology focused firm, not just for performance reasons but also because the algorithm, the process is our strategy, this is a fully automated process at its core, the code we developed to build our process is the product itself, and as such we need tools that enable us to build our product that will allow us to focus on the business problems without having to worry so much about the implementation details of the internals of databases. At the same time, we’re concerned to make sure we have tools we need.

Some of these CEP solutions seem like completely packaged solutions that strive to deliver complete trading algos against the market that will achieve low market impact etc., we’re not looking for that sort of thing because we are in the business of designing algorithms that get the best execution. So those are right out of the box. Streambase’s support paradigm we considered appropriate because Streambase is suited to working with developers, people who are actually able to build systems and algos that will achieve the kinds of detailed and specific execution properties we want. They have the infrastructure in place to work with us to make sure that what we’re trying to achieve is feasible given the internals of their database. This is one of the important characteristics that made Streambase win for us. The extractions were clean, it didn’t seem like they built their system around satisfying the needs of a small number of clients, the Streambase system is a platform for developing solutions, it’s not a solution itself. The idea of using a platform was extremely appropriate for our needs, because our goal is not to buy a packaged solution, our goal is to develop something using well-designed, modular tools and Streambase’s well designed, modular tools were appropriate."

Eric Pritchett said, "The process we went through was we selected a universe of potentials, we collected information about them the best we could with phone calls, researching websites, we went to conferences in New York, we did a little narrowing at that stage, we had detailed demos and interaction with the teams from the companies Jeff mentioned, we then narrowed that down to our final two: Streambase and Aleri. We had extensive, on-site working sessions with both of those teams and we did a thorough evaluation of the support paradigm and got under the covers with developers from both companies, that took about two months."

I have no reason to believe Eric or Geoff were lying, nor do I have a way of checking whether they downloaded other CEP vendors' software. They didn't actually claim that they downloaded every CEP solution in their evaluation. The CEP vendors who are attacking this story would be better served by getting Wall Street customer wins and getting those customers to speak on the record so that I can write case studies about them.

©2009 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.