Monday, December 19, 2005

General Contractors

My Finetix colleague, Chris, just reminded me of the lunchtime conversation we had a few days ago with one of our good friends from RiskFocus. It got me thinking about an interesting model for development.

Think of what a General Contractor does when you do construction on a home. He gets (hopefully) the best-of-breed specialists to perform certain functions. He has one company to do the foundation, another to do the framing, another to do the electrical wiring, and another to do the sheet rock and spackling. Each of these tasks is done efficiently by the tradesman, and then it is understood that they move on to another construction job.

I am wondering if anyone has followed this model in developing systems. A "General Contractor" gets the best consultants around to perform a specific specialist task, something that they do best. The job lasts for one month, and then the contractor moves to a different development job. There would be a specialist to write the framework, a specialist to plug in the security and entitlements, a specialist to write the Web Services and fix any interoperability issues, a specialist to write the calculation engines, and then a GUI specialist to plug in a front end. Each member of this well-oiled team would be versed in the various interfaces and abstractions that the other members of the team use, so that the security specialist would come in, and know immediately where to hook in his stuff to various parts of the framework.

The members of the team would rotate amongst the various specialties so they do not end up getting tagged as a master in only a single discipline.

I have never seen this model being used, but I would be curious if anyone has ever tried it.

Your comments are welcome.


©2005 Marc Adler - All Rights Reserved

Sunday, November 27, 2005

The Wall Street Stack

It has been done plenty of times on Wall Street..... what I call the Wall Street Stack.

I just came off of a large project where I architected (yet another) one. I am current involved in one now. So are plenty of other people that I know in other companies.

Everyone's trading applications seems to be the same. Here are the standard elements:

1) The UI layer. The UI layer is usually designed around the Model-View-Controller pattern. It generally uses some third-party UI suite, such as Infragistics, Component One, or Syncfusion.

2) The UI usually contains an Outlook-style navigation bar on the left side. On the right is usually an MDI-ish area. There are multiple tabbed controls, each containing a grid control. The grid is a respresentation of the underlying business objects.

3) Filter criteria that is used to filter the results on the grid. The filter UI is usually placed on the same panel as the grid and consists of a number of comboboxes and edit fields that represent dates and static criteria. Filters can usually be translated into an SQL statement.... a dataview-like pattern can be applied to the underlying model to produce a filtered view.

4) User preferences and workspace persistence. Users can save the layout of the workspace, and load the layout when they log onto a new session from any machine.

5) Entitlements by role. Entitlements control various aspects of the UI, such as what menus, toolbar buttons, and controls are available to the user.

6) A Data layer that sits between the UI layer and the middle-tier. The data layer can contain a business facade, smart business objects (that are decorated with .NET attributes), a data access layer (contains a reference to the cache), and service providers that interface with a database and/or a middle-tier. The data layer can communicate with the middle-tier in both synchronous and asynchronous fashion.

7) Middle-tier (usually Java-based) that accesses the underlying database. Communication with the middle-tier is through some variant of XML (usually SOAP). (Sorry, .NET middle-tiers have not caught on yet in Wall Street ... I cite projects at Lehman, Wachovia, Citicorp, Goldman...)

8) Schemas shared between the client and middle-tier.

9) Subscription to asynchronous events, usually by some mechanism as a message bus (Tibco RV or EMS), or Smart Sockets (which is now owned by Tibco). When something happens to the underlying data, the middle-tier will broadcast an event to all of the clients that are hooked up to it. Each client has something that I call a "service agent", which is like a web service in reverse. The service agent receives the subscription event, and propogates the evented data up the chain.

10) A rich event manager. The event manager can be thought of as a mini-Tibco that resides totally within the client. The event manager should support wildcarding.

This stack is being implemented over and over again. The closest thing that I have seen to a corporate standard is the OCEAN framework that Goldman has.

There has to be a vendor who can commercialize this stack and sell it across the various financial companies. Anyone want to start a company with me????

As an aside, the Microsoft Composite Application Block bears close watching. However, it is only available for .NET 2.0. Many of the current .NET-based projects are still using 1.1.

©2005 Marc Adler - All Rights Reserved

The Wall Street Stack

It has been done plenty of times on Wall Street..... what I call the Wall Street Stack.

I just came off of a large project where I architected (yet another) one. I am current involved in one now. So are plenty of other people that I know in other companies.

Everyone's trading applications seems to be the same. Here are the standard elements:

1) The UI layer. The UI layer is usually designed around the Model-View-Controller pattern. It generally uses some third-party UI suite, such as Infragistics, Component One, or Syncfusion.

2) The UI usually contains an Outlook-style navigation bar on the left side. On the right is usually an MDI-ish area. There are multiple tabbed controls, each containing a grid control. The grid is a respresentation of the underlying business objects.

3) Filter criteria that is used to filter the results on the grid. The filter UI is usually placed on the same panel as the grid and consists of a number of comboboxes and edit fields that represent dates and static criteria. Filters can usually be translated into an SQL statement.... a dataview-like pattern can be applied to the underlying model to produce a filtered view.

4) User preferences and workspace persistence. Users can save the layout of the workspace, and load the layout when they log onto a new session from any machine.

5) Entitlements by role. Entitlements control various aspects of the UI, such as what menus, toolbar buttons, and controls are available to the user.

6) A Data layer that sits between the UI layer and the middle-tier. The data layer can contain a business facade, smart business objects (that are decorated with .NET attributes), a data access layer (contains a reference to the cache), and service providers that interface with a database and/or a middle-tier. The data layer can communicate with the middle-tier in both synchronous and asynchronous fashion.

7) Middle-tier (usually Java-based) that accesses the underlying database. Communication with the middle-tier is through some variant of XML (usually SOAP). (Sorry, .NET middle-tiers have not caught on yet in Wall Street ... I cite projects at Lehman, Wachovia, Citicorp, Goldman...)

8) Schemas shared between the client and middle-tier.

9) Subscription to asynchronous events, usually by some mechanism as a message bus (Tibco RV or EMS), or Smart Sockets (which is now owned by Tibco). When something happens to the underlying data, the middle-tier will broadcast an event to all of the clients that are hooked up to it. Each client has something that I call a "service agent", which is like a web service in reverse. The service agent receives the subscription event, and propogates the evented data up the chain.

10) A rich event manager. The event manager can be thought of as a mini-Tibco that resides totally within the client. The event manager should support wildcarding.

This stack is being implemented over and over again. The closest thing that I have seen to a corporate standard is the OCEAN framework that Goldman has.

There has to be a vendor who can commercialize this stack and sell it across the various financial companies. Anyone want to start a company with me????

As an aside, the Microsoft Composite Application Block bears close watching. However, it is only available for .NET 2.0. Many of the current .NET-based projects are still using 1.1.

©2005 Marc Adler - All Rights Reserved

Sunday, November 13, 2005

Full-Layer vs Bottom-Up Reachitecture and Scrum

Mission
To reengineer a credit derivates trading system using Scrum. (30-day sprints)

Findings
The existing application is somewhat monolithic, with direct database access from the UI layer, and business rules embedded in the UI.

Things to consider
1) The users want to see visible functionality after each sprint. We need to rearchitect, but our refactoring must not affect the current UI.
2) We need to implement new structured products after the 3rd sprint.

Question
Would you rearchitect from the bottom up and devote each sprint to deliver a fully-tested foundation layer, working your way up until you hit the UI layer? Or, would each sprint consist of implementing the entire stack with ever-increasing functionality?

I would love to hear your opinions.

©2005 Marc Adler - All Rights Reserved

Microsoft Discovers Scrum

http://developers.slashdot.org/developers/05/11/13/0545244.shtml?tid=109&tid=8

©2005 Marc Adler - All Rights Reserved

Friday, November 11, 2005

Bovine Memories

Posting about my Microsoft contract brought back some interesting memories.

When I started the Microsoft contract to write some GUI-based management tools for their new SQL Server, Microsoft and IBM were still putting the finishing touches on OS/2 Presentation Manager. So, I started writing the software using an internal Microsoft character windowing system called COW .. short for Character-Oriented Windows. This was the only user interface system that MS had for OS/2. (Remember, SQL Server started life as an OS/2 product.) COW was used most notably in OS/2's LAN Manager. I forgot what the API was like, but I still have all my COW libraries and docs on 5 1/4" floppies in a filing cabinet... I just can't bear to part with those disks!

I used to get these 100 lb boxes shipped to me every week from Microsoft. Tons of manuals for SQL Server, OS/2 PM, LAN Manager, OS/2, etc. Thankfully, these went into the trash can a few years ago.

My manager on that consulting contract, Bob Muglia, is now a bigwig at Microsoft. He just got another bump in rank in the most recent corporate restructing at MS.

My experience with giving MS a fixed-price contract taught me that time-and-materials is the only way to go! It didn't help that Microsoft was in a big pissing match at the time with Sybase.


©2005 Marc Adler - All Rights Reserved

Wednesday, November 09, 2005

Just Who Do You Think You Are! (Part 4)

The left episode left Marc writing Windows-based equity workstations for Goldman by day, and packing word processors and text editors at night. (Because the ME Text Editor was one of the first 50 applications written for OS/2, Marc got to go down and show the editor at the IBM OS/2 booth at 1988 Spring COMDEX... learning OS/2 turned out to be a good move, as we shall see in a minute.)

Two opportunities came along at the same time that Marc could not pass up. Marc's wife was a Cobol/CICS for Drexel Burnham at the time (remember Michael Milkin???). Somehow, her boss's boss found out that there was a developer who was writing Windows software for Goldman at the time, and developer who could be enticed away for a certain sum of money. In addition, a new development manager who had just been hired by Microsoft had been looking for an OS/2/PM developer who could help him write software for his new group. Marc could not refuse either, so he accepted both!!!

So, now it was writing a DOS and Windows trading system for Drexel, and helping Microsoft write software for a product that they had licensed from Sybase and renamed.... SQL Server. Marc also started writing articles for Microsoft Systems Journal, which was run by the great Tony Rizzo, bless his soul!!!

Believe it or not, the Drexel contract was the more interesting of the two by a longshot! The problem was that, whatever was written for the Windows version of the trading system had to be replicated on the DOS version as well.... the traders had laptops that could not run Windows very well.

So, Marc had an idea..... why not write the Windows-based trading system in C, using the Windows API. And, for the DOS version, use the same API. This involved reimplementing the Windows API in a DOS library. This led to another product.... the Magma Extensible Window and Event Library... or MEWEL for short.

Next chapter.... MEWEL kicks Magma Systems into high gear....

©2005 Marc Adler - All Rights Reserved

Gentile Pair Programming

http://samgentile.com/blog/archive/2005/11/08/32084.aspx

©2005 Marc Adler - All Rights Reserved

Let the Traders Have Some!

In the good old days, when I was designing trader workstations for Goldman and Drexel Burnham, we use to have close interaction with the traders themselves. We were in an Extreme Programming mode by necessity. Traders were giving us constanmt feedback, and we were able to stand over their shoulders while they traded so we could see how they interacted with the data.

Nowadays, the development staff is usually isolated from the traders... we get our specs from the Business Analysts. Not only that... you often get screen layouts from the BAs too! This often results in Windows apps that look like the old CICS screens... miles and miles of edit and comboboxes, modal interactions, etc. The traders don't mind... they are used to spending a good portion of their days in Excel, or they are used to the austere look of a Bloomberg terminal.

UI design has advanced over the past few years. Let the developers stand over a trader for a while. Let us see how they put a trade together. Let us try to make things easier for them.

©2005 Marc Adler - All Rights Reserved

Tuesday, November 08, 2005

Index and Bespoken Tranches

In the credit derivatives trading system we are building, we need to add support for various structured products. Two of the products are index and bespoken tranches. Other instruments that we will eventually have to implement are NTDs, CDOs, and CDO2.

We all know what a Credit Default Swap (CDS) is, right? It's a form of insurance that someone takes out on a company. One party (the buyer of the insurance) pays a periodic premium to a counterparty (seller of the insurance) for insurance on an underlying instrument (bond or other credit instrument). If there is a "credit event" on the underlying (ie: a company goes bust, misses an interest payment, or the CEO runs away to Bermuda with his secretary), the seller of the insurance must pay the buyer a certain sum of money, known as the "recovery rate". The CDS has some parameters, such as the seniority (who is first in line to get paid in case the company goes belly up), the tenor (how long the buyer has to pay the premiums for), and the notional (the value of the underlying instrument).

There are Indexes on CDS's. We can create our own index, such as the bonds of all of the American auto manufacturers. Or, we can use an exchange-manufactured index.

CDS indices are so popular that there is a CDS index on the Dow. The most widely-traded CDS index is the one for North American Investment Grade instruments. It accounts for over 80% of the CDX volume that is traded.

So, now we know what a CDS Index is. On to Tranches....

A Tranche is a certain level of risk. According to Investopedia:

Tranche is a term often used to describe a specific class of bonds within an offering wherein each tranche offers varying degrees of risk to the investor. For example, a CMO offering a partitioned MBS portfolio might have mortgages (tranches) that have one-year, two- year, five-year and 20-year maturities.

For the above-mentioned index, there are 5 tranches. The Equity tranche covers the first 3% of losses on the credit instruments. The Mezzanine tranche covers from 3-7%. The two Senior tranches cover 7-10% and 10-15% respectively. The Super Senior tranche covers the last 15-30% of losses.

Index Tranches are for active traders. A Bespoken Tranche is one that is put together on the wishes of an investor. It is usually created for long-term investment.



©2005 Marc Adler - All Rights Reserved

Monday, November 07, 2005

Wife + Chaos = Scrum

My wife has inherited this credit derivatives project which seems to be in a huge mess.

She is curious about Scrum, and wants to know if it will bring order to her chaos.

She and her colleagues at the Tier-1 powerhouse have been invited to the Finetix Scrum breakfast on Nov 30.

©2005 Marc Adler - All Rights Reserved

Sunday, November 06, 2005

Be Afraid ... Be Very Afraid

Looks like there are enough serious bugs in the first release of Visual Studio .NET 2005 to make me question recommending it for use (not that any of the tier-1 banks that we deal with are likely to jump with both feet into barely-released software).

Too bad .. we can really use generics on the current project, where we would like to implement strongly-typed caches of various trading instruments.

©2005 Marc Adler - All Rights Reserved

Tuesday, November 01, 2005

I Got the Clearcase/UCM Blues

Sitting here, suffering from ClearCase/UCM .... we have probably lost several hours of work because of UCM troubles.

Deglan warned me about UCM ..... and anything that makes you long for the sweet embrace of SourceSafe must be a new form of pain.

©2005 Marc Adler - All Rights Reserved

Sunday, October 30, 2005

Groupism vs Individualism

Debate on Groupism vs Individualism on the Yahoo Scrum Development Group.

©2005 Marc Adler - All Rights Reserved

Scrum Puzzlement

After the first week at my new client, I have a burning desire to learn more about the Scrum world and apply it to the client. I found out that Yahoo Groups has a group called 'scrumdevelopment', and I have aggregated their feeds to RSSBandit.

I admit that I am new to Scrum. As a development manager, I had always practiced an XP-style of development. Fast iterations, constant feedback, daily meetings, etc. But never the kind of communistic approach that Scrum encourages.

There are two things that I heard about Scrum before coming to Finetix.

1) Scrum is having a hard time being adopted by Wall Street companies.
2) Scrum is much better suited to new development than to enhancing an existing project.

There are two reasons why I see an issue with Scrum and Wall Street companies. First, the inability to commit a team to a Scrum without distractions from anyone outside the team. It is difficult to dislodge a team from attending to daily production problems. It is difficult to tell a revenue-producing trader to leave a team alone while they are in the midst of a sprint. It is hard to tell a team member to skip all of the various classes that they are signed up for.

The second reason is more important. In case you have not heard, Wall Street gives amazing bonuses. And, bonuses are usually ties to individual accomplishments. The tenure of a developer at a Wall Street company is a relatively short stay, and you had better believe that a developer wants to load a resume up with as much individualism as possible.

As far as the issue of using Scrum for existing projects, how do you dislodge a team from support issues on the existing product? How can you have them concentrate on new development when a trader is calling you, screaming about a feature that is broken?

One nice thing is that our manager is very enthusiastic about Scrum. It has worked nicely for a Finetix team in London, but they have been doing new development.

I have a lot to learn about Scrum. The people at Finetix will be amazingly useful in helping adapt to this new way of thinking. I am all ears! And, if it is successful with my project, then perhaps it can be used on my wife's project as well, where she has run into the exact scenario as I have with my new client.


©2005 Marc Adler - All Rights Reserved

Saturday, October 29, 2005

New Finetix Website

Finetix launched a new website. Lots of case studies on their.

We had a launch party at the Finetix NYC office. Steve brought out the good scotch.

Next time, boys, have some Guinness on hand! No more of this lite-beer crap! And have some Bailey's and Jameson's so we could indulge in a few Car Bombs.

(I was introduced to the Car Bomb in 2003 while playing at the annual Colorado MahlerFest, and the orchestra never sounded better!)

©2005 Marc Adler - All Rights Reserved

Just Who Do You Think You Are! (Part 3)

In our last episode, Marc was left without a company to market his famed Unix-based word processor, WPScribe. The company who was in charge of marketing it had blown the company's monetary resources on the excesses of the mid-80's.

Marc was a Unix hack at that time.... even though he wrote a word processor (and, also, a spreadsheet for Unix, WPSpread), he loved ready through line upon line of the Unix kernel.

A small but growing company based on Long Island came a callin' .. their name was Computer Associates. Fresh off their buyout of SuperCalc, they were trying to come up with a window and application management system for all of their future applications. You see, CA envisioned themselves at the time as a software development company, even though very few things (if any) were developed by them. Marc was interested in the notion of working on this project, but he would have to do it all on the IBM PC and MS-DOS! (By the way, the window manager project was cancelled on the day that Microsoft announced Windows...)

Marc took the challenge.... and in the process, ported the word processor over to DOS. However, there was no marketing organization set up to sell the beast. At the time, a movement called Shareware had just gotten underway, and had some momentum. So, Marc decided to release the word processor as a shareware product, selling it for $35 a copy and renaming it "New York Word" (to give it sort of an urban feel).

The word processor did very well in the shareware marketplace, selling several thousand copies. The initial price was $35, but Marc, who was now doing Windows consulting on Wall Street, tried to dissuade people from buying it by raising the price to $47. That only convinced people that the product was now $12 better than it was before.

The largest shareware distributor at the time was an organization called PC-SIG. They decided to embark on a big marketing campaign by highlighting several shareware program. The gist of the ads was to draw a characterture of the author of the program, write a little blurb, and show off the ad in magazines like PC Magazine and PC World. So, if you have old copies around, you can find a drawing of Marc with an elongated face and a lot more hair than he has now!

Now that Marc had a word processor, he needed a nice text editor to go along with it. So, New York Word was stripped down, several programmer-friendly features were added, and since Marc was into compilers, he put together a C-like macro language with which you could program the editor. The editor was called ME, short for Magma Editor. The unique thing about this editor was that it came with full source code, the first commercial text editor to be marketed that way. Advertisements in magazines like Dr. Dobbs Journal and Computer Languages followed, and the editor was launched to an unsuspecting public.

So now, Magma Systems was born. It consisted of Marc doing his Goldman Sachs gig during the day, and packing boxes full of editors and word processors at night. And, occasionally fielding the tech support call from Europe at 3AM in the morning... people who had no idea that the clocks were different when you crossed over the Atlantic.

The best is yet to come....


©2005 Marc Adler - All Rights Reserved

Friday, October 28, 2005

Flying to BTV

Supposed to be a beautiful day on Saturday. Taking the Cessena 172 up to Burlington, Vermont. I think it's too late in the year to experience the full Vermont foliage from the air, but nevertheless, a beautfil flight up the Hudson! Coming back at night, so I get to put in some required nighttime flying hours.

©2005 Marc Adler - All Rights Reserved

Tuesday, October 25, 2005

Dueling Household

My wife, the superstar Wharton grad who walks on water, started at a new client yesterday. She is managing a large team doing a rewrite of an existing credit derivatives system for a Tier-1 financial powerhouse who is not the same Tier-1 that I am currently consulting for.

Their technology is 100% Java.... not surprising, since all of the architecture has been done by a very Java-centric major consulting firm.

Dinner conversation should be very interesting for the next few months.....

©2005 Marc Adler - All Rights Reserved

Monday, October 24, 2005

A Scrumptous Meal

Finetix is holding a Breakfast Seminar on Scrum. This is from my colleague's blog:

In association with Ken Schwaber, we are holding a Breakfast Seminar on Nov 30th 2005 in mid town Manhattan, This 2-hour session is targeted to business and IT professionals working in Wall Street. The team of the breakfast is introduction to Scrum with examples and Case Studies of client projects and experiences. If you are interested in attending, please register by sending an email to: marketing@finetix.com

©2005 Marc Adler - All Rights Reserved