Event Stream Processing

Marco blogs about Event Stream Processing (ESP) at http://rulecore.com/espblog/. There is also a Wikipedia article about it.

ESP is especially interesting in the financial markets, where you want to perform real-time decision making on a stream of quotes coming from your market data distribution infrastructure. ESP gives you opportunities for things like algorithmic trading and real-time risk management.

Companies in this space include Gemstone (Gemfire), KX (KDB+), Vhayu, Streambase, Aleri. I would like to see Microsoft get into this space with a future version of SQL Server. (Since I have some meetings coming up with some people from Microsoft, I will be sure to ask this question.)

I will definitely be exploring this space in the near future.

©2006 Marc Adler - All Rights Reserved

Conflation

The word conflation sounds like a condition that requires you to take a few spoonfuls of Pepto Bizmol. But, it is a term that is used when discussion efficient market data distribution.

The technique has been around for a while. I have always referred to it as aggregation. The term conflation means the combining of data in messages to form one message. You use conflation of messages to overcome slow consumers, limited message queues, and any situation where a message publisher has to throttle the outgoing stream of messages.

Let's say that you are a consumer of stock quotes. The market data distributor receives 1000 quotes for MSFT in one second, but you are only interested in the latest price every second. So, the market data distributor will conflate all of the MSFT quote messages into a single message that gets broadcast to you every second. This means that the server will only send you the last bid/ask price.

You can choose various parameters for the conflation. For instance, as mentioned above, you might want time-based conflation (every 2 seconds, every 30 seconds, etc), you may want conflation based on the size of your message queue, or you might choose your own custom way of doing conflation.


©2006 Marc Adler - All Rights Reserved

Consulting Firms - What Should We Charge?

(This was a post on my blog from a few months ago that I took down. The numbers in this post were arrived at during the course of one train ride from New Jersey to NYC, and in no way are derived from any real consulting firm. These are numbers that should be considered if you are going to start your own IT consulting firm. I also included some of the comments that the original post garnered.)

Consulting Firms - What Should We Charge?

On another blog, a person who is in charge of some consulting dollars (or pounds) at his company questioned why they should pay $X to one consulting company when some temp contractors could do it for $0.5X.

That got me thinking (yet again) about how much high-end consultancies should charge a client per hour or per professional day. I came up with some numbers that I would like to share.

Before I start, some things I want to mention:1) Please feel free to comment in case my numbers are way off base.2) The numbers have no connection whatsoever with Finetix, my former employer. These numbers were arrived at by opening up a random number of fortune cookies and summing up the lottery numbers. 3) These figures are for the US marketplace. I have no idea what the British equivalents to Social Security Tax and 401K plans are.

Fixed Costs Per Employee
------------------------
1) Salary
We want to have a consultancy that has top technical people who have some amount of financial background. We are looking for .NET and Java experts who have done some work on Wall Street in the past, and who can at least tell me what a bond is. In a previous blog posting, I mentioned that the current base salary on Wall Street for this level is about $150,000.

2) 401K/retirement Plan
Let's say that you have a 3% 401K. That adds another $5000.

3) Bonus
Always a tricky thing. Let's take $10,000 for a job well done.

4) Insurance
We pay 2/3 of an employees medical and dental insurance. This amount to about $7000 per year. We also have to pay some other taxes, such an unemployment insurance, etc.

5) Social Security tax
We pay about 7.15 of the person's first $90,000 in income. This is about $6000.

Subtotal - We are paying about $178,000 in salary and benefits per employee.

Costs of Running the Business
-----------------------------

1) Back office staff to support 40 delivery people. We need about 10 people. There are 3 partners, director of Human Resources, director of Benefits, director of Recruiting, two salespeople, director of marketing, plus administrative assistant. If we wanted to chop some dollars, we could probably combine the HR and Benefits. Some of the people are paid totally on commission, some partly, and some are strictly salaried. The partners pay themselves a fixed base salary. So, I am going to put this roughly at $1M.

2) Rent
Nice offices in New York, satellite offices in various locations. Also, cost of utilities, cleaning services, a nice tip for the elevator operators and cleaning people, etc. Let's estimate $400,000.

3) Office Supplies, Furniture, Postage, ISP, etc.
Let's say $100,000.

4) Other items:
- Interest on the Float. The "float' is the number of days between the time you pay your employees and the time you finally collect money from the client.
- Client "incentives" - that nasty, unspoken cost of doing business.
- Marketing Budget
- Conferences and professional organizations

5) Corporate Taxes!!!!

Subtotal - Let's guess about $2M. If we have 40 consultants generating revenue, this is about $50,000 per consultant.

So, now we have the cost per consultant at around $230,000 per year. There are about 230 possible consulting days per year (365 days - 104 weekends - 15 vacation - 10 holidays plus some sick days). This means, if the consultant was billing for all 230 days, we would have to charge the client $1000 per day just to break even. Now, let's say that each consultant is allowed about 10 days of bench time per year. This brings our daily rate up to about $1100.How much profit margin should the partners have? From dealing with various headhunters, I would assume a good profit margin to be about 25-30%. This means that we should be charging a client about $1400 for a consultant.

Our consulting company has a mixture of various skill levels. Some people are more junior than others. So, we create a rate card that goes from $1200 per day for a junior person, to $1600 per day for a senior person. This comes out to $150 to $200 per hour.

Now, at that rate, we are relatively cheap compared to other consulting companies. Microsoft Consulting charges at least $2000 per day. IBM and BearingPoint charge at least $250 per hour for a junior consultant, up to about $450 an hour for a director.

Who are our customers? Certainly, the tier 1 and tier 2 financial institutions. Pharma companies are not going to pay these rates. We are looking to deal with clients who can appreciate our talent pool and who can afford to pay.

Wanna start your own consultancy? These are the numbers that you will have to deal with.


8 Comments:
· I think you're a little on the low side. My former employer charged $250 to $320 per hour, and we had pharmas and other corporates in our clientelle.In yours costs, perhaps factor in some free consulting time to fix screw-ups, late deliveries, etc.What about training? If you want to justify your rate you need highly trained, up to date employees. This is why I left, my employer made no investment in me. I had to train myself on my time, which left me often working around the clock.10 days on the bench per year! Man, I do NOT miss consulting!
By Burnt Out, at 5:38 PM

· Wouldn't the partners take a % of the profits rather than a salary?What about training, free days to clients, sick days?Why isn't the employees bonus based on a % of the profits the company makes?Based on your calculations the is a backoffice person for every 4 consultants? I hate to see the size of the backoffice for 1000 consultantsWhy don't I offshore some of my the work?Can't your consultancy do fixed price?
By Anonymous, at 6:30 PM

· Wouldn't the partners take a % of the profits rather than a salary?Possibly. Depends on how risk-averse the partners were. If I was a partner, I would pay myself a low salary ($100K) and then at the end of the year, divvy up the profits.What about training, free days to clients, sick days?I mentioned 5 sick days in my original post. Training is nice, and I would like to give at least a week of training, so that can be done during the 10 days of bench time. OK, let's add in about $100K for training. Free days to clients? Never heard of this!Why isn't the employees bonus based on a % of the profits the company makes?Because many smart people will negotiate a guaranteed fixed bonus when they join, especially since the debacles of 2002 and 2003. But I might give an added "incentive" bonus to some of my best performers at the end of the year. OK, let's add another 100K.Based on your calculations the is a backoffice person for every 4 consultants? I hate to see the size of the backoffice for 1000 consultants10 people should be a constant. Even when your company grows to 100 people. In fact, at the start, I might consider consolidating the benefits and HR functions, and make the partners do more marketing. So, we can cut our staff back to 8.Why don't I offshore some of my the work?Never, never! This is what distinguishes my shop from other larger ones. Strictly all client-facing work.Can't your consultancy do fixed price?I abhor fixed-price contracts. We could do it, but perhaps under duress!One thing that I forgot to mention was finder’s fees. If I hire a person through an employment agency, then I have to pay then 15-30% of the employee's first year salary. That's a significant chunk. Let's assume $30,000 for 10 people, and that's another $300,000.So, we have added another $500,000 in costs to our model. That's about $10K per employee. Gotta bump up the rate card by $50 per day. So, our rate card is now $1250 through $1650. And, coincidentally, $1650 per day is what a lot of the top-tier banks pay for senior technical people.

©2006 Marc Adler - All Rights Reserved

The Start of a New Adventure

On the eve of starting a great, new, exciting position, I would like to recap my experiences over the past 4 weeks in searching for a new job. One of the motivating factors in looking for a new job was that I wanted to have an important role in shaping the development and architecture of a major financial system, and do so at a senior level. Being a hands-on coder was OK for me on a part-time basis, but I wanted to manage and mentor developers who would be doing most of the coding while devoting more of my time to learning about new technologies in the financial arena, meeting with vendors, sitting on enterprise architecture committees, etc.

It’s always good to have some throwaway interviews, especially if you have not interviewed in quite a while. As soon as I announced my intentions of moving on to something different, I received a number of calls for interviews. I knew that some of the positions were not right for me, but I needed to get some spit and polish on my interviewing style before going out to the big boys.

The first interview was for a consulting company that claimed to do some financial work, but whose main focus was content management. Nevertheless, I needed to meet with people, so I went down for the interviews. The company had a “boiler room” full of cold-callers, something that makes my blood curdle. I met with the “HR lady” and then (believe it or not), the Head of Business Development. What kind of company would have you meet with their head salesperson? After my ideas about how a consulting company should be run went right over the salesguy’s head, I met with one of the partners. He was more perceptive, in that when I told him that I did not want to work for a consulting company that had the singleton, bodyshopping model, I could detect the dark cloud passing over his face and his immediate desire to end the interview as soon as possible.

The next day was another throwaway. A hedge fund had been after me for a few months. The headhunter told me that, as part of the job spec, the hedge fund requested that candidates live in New York City so that they could be on constant call. 14 hour days were the minimum. Joking with the headhunter, I told him that if I were to work 14 hour days, that I would want my current salary * 1.75 as a base. He called me back a week later and said that the hedge fund still wanted to talk to me. After an initial phone interview, I went down to see the people from the hedge fund. The fund is technically astute; they were developing totally in .NET 2.0, although they were doing all ASP.NET apps. The only thing that the first two interviewers, both developers, did was to repeatedly thumb through my resume, and exclaim that I had done a lot of stuff. Finally, I interviewed with the CTO. We talked about a lot of conceptual things, and he asked me one SQL question, which I got about half right. (I happen to have a mental block when it comes to writing SQL during an interview – give me SQL Query Analyzer, and I can work up the answer in 5 minutes). When the CTO asked me about my compensation requirements, I told him that I thought that the recruiter communicated that to him already. The recruiter never did, and when I mentioned the figure to him, he rolled onto the floor laughing. We ended the interview about 8PM, and he told me that his day was just beginning.

Two interviews without a good tech question.

The next interview was with another, very small hedge fund. The CTO was a really nice guy, except that I wanted his job! The pay was terrible. About 60% of my current base, although a 30-40% bonus was typical. To his credit, the CTO knew that he could not support my compensation level, and we both knew that it would have been a mismatch if I had joined.

Still no tech questions.

Later that day, I went down for a face-to-face with a very small company that specialized in financial tools. I had actually spoken briefly to this company a few months ago, but they really wanted a GUI developer who specialized in C++. Seeking a good throwaway interview with some good, hard tech questions, I contacted this company again to see if the position was open. It was! I had an initial one-hour phone conversation with their senior developer, and then was invited down to their offices for a second round. The questions I was asked included:

- Design the architecture and the API for a logging mechanism
- Implement a breadth-first and depth-first search for a tree.
- Design a hash table.
- Implement a workflow manager.
When the CEO of the company walked in to interview me, I almost fainted. The CEO looked like he was 12 years old. (He is actually a MIT grad.) I felt like I was interviewing my son. OK, I think that we both realized at that point that there was a slight cultural difference!

A phone interview with one of the big IBs followed. The job spec was for a C# developer for an Order Management System, but the recruiter kept insisting that it was a Managing Director’s position. I told the recruiter specifically that I did not want an Associate Director or Director position at this company, but she kept changing her story with regards to the title. The phone interview was a non-event. No tech questions; just a manager going over my past projects and asking me where I wanted to be five years from now. A very, very dry interview.

Then came two wonderful interviews with two different groups at an IB, one for consulting and one for full-time. I got offers from both these interviews. More on this later. I was asked to design a crossing engine. I was asked about the .NET garbage collector. I was asked to draw an architecture for a trading system. I was asked about bottlenecks in the system.

I got verbal offers from both of these groups the next day.

I knew that I was going to take one of these two positions, but I went on two other interviews as backups. One was with a big ad agency who needed rearchitecture of some big websites. Nice company located in the hinterlands of New Jersey, and they came through with a very nice consulting offer at a pretty good rate. It would have been a good change of pace, and I would have taken it if the other offers did not come through. Another interview was a phone talk with the head dev and the manager of a group at another IB. Their systems were in a mess, and they needed rearchitecture and new development on their big Private Banking systems. Not really a position that I wanted. Both of these interviews contained no tech questions.

One other background conversation was with a small company that specialized in software and consulting services for hedge funds. Really nice guys, and would have been great if I had wanted to bill by the hour and customize software for individual funds, but they were looking at me to take over the CTO role. Unfortunately, they did not have a sufficient revenue stream to support non-client-billable activities, and sadly, it would have been more of singleton consulting.

In the course of interviewing, I met some very good recruiters who I will be sure to use in my new position when I need staff. I also met some very, very bad recruiters who a) did not communicate salary requirements to their clients, b) did not give me feedback, c) lied about the titles of the positions. Among the goodies I met were Matt of SH and Drew of CMA. Cheers guys!

I was disappointed in the level of tech questions that I received, and was surprised that most of the interviewers would just spend time gawking at my resume. They were actually embarrassed to ask me the generic tech questions (ie: what is a virtual function, what is a delegate, etc). However, the design-a-system questions were good, and I have to remember to use some of them, especially the one about designing a workflow engine.

My new position is a senior-level position at one of the IBs. I will be in charge of architecting some new trading systems and rearchitecting some older ones. I will have a small staff of good people. My manager is a very dynamic personality, and I am hoping that we can move this organization forward in terms of trading technology. I will be involved in new technologies (to me, at least), and will be investigating low-latency market data distribution, algorithmic trading, universal GUIs (my Wall Street Stack), and more.

©2006 Marc Adler - All Rights Reserved

Testing, Testing

Anyone out there still?

Important announcement coming soon....

©2006 Marc Adler - All Rights Reserved

Next Move

I am in the market for new opportunities. My interests are hedge funds, trading technology, .NET.

If you have something interesting going on, please email me at magmasystems@yahoo.com

I am going to take a few weeks off, investigate technologies that I have been too busy to investigate, and watch the rest of the World Cup.

©2006 Marc Adler - All Rights Reserved

Things to Learn

Going through some of the reading from SIA. Tons and tons of stuff out there. On my "to-learn" list are:

MDDL - market data definition language, yet another XML-based way of transmitting financial info

Gigaspaces - grid-based infrastructure

Vhayu - real-time and historical data storage with real-time calculations and event broadcasting

Wombat - another high-speed market data and messaging platform

©2006 Marc Adler - All Rights Reserved

Another Interesting Job Posting

http://newyork.craigslist.org/mnh/sof/173118487.html

They are located on the 4th floor of a building on Franklin Street, which is in the area of New York City called TriBeCa. In the old days, Franklin Street had many spacious lofts ... which got me thinking back to the days where a lot of young software companies worked out of lofts or apartments. In fact, in 1986, I worked for a young startup called Multex. There were 3 or 4 of us sitting in an apartment on West 53rd Street, coding apps for Goldman Sachs. Multex went on to expand, go public, and get bought out by Reuters.

Former Multex employee Brett Schlussman left just after me, and formed NetLogic with another former Multex guy (Michael St. Hippolyte) and a developer from Bankers Trust. They worked out of Brett's apartment in the 20's, taking Brett's Windows-based 3270 emulator that he developed for First Boston and selling it to Eicon.

We have interviewed a number of ex-Multex employees recently at Finetix. These were people who joined Multex after the Reuters merger. They had no idea that they were being interviewed by Multex employee number 5.

©2006 Marc Adler - All Rights Reserved

SIA Technology Show 2006

I stopped by the Securities Industry Association show at lunchtime today. Walked around the exhibit hall and saw 1000 different instances of "the Wall Street Stack". Ow .... my head hurt from thinking about how many times the same stack has been rewritten in different ways for the sole purpose of displaying real-time charts and blotters. If you were a financial software company, how much would you pay for a completely operational version of The Stack?

Picked up a least 50 magazines, newsletters, and newspapers on trading technologies (hardly anyone was giving out T-Shirts this year -- I am a real T-Shirt collector). Waters Magazine still seems to be the best one out there for hard-core Wall Street developers.

There was a nice article on pages 21 and 22 of the Summer 2006 issue of Windows in Financial Services on a framework that I helped architect at Wachovia last year. The framework is what they now call OneSource. Several apps in Wachovia are based on OneSource, and I am happy to report that the ClientNet application that I worked on last year is being rolled out worldwide at Wachovia. One of the mags had a nice bit on the adoption of grid technololgies within Wachovia --- my man R.O. rolled that stuff out.

Larry from Finetix gave a nice talk on Quality Assurance in Capital Markets. Hopefully, some past and current clients attended that one! Larry is one of the best in the business.

Fourth of July coming up .... we have to get ourselves out to Block Island


©2006 Marc Adler - All Rights Reserved

Derman is in the House

Famed quant Emanuel Derman is blogging.

http://www.wilmott.com/blogs/eman/index.cfm

I just started reading his book My Life as a Quant. Interesting to read his experiences at Bell Labs at the start of the whole Unix/C movement in the late 1970's.

I will definitely have some more comments as I read through the book.

©2006 Marc Adler - All Rights Reserved

Interesting Way to Recruit People

http://newyork.craigslist.org/mnh/sof/168208451.html

©2006 Marc Adler - All Rights Reserved

YABGT (Yet Another Bill Gates Tribute)

I will join all of the other bloggers out there and put in my two cents about Bill Gates.

I had my first and only brush with Gates back in 1991 or 1992. I was a contributing editor for Microsoft Systems Journal back in the old days. One day, I get a call from my editor. He tells me that Bill Gates is stopping in New York for a night, on his way back to Seattle from a European trip. He wanted to have dinner with the MSJ contributing editors to tell them about some exciting stuff in the industry. Would I be interested?

We all met at a Japanese hotel on Park Ave at 32nd St. This hotel had a beautiful restaurant, and Gates, being a sushi fanatic, reserved a tatani room for us.

We all get there at 8PM, and then in walks Gates, followed by his flack, Jon Lazarus (who used to be the publisher of MSJ). In attendance was myself, Ross Greenburg (author of RamNet, and later, some virus-checking program), Greg Comeau (author of Comeau C++), Charles Petzold (author of a book), and Tony Rizzo (my editor).

Gates turned out to be a great guy, despite the constant leg movements under the table. Very personable. He spent most of the night talking about getting Dave Cutler from DEC, and the new operating system he was working on (Windows NT). Every word was preceeded by a stern warning from Lazarus never to repeat what was said.

Of course, the very next week, PC Week featured a big story on Cutler and NT!!!

©2006 Marc Adler - All Rights Reserved

Never thought they could beat Spiderman!

http://www.derivativesstrategy.com/magazine/comix/

©2006 Marc Adler - All Rights Reserved

Housing Price Futures

I heard about this on NPR radio the other day ... A great way of hedging a housing market. If you are buying now, and are afraid that the housing market might fall through the floor in your town, then you can short the housing future. If you think that your area might see an uptick in housing, then long the future.


(from a CBS Marketwatch article)

The Chicago Merc introduced cash-settled futures and options based on housing markets in Boston, Chicago, Denver, Las Vegas, Los Angeles, Miami, New York, San Diego, San Francisco and Washington, as well as a weighted composite index.

The contracts are priced by multiplying the index value by $250; the average contract size for the various cities is roughly $55,000, although they vary by region, according to Sayee Srinivasan, associate director of research and product development at the CME. He said the exchange is considering adding contracts for other booming real-estate markets such as Phoenix and Orlando.

"The futures markets like volatility, so the contracts based on the most volatile housing markets might see the most volume," Srinivasan said.

For each region, there are four contracts, which expire on a quarterly basis, so there are currently contracts for August 2006, November 2006, February 2007 and May 2007.

Although the contracts have only been trading for a very short period, most of the interest is looking further out on the curve with the May 2007 contracts getting the most action initially, said Fritz Siebel, senior broker at Traditional Financial Services Inc.

There could be demand for longer-dated contracts, something the CME is considering if the existing contracts prove popular enough.

Siebel noted the early interest has been in derivatives based on hot West Coast markets in San Diego, Los Angeles and Las Vegas, and also Miami on the East Coast.


©2006 Marc Adler - All Rights Reserved

FX Settlements

The CLS website has a great picture on how settlements work for FX trades. Here it is:





CLS is a bank set up by a consortium of different financial firms. It what my client uses for settling their FX trades.


©2006 Marc Adler - All Rights Reserved

Services for the Wall Street Stack

I have blogged before about the need for a standardized Wall Street Stack. At my current client, there are several different frameworks that have been built over the years, but no framework has gained enterprise-wide acceptance.

These are the services that are generally needed for a complete application framework. If you can think of any other, let me know.


Authentication - logging in to various databases, web servers, internal systems, etc. Best implemented through an enterprise-wide authentication service (using the user's Kerberos token, etc)

Authorization (Entitlements) - what actions can a user perform within an application and what data is a user entitled to see. Best accomplished through an enterprise-wide system.

Logging - log interesting things that happen in an application. Log4Net is a good framework.

User Preferences - load and save the state of the UI

User Data - each application may want to persist the last state of the application when the user logged off

Configuration - Each application should be configurable through a series of XMl files.

General Services Manager - drives the entire service locator framework

Applet Management - load applets from .NET assemblies

Internal Event Broker - lightweight communication between applets

Data and Persistence layer - ways to load, query, update and cache data. Should be configurable so that we can go through different transport layers

Service Agents - processed async notifications that come to us through subscriptions.

Shell - handles initialization of the application, calls all of the various services, provides presentation services for the applets

UI - menu manager, status bar manager, navbar manager, toolbar manager

Security/Cryptography - does the data need to be encrypted and decrypted? (passwords, sensitive data)

Exception Handling - provides a unified way of handling errors that occur in shell and the various applets

Threading - model for async processing


©2006 Marc Adler - All Rights Reserved

Universal DBs

Lately, I have been talking with IT directors at various financial firms. There seems to be one consistent message --- The holy grail that all of the financial firms are seeking is the universal database, where all trades across all asset classes can be held.

This is especially problematic at financial firms who keep buying other firms (Wachovia, Bank of America).

OLAP is a hot skill right now, and companies what to perform sophisticated queries over huge amounts of data. Finetix currently is working in the OLAP space int he commodities area of a major financial firm. This firm has asked Finetix to invent a cloning machine so we duplicate our 3 OLAP stars and sprinkle them throughout the firm. Maybe I should start attending Andrew Brust's presentations at the various .NET user groups.

©2006 Marc Adler - All Rights Reserved

How Do They Find Me?

I have been getting a bunch of calls from recruiters at my desk at my current client. I think that they know the telephone prefixes of the IT departments at the various Wall Street companies. The recruiters call every single number with that prefix in the middle of the night, and wait for the voice mail to kick in. So, they can get a name of a person at a desk at a Wall Street company.

The calls then come....

Recruiter : Hi Marc. I know someone who worked with you at your last company and they thought you were great.

Me (seeing the bullshit form a mile away) : That's fantastic! What company was that?

Recruiter : (sound of frantic typing in the background). Did you say that this was Marc Adler? Hmmm... it was ... XYZ.

Me : Who was the person that referred you?

Recruiter : Let me tell you about this position.

Me : (hangs up)

A recruiter's opening line says everything about his veracity and ethics. If a recruiter comes across like a telemarketer, I will not deal with him. A better line might be :

Recruiter : Hi Marc. I have been retained by Bank of America to hire a team that needs to build a credit derivatives system. Their platform will be a .NET front end, Java backend, wired together by Tibco EMS. Since you are working now for XYZ, and since I know that their credit derivatives systems use that precise technology, I was wondering if you or any of your teammates would be interesting in implementing a system like this over at BofA. Of course, the compensation would be above and beyond your current salary, which given your level, I could probably take a guess at.

Why is this good? I find out the name of the company who is hiring, and I already know that it's not with the company that I am currently with. (The worst thing is to find out that the job is for your position at your current company!). I see that the recruiter has taken the time to find out that I am with the credit derivs area, and has taken the time to know that my current client uses a particular technology. I see that the recruiter is aware of current salaries and could divine what it would take for an established player to move to another company.

I don't want to have to play games with a recruiter to find out about a position. Be straight, be honest. If I am truly interested in something, then I will not do an end-run around you.

Also, do you guys realize that, if you call me in the middle of the day, then the 5 people who sit around me usually hang on every word that I say on the phone?


©2006 Marc Adler - All Rights Reserved

Interesting Call From Microsoft Recruiting

It used to be where millions of applicants would send their resumes to Microsoft in hopes of a job. Howver, this is not your father's Microsoft anymore. The stock has been in the shitter for the past few years (I sold mine for a split-adjusted price of $30 in 2001, and the stock has never gotten higher), and the options program has been dessimated.

Somehow, a Microsoft recruiter got a hold of my name from some past colleague. There is a developer evangalist position open in the New York/New Jersey area. I happen to know the person who just left that position; he had been a familiar face at the various New Jersey .NET User Group meetings. This is not a bad position .... you get to go to companies and user groups around the NYC/NJ area and talk about the latest MS technologies from a developer perspective.

So, this recruiter kept calling and calling. I was not interested in the position at all, but I wanted to get another data point and hear what MS was offering senior-level architects and devs now. She would call me in the middle of the day, at my desk at my client, and I would patiently explain to her that I could not talk about a new job with 5 people sitting around me. She would make plans to call me in the evening, but then had to go pick up her kids somewhere, watch her neighbor's kids, etc. Always some excuse.

We finally managed to connect this week. She started to explain the job to me, but I told her that I was already very familiar with the responsibilities of the position. I told her to get down to brass tacks. What were they paying? I told her what my current base salary was so that we could start negotiating. My hopes would be that she could come close to the base,

She looked in her little black book, muttered to herself "hmmm... since this IS in the New York area, we have to give you a boost in the base", and came up with a number that is about 40% less than what I am making in base salary now.

To top it off, she then asked me why I would leave my current position for one that paid 40% less. This got me a bit steamed. I told her that:

a) It was YOU who have been persuing me, not the other way around. I never said anything about wanting to leave my current position.

b) I would NEVER leave for 40% less, unless it was topped off by outright stock grants that would raise my total comp to a figure that was way over the base (something that they will not guarantee for this position).

c) I told her that I was an avid reader of the Mini-Microsoft blog (she said something like "Oh No!"), and that under the current conditions at Microsoft, I don't think that I would be interested in subjecting myself to that type of lifestyle. In addition, I was not interested in interviewing with some 25-year old pisher who would be asking me logic questions like "How Do You Move Mount Fuji?".

She asked me if I had any colleagues who would be interested, whereby I replied that all of my colleagues who might be interested were making a lot more money than you just offered. I told her that I would never consider at MS until there was evidence of a complete culture change and/or there was sustained upwards movement in the stock price. I told her that she should study Mini-Microsoft very carefully, and note the exodus from her her company.

What an amazing turnaround for the prestige of Microsoft, a company that I used to admire. I consulted for MS for 3 years back from 198901992, writing the first front end for SQL Server. My former MS manager is a group VP, and is a millionaire many times over. But, I am afraid that most of the bright lights there have now been dimmed.



©2006 Marc Adler - All Rights Reserved

What I am doing now

At my new client, one of the Tier-1 IBs on Wall Street, I am now in charge of a FX Exception Processing System. What this system does is allow FX Operations to examine and fix any FX trades which did not go through for one reason or another.

The front end is all C#/.NET, communicating with a Java trade engine using both Web Services and a home-grown XML message broker. Sybase is the database, and from the backend, connectivity is to both CLS and to SWIFTNet.

The GUI uses the Infragistics toolset, and is very grid-intensive. The real-time element is not crucial, and the trade volume is light enough so that we do not have to consider switching to the SyncFusion grid right now.

Any FX Trade Events (in this system, notification of a broken trade is called an 'event') that occur are sent from the backend to our GUI using the message broker. All FX Trades are also sent to our GUI. If the trade that comes in matches a filter criteria that is set up in one of the grids, then the trade is updated (or, in some events, the trade is deleted from the grid).

As more and more groups come on line and use our system, we will be runing into scalability issues. For example, let's say that you have a grid open that is monitoring all trades that occur today. Let's also say that no filtering criteria is set up, other than the fact that you want to look at today's trades. This means that every new trade will be inserted into the top of the grid. This is OK for now, as the trade volume is relatively light. But, more groups have expressed using our GUI. We anticipate that, by the end of the year, we might experience 10x the volume we have now.

I am going to be writing some stress-testing tools that will help us analyze the possible impact of other groups using our system. More on this later.

©2006 Marc Adler - All Rights Reserved

Comings and Goings

In the United States, men are glued to their TV sets, watching the National Football League's annual college draft. This is the time when 26 football teams throw a ridiculous amount of money at elite college atheletes.

On Wall Street, the annual post-bonus dance is almost complete. As people received their bonus checks in February or March, news comes almost every day that this person or that person has departed for company X or Y. This has dessimated several's company's IT departments, as the one person who knows a certain application inside and out is suddenly no more .... and the mad scramble for talent starts anew as companies try to backfill these positions.

(Wall Street is very small ... never burn your bridges.)

Just like the NFL draft, those of us in Wall Street IT sit back and await news to see where our colleagues end up. Such is the situation with me. I sit back and monitor the blogs, anxiously waiting to see where Matt, Pin, and Deglan end up. Matt announced his resignation on his blog, and invited all comers to bid on his services. In his last blog post, he said that he had many offers from his blog readers. Whoever the lucky company is who ends up with Matt, it will be money well spent.

The bar has been raised slightly in terms of compensation packages for senior developers. On Wall Street, a total compensation package of slightly over $200K is becoming more common. The calls from headhunters are becoming more frantic, as Wall Street scrambles to fill positions of their dearly departed developers. The base salary seems to still be $150K, but on Wall Street, bonus is everything. Managing Directors typically make 200-300% in bonus. Developers are still in the 30-40% range in most companies,except for the premier companies, where your 18-hour days might get you upwards of 80%. But, as a colleague of mine told me recently, many a marriage has been sacrificed in exchange for that bonus.

©2006 Marc Adler - All Rights Reserved

College Admissions

I just returned from a trip to California with my son. We have just started to look at colleges for him. He is a junior in high school, but this is when kids start considering colleges seriously. We went to see U Cal at Berkeley, Stanford, and Pomona. All three very different colleges.

Warning to parents: good grades and good standardized test scores just don't cut it anymore. Stanford rejects an awful lot of kids who get 1600's on their SATs. All of the colleges want to see rigorous courses in high school, great grades, great SAT I and SAT II and AP/IB scores. But they also want students that are unique in some way. The essays that kids have to write for their college applications are extremely important.

It is not only amazing difficult to get into the top-tier schools. It is also getting increasing difficult to get into the upper-middle tier schools. Applications are skewed by professional college counselors who you can hire for $10,000 to $30,000 .... you wonder if a kid gets into a certain college because of his/her own worth, or because daddy's paycheck is fat enough to hire a pro.

Now, we have to capture the one big intangible that my son has into an essay .. the fact that he is a leader ... not in the academic sense (ie: he is not the editor of the school newspaper nor the quarterback of his school's football team) ... but the fact that he has packs of kids that follow him around, that kids are constantly calling him and messaging him, that he can snap his fingers and a whole cadre of students will appear to schlep his drumset from gig to gig. In other words, someone who has that special magnetism that command attention amongst his peers (think Tony Soprano).

I wonder if Stanford is interested in the future leader of La Cosa Nostra.....

(To the British readers here .. does the same thing apply to Oxford and Cambridge? Is most of the UK clamoring to get into these two schools?)

©2006 Marc Adler - All Rights Reserved

Event Manager Code - Possible Enhancements

Now that I have posted the code for the Event Manager that I have used in several projects, I would like to list some possible enhancements that I have considered:

Precompile regular expressions
The Event Manager supports wildcards. The .NET RegularExpression classes are used to perform matching of a published topic with a wildcarded subscription. Howver, I do not pre-compile the regexps. This is a very simple change to make.

Enhanced regular expressions
We can expand the regular expressions that wildcarded subscriptions will accept. For example, to subscribe to an insert or delete action for a trade, we should be able to have a wildcard topic that looks like this:
Magmasystems.TradingSystem.Trade.[InsertedDeleted]

Stronger typing of args for event declarations
The third argument of EventManager.Fire() is anything that derives from EventManagerArgs. We can add more type safety by specifying the subclass of EventManagerArgs that a publisher expects to send and that the corresponding subscribers expect to receive.

Scoping
Microsoft's CAB has the notion of event scoping. We can limit a published event to subscribers that run in the same thread (or a certain thread) as the publisher. We can limit the scope to a named "applet". Otherwise, by default, the event is published globally.

EventManager.Register(this)
I am not totally thrilled with the fact that, for every object that publishes or subscribes, a call to EventManager.Register(this) is called. The Register() function will examine the passed class, using reflection on the class to build up a dictionary of publishers and subscribers to a topic.

Dynamically register the publisher and subscriber
Instead of using the static way of decorating a method, we may want to add publications and subscriptions dynamically. For instance, we should be able to say something liek this:

EventManager.Subscribe("Magmasystems.TradingSystem.Trade.*", this.OnTradeEvent);

Sinks for event publishing
Implement Sinks so a class can have a crack at an event args before it is sent to the subscribers.



©2006 Marc Adler - All Rights Reserved

Event Manager Code - EventManager.cs

Here is the code for the main EventManager class.

There are a number of ways that I would like to improve the code, but just never have had the time. When I get a few free moments, I will discuss some of the enhancements that I would like to put in the code eventually.

// for MethodImplAttrbute
using System;
using System.Collections;
using System.ComponentModel;
using System.Reflection;
using System.Runtime.CompilerServices;
using System.Text.RegularExpressions;
// for ISynchronizer

// --------------------------------------------------------------------
//
// This code is (C) Copyright 2005 Marc Adler
//
// --------------------------------------------------------------------

namespace Magmasystems.EventManager
{
  public delegate bool EventManagerEventHandler(object sender, string topicName, EventArgs e);

  public class EventManager
  {
    #region Delegates

    // Internal delegate used to help us fire async events
    private delegate void AsyncFire(Delegate del, object[] args);

    #endregion

    #region Variables

    // The singleton EventManager
    private static EventBroker _theEventManager = null;

    // A Dictionary of EventManagerEventInfo classes, indexed by the topic name
    private Hashtable _theEventDictionary = null;

    private Hashtable _theWildcardSubscribers = null;

    // Used to disabled and re-enable the firing of events
    private bool _isEnabled = true;

    #endregion

    #region Constructors and Static Instance

    protected EventBroker()
    {
    }

    // This is the way that the EventManager singleton is accessed
    static public EventManager Instance
    {
      get
      {
        if(_theEventManager == null)
        {
          _theEventManager = new EventBroker();
          _theEventManager._theEventDictionary = new Hashtable();
          _theEventManager._theWildcardSubscribers = new Hashtable();
          _theEventManager._isEnabled = true;
        }
        return _theEventManager;
      }
    }

    #endregion

    #region Properties

    private Hashtable Dictionary
    {
      get
      {
        // make sure that the dictionary and event manager are instantiated
        return _theEventDictionary;
      }
    }

    private Hashtable WildcardDictionary
    {
      get
      {
        // make sure that the dictionary and event manager are instantiated
        return _theWildcardSubscribers;
      }
    }

    public bool Enabled
    {
      get { return this._isEnabled; }
      set { this._isEnabled = value; }
    }

    #endregion

    #region General Methods

    static public void Register(object o)
    {
      Type type = o.GetType();

      MethodInfo[] methods = type.GetMethods();
      foreach (MethodInfo mi in methods)
      {
//        if(mi.DeclaringType != type)
//          continue;
        object[] oAttrs = mi.GetCustomAttributes(true);
        foreach (object oAttr in oAttrs)
        {
          EventManagerEventInfo evInfo;
          if(oAttr is EventSubscriberAttribute)
          {
            EventSubscriberAttribute evSubAttr = oAttr as EventSubscriberAttribute;
            evInfo = FindTopicEntry(evSubAttr.Topic, true);
            if(evInfo != null)
            {
              EventSubscriberInfo subInfo = evInfo.AddSubscriber(evSubAttr.Topic, o, evSubAttr.IsBackground, mi);
              if(subInfo.HasWildcards)
              {
                EventBroker.Instance._theWildcardSubscribers[evSubAttr.Topic] = evInfo;
              }
            }
          }

          else if(oAttr is EventPublisherAttribute)
          {
            EventPublisherAttribute evPubAttr = oAttr as EventPublisherAttribute;
            evInfo = FindTopicEntry(evPubAttr.Topic, true);
            if(evInfo != null)
            {
              evInfo.AddPublisher(evPubAttr.Topic, o);
            }
          }
        }
      }
    }

    static private EventManagerEventInfo FindTopicEntry(string topicName, bool createIfEmpty)
    {
      EventManagerEventInfo evInfo  ;

      topicName = topicName.ToUpper();
      object oEventInfo = EventBroker.Instance._theEventDictionary[topicName];

      if(oEventInfo == null)
      {
        if(createIfEmpty)
        {
          // Allocate a new entry
          evInfo = new EventManagerEventInfo();
          EventBroker.Instance.Dictionary[topicName] = evInfo;
        }
        else
        {
          evInfo = null;
        }
      }
      else
      {
        // Use the existing entry
        evInfo = oEventInfo as EventManagerEventInfo;
      }

      return evInfo;
    }

    #endregion

    #region Ways to Fire and Event

    [MethodImpl(MethodImplOptions.NoInlining)]
    static public void Fire(object sender, string topicName, EventArgs args)
    {
      if (EventBroker.Instance.Enabled == false)
        return;

      EventManagerEventInfo evInfo = FindTopicEntry(topicName, false);
      if(evInfo != null)
      {
        evInfo.Fire(sender, topicName, args);
      }

      foreach (object oKey in EventBroker.Instance._theWildcardSubscribers.Keys)
      {
        string keyName = oKey as String;
        Regex regExp = new Regex(keyName, RegexOptions.IgnoreCase);
        if(regExp.IsMatch(topicName))
        {
          evInfo = EventBroker.Instance._theWildcardSubscribers[oKey] as EventManagerEventInfo;
          evInfo.Fire(sender, topicName, args);
        }
      }
    }

    #endregion

    #region Inner Class for EventManagerEventInfo

    /// 

    /// EventManagerEventInfo
    /// This class represents information about a single event
    /// 
    public class EventManagerEventInfo
    {
      #region Variables

      // Multicast delegate of all subcribers to this event
      private event EventManagerEventHandler _eventHandler;

      // The list of classes that publish this event
      private ArrayList _publishers;
      // The list of classes that subscribe to this event
      private ArrayList _subscribers;

      #endregion

      #region Constructors

      public EventManagerEventInfo()
      {
        this._publishers = new ArrayList();
        this._subscribers = new ArrayList();
        this._eventHandler = null;
      }

      #endregion

      #region Events

      public event EventManagerEventHandler EventManagerEvent
      {
        add { this._eventHandler += value; }
        remove { this._eventHandler -= value; }
      }

      #endregion

      #region Properties

      public EventManagerEventHandler EventHandler
      {
        get { return this._eventHandler; }
      }

      public ArrayList Publishers
      {
        get { return this._publishers; }
      }

      public ArrayList Subscribers
      {
        get { return this._subscribers; }
      }

      #endregion

      #region Methods

      public EventSubscriberInfo AddSubscriber(string topicName, object objectRef, bool isBackground, MethodInfo mi)
      {
        EventSubscriberInfo subInfo = new EventSubscriberInfo(this, topicName, objectRef, isBackground, mi);
        this._eventHandler += new EventManagerEventHandler(subInfo.OnPublisherFired);
        this.Subscribers.Add(subInfo);
        return subInfo;
      }

      public EventPublisherInfo AddPublisher(string topicName, object objectRef)
      {
        EventPublisherInfo pubInfo = new EventPublisherInfo(this, topicName, objectRef);
        this.Publishers.Add(pubInfo);
        return pubInfo;
      }

      public void RemoveSubscriber(EventSubscriberInfo sub)
      {
        this.Subscribers.Remove(sub);
      }

      public void RemovePublisher(EventPublisherInfo pub)
      {
        this.Publishers.Remove(pub);
      }

      #endregion

      #region Event Firing

      public void Fire(object sender, string topicName, EventArgs args)
      {
        // This handles the event firing to the SubscriberInfo class. In turn, the
        // SubscriberInfo object will invoke the actual delegate. The SubscriberInfo
        // object will determine whether the delegate shoul dbe called synchronously
        // or asynchronously.
        if(this.EventHandler != null)
        {
          this.EventHandler(sender, topicName, args);
        }
      }

      #endregion
    }

    #endregion

    #region Inner class for EventPublisherInfo

    public class EventPublisherInfo
    {
      #region Variables

      private EventManagerEventInfo _evInfo;
      private WeakReference _objectRef;
      private string _topicName; // the upper-cased name
      private string _displayName; // used to generate context menus at runtime

      #endregion

      #region Constructors

      private EventPublisherInfo()
      {
        this._objectRef = null;
      }

      public EventPublisherInfo(EventManagerEventInfo evInfo, string topicName) : this()
      {
        this._evInfo = evInfo;
        this.DisplayName = topicName;
        this._topicName = topicName.ToUpper();
      }

      public EventPublisherInfo(EventManagerEventInfo evInfo, string topicName, object objectRef) : this(evInfo, topicName)
      {
        this._objectRef = new WeakReference(objectRef);
      }

      #endregion

      #region Properties

      public WeakReference Publisher
      {
        get { return (this._objectRef.IsAlive == true) ? this._objectRef : null; }
        set { this._objectRef = value; }
      }

      public string DisplayName
      {
        get { return this._displayName; }
        set { this._displayName = value; }
      }

      #endregion
    }

    #endregion

    #region Inner class for EventSubscriberInfo

    public class EventSubscriberInfo
    {
      #region Variables

      private EventManagerEventInfo _eventInfo; // ref back to the holding container
      private WeakReference _objectRef;
      private MethodInfo _methodInfo; // The method that the event firer should Invoke
      private bool _isBackground;
      private EventManagerEventHandler _delegateForAsync;

      private string _topicName; // we may have wildcards
      private bool _hasWildcards; // to help determine whether to use RegEx or not

      #endregion

      #region Constructors

      private EventSubscriberInfo()
      {
        this._objectRef = null;
        this._isBackground = false;
        this._hasWildcards = false;
      }

      public EventSubscriberInfo(EventManagerEventInfo evInfo, string topicName) : this()
      {
        this._eventInfo = evInfo;
        this.TopicName = topicName; // use the property so that formatting is done
      }

      public EventSubscriberInfo(EventManagerEventInfo evInfo, string topicName, object objectRef) : this(evInfo, topicName)
      {
        this._objectRef = new WeakReference(objectRef);
      }

      public EventSubscriberInfo(EventManagerEventInfo evInfo, string topicName, object objectRef, bool isBackground, MethodInfo mi) : this(evInfo, topicName, objectRef)
      {
        this._isBackground = isBackground;
        this._methodInfo = mi;

        if(isBackground)
        {
          this._delegateForAsync = (EventManagerEventHandler) Delegate.CreateDelegate(typeof (EventManagerEventHandler), objectRef, mi.Name);
        }

      }

      #endregion

      #region Properties

      public object Subscriber
      {
        get { return (this._objectRef.IsAlive) ? this._objectRef.Target : null; }
      }

      public bool IsBackground
      {
        get { return this._isBackground; }
        set { this._isBackground = value; }
      }

      public string TopicName
      {
        get { return this._topicName; }
        set { this._topicName = FormatTopicName(value); }
      }

      public bool HasWildcards
      {
        get { return this._hasWildcards; }
      }

      #endregion

      #region Methods

      private string FormatTopicName(string topic)
      {
        // We want to make things easy for matching publishers and sunscribers.
        // 1) We always use upper-case topic names.
        // 2) Replace the dots with slashes so that the dot is not taken as a regexp
        //    character by the pattern matcher.
        // 3) Let us know whether this topic has a wildcard in it so we know whether
        //    to use the slow regexp matcher or the faster Equals operator.
        if(topic.IndexOfAny(new char[] {'*'}) >= 0)
          this._hasWildcards = true;

        return topic.Replace('.', '/').ToUpper();
      }

      #endregion

      #region Event Firing

      /// 

      /// This gets called whenever the event manager fires an event.
      /// This does the hard work in event firing. It eventually calls
      /// the subscriber's delegate in order to process the event. it also determines
      /// whether the delegate should be called synchronously or asynchronously.
      /// 
      /// 
      /// 
      /// 
      /// 
      // 
      public bool OnPublisherFired(object sender, string topicName, EventArgs args)
      {
        // If the object that this subscriber is bound to has been garbage-collected, then
        // remove the subscriber from the EventInfo's subscriber list and return.
        if(this.Subscriber == null)
        {
          this._eventInfo.RemoveSubscriber(this);
          return true;
        }

        if(this.IsBackground)
        {
          AsyncFire asyncFire = new AsyncFire(InvokeDelegate);
          asyncFire.BeginInvoke(this._delegateForAsync, new object[] {sender, topicName, args}, new AsyncCallback(Cleanup), null);
        }
        else
        {
          object oRet = this._methodInfo.Invoke(this.Subscriber, new object[] {sender, topicName, args});
          if(oRet is Boolean)
            return (bool) oRet;
        }

        return true;
      }

      private void InvokeDelegate(Delegate del, object[] args)
      {
        ISynchronizeInvoke synchronizer = del.Target as ISynchronizeInvoke;
        if(synchronizer != null) // Requires thread affinity
        {
          if(synchronizer.InvokeRequired)
          {
            synchronizer.Invoke(del, args);
            return;
          }
        }

        // Not requiring thread afinity or invoke is not required
        del.DynamicInvoke(args);
      }

      private void Cleanup(IAsyncResult asyncResult)
      {
        asyncResult.AsyncWaitHandle.Close();
      }

      #endregion
    }

    #endregion
  }
}

©2006 Marc Adler - All Rights Reserved

Event Manager Code - EventManagerArgs.cs

These are two minor argument classes that are used to send information from the event publisher to the subscriber. The args are sent as the third parameter in the EventManager.Fire() function.

using System;

namespace Magmasystems.EventManager
{
 /// 

 /// Summary description for EventManagerArgs.
 /// 
 public class EventManagerArgs : EventArgs
 {
  public EventManagerArgs()
  {
  }
 }

        // This is used to pass string data between the publisher and subscriber
 public class EventManagerMessageArgs : EventManagerArgs
 {
  private string _message;
  
  public EventManagerMessageArgs(string msg)
  {
   this._message = msg;
  }

  public string Message
  {
   get { return this._message; }
  }
 }
}

©2006 Marc Adler - All Rights Reserved

Event Manager Code - EventSubscriberAttribute.cs

The EventSubscriberAttribute is used to decorate a C# function that is the recipient of a subscribed event from our event manager. The topic name can be a wildcard string, just like Tibco. For instance, to subscribe to all actions that happen to a trade, you can subscribe to topic "TradingSystem.Trade.*".

Notice that the attribute is declared with AllowMultiple=true. This lets a function have multiple subscription declarations.

The final thing to notice is the IsBackground flag. If a subscriber is declared with IsBackground=true, then the event will be dispatched to the subscriber on a worker thread (internally, Begin/EndInvoke is used).

Here is the code:

using System;

// --------------------------------------------------------------------
//
// This code is (C) Copyright 2005 Marc Adler
//
// --------------------------------------------------------------------

namespace Magmasystems.EventManager
{
 /// 

 /// EventSubscriberAttribute
 /// 
 [AttributeUsage(AttributeTargets.Method, AllowMultiple=true)]
 public class EventSubscriberAttribute : System.Attribute
 {
  private string _topicName;
  private bool   _isBackground;
    
  public EventSubscriberAttribute()
  {
  }

  /// 

  /// EventSubscriberAttribute
  /// 
  /// The name of the event
  public EventSubscriberAttribute(string topicName)
  {
   this._topicName    = topicName;
   this._isBackground = false;
  }
    
  public string Topic
  {
   get
   {
    return this._topicName;
   }
   set
   {
    this._topicName = value;
   }
  }

  public bool IsBackground
  {
   get
   {
    return this._isBackground;
   }
   set
   {
    this._isBackground = value;
   }
  }
 }
}

©2006 Marc Adler - All Rights Reserved

Event Manager Code - EventPublisherAttribute.cs

The EventPublisherAttribute is a C# attribute that decorates a function that publishes an internal event. Any function that publishes an event out to our little event broker should decorate itself with this attribute.For example:

[EventPublisher("Brokerage.TradingSystem.Trade.ReceivedFromBackend")]
public void OnTradeReceived(Trade trade)
{
  .......
  EventManager.Fire(this, "Brokerage.TradingSystem.Trade.ReceivedFromBackend", 
                    tradeArgs);
}

Here is the code:

using System;
// --------------------------------------------------------------------
//
// This code is (C) Copyright 2005 Marc Adler
//
// --------------------------------------------------------------------

namespace Magmasystems.EventManager
{
 /// 

 /// EventPublisherAttribute
 /// 
 public class EventPublisherAttribute : Attribute
 {
  private string _topicName;
  
  public EventPublisherAttribute()
  {
  }

  /// 

  /// EventPublisherAttribute
  /// 
  /// The name of the event
  public EventPublisherAttribute(string topicName)
  {
   this._topicName = topicName;
  }
  
  public string Topic
  {
   get
   {
    return this._topicName;
   }
   set
   {
    this._topicName = value;
   }
  }
 }
}

©2006 Marc Adler - All Rights Reserved

A Lightweight IntraApp Event Broker for C#/.NET

Over the years, I have managed to write a repository of interesting classes that has helped me in architecting and developing systems. Over the coming months, and as time allows, I will submit some of these classes for your use.

I wrote the EventBroker about two years ago for a trading system that had Tibco connectivity to a message bus that published quotes. I wanted a symmetrical processing model from the back end to the front end, and also, within the front end. So, I wrote a .NET/C#-based lightweight event broker that was like a min-Tibco within the GUI.

The Microsoft CAB has something similar, and when it came out, I took a few ideas from it (such as the handling of WeakReferences) and integrated it into my own Event Broker. CAB differs from mine in several respects .... namely, it has the concept of Event Scope, and it also has a different way of decorating code (it decorates event declarations, while mine decorates methods).

However, this event broker has been used in several trading systems and seems to function well.

In the next several posts, I will post the code of the Event Broker. You are free to use it as long as my copyright is maintained and proper acknowledgement is given. Also, I am anxious to hear of any improvments, enhancements and bug fixes you might make to any of the classes that I publish here. You can always reach me at magmasystems at yahoo dot com.

The Event Broker

The application uses a lightweight Publish/Subscribe (PubSub) model internally in order to notify the controller or UI layer of interesting events. A typical interesting event is a change in the underlying data model.

The mechanism that implements PubSub within the application is called the Event Broker. Event publishers and event subscribers will be automatically registered with the Event Manager by the use of customized .NET attributes.

The use of the Event Manager in no way precludes the application from doing its own event management using the standard .NET event/delegate mechanism.

Using the EventBroker in a class

Any class that wants to use the Pub/Sub capabilities of the EventBroker should do two thing.

1) Include a reference to the EventBroker's assembly. Also, reference the namespace within your class:

using Magmasystems.TradingSystem.EventManager;

2) Register the class with the EventBroker. A good place to do this is in the class' constructor. To register the class with the Event Broker, all that needs to be done is to put in the following line of code:

EventBroker.Register(this);

There are several .NET attributes that can be used to decorate functions in order to define them as either Event Publishers or Event Subscribers. When one of these attributes is constructed, the EventTopic parameter is examined. The EventTopic is added to the EventManager's internal Dictionary. A Dictionary collection is used in order to provide fast access to a particular Topic/Event pairing.

[EventPublisher(string EventTopic)]

This registers a function or class to be an event publisher. The class will publish an event with a specific name. Any subscribers to that topic will be notified.

[EventSubscriber(string EventTopic, options)]

This registers a subscriber to an event. The attribute should be placed directly above the function that will be used as the event handler. The options can be one of the following:
* Background - the handler is called asynchronously

The string-based topic name can be the actual name of a published topic, or it can be a wildcarded topic like "Magmasystems.TradingSystem.Trade.*".

To fire an event, the static method EventManager.Fire() is called

EventManager.Fire(object obj, string eventTopic, EventArgs args);

Event handlers have the signature:

public delegate bool EventManagerEventHandler(object sender,
string topicName, EventArgs e);


Here is a piece of code that publishes an event (The recipient could be a controller that processes events from views and from the models):


[EventPublisher("Magmasystems.TradingSystem.Controller.Action.GetTrades")]
private void GetTrades()
{
GetTradeArgs args = new GetTradeArgs();
args.CUSIP = "12345";
EventBroker.Fire(this, "Barcap.Gcdit.Odc.Controller.Action.GetTrades", args);
}

Here is a piece of code from a controller that subscribes to an event


[EventSubscriber("Magmasystems.TradingSystem.Controller.Action.GetTrades")]
public void GetTrades(object sender, string topicName, System.EventArgs e)
{
GetTradesUnitOfWork.GetTradesArgs args = e as GetTradesUnitOfWork.GetTradesArgs;
if (e == null)
return;

ExecuteAction(new GetTradesUnitOfWork(this._context.UIApplicationContext, args));
}


There is a .Net event associated with every topic that is registered. Events are really Multicast Delegates. Every subscriber to an event adds an event handler to the list of delegates. When an event is fired, the delegate list is traversed and each delegate will be invoked either synchronously or asynchronously. The options can include the Background flag. This tells the event firing mechanism that the underlying delegate should be invoked asynchronously. If the Background flag is not present in the attribute declaration, then the underlying delegate will be invoked synchronously.


Guidelines for Usage

Any class can publish an event, and any class can subscribe to that event. The event bus is global to the entire application.

The line of code that registers a class with the EventBroker is best put in the constructor of the Base Class.

Following our adoption of a namespace convention, we have also adopted a general naming convention for events. This convention is:

companyname.department.application.domainobject.action




©2006 Marc Adler - All Rights Reserved

Note To Self - Traversing Rows in a Single Band

I am not really a hardcore Infragistics person. I find it amazing that some of the features that I consider essential in their UltraWEBGrid offering are not found in their UltraWINGrid product.

I am writing a prototype of an app that has multiple levels of grouping. I need to be able to double-click on the column header of a band, and have it do something to only the rows in that band. For instance, if the user double-clicks on the column that says 'Selected', then I want every checkbox in that group to be toggled without affecting the rows in any other group.



Sounds easy, right? Find out which header you double-clicked on, and then get the child rows that are associated with that header. The problem is that headers are not considered true rows within UltraWinGrid. They are "UI Elements". They do not have references to the rows that are directly beneath them.

After a while of messing around with the HeaderDoubleClicked event, I finally decided to base my procesing off of the MouseDoubleClick event.

Here is what I finally came up with after a few hours of tinkering around. (By the way, the function 'ToggleSelected' is a good way of walking down the entire tree of rows in a grouped grid.)

private UltraGridRow m_firstRowOfTheHeaderThatWasDoubleClickedOn = null;

private void dgTrades_MouseDoubleClick(object sender, MouseEventArgs e)
{
  Infragistics.Win.UIElement element =
  this.dgTrades.DisplayLayout.UIElement.ElementFromPoint(new Point(e.X, e.Y));
  if (element != null)
  {
    this.CalcGridRowForDblClick(element);
    if (element is Infragistics.Win.TextUIElement)
    {
      HeaderBase hdr = element.GetContext(typeof(HeaderBase)) as HeaderBase;
      if (hdr != null)
        this.dgTrades_DoubleClickHeader(hdr);
    }
  }
}


private void dgTrades_DoubleClickHeader(HeaderBase hdr)
{
  if (hdr.Column.Key == "Selected")
  {
    if (this.m_firstRowOfTheHeaderThatWasDoubleClickedOn != null)
      this.ToggleSelected(e.Header, 
                          this.m_firstRowOfTheHeaderThatWasDoubleClickedOn);
  }

  this.m_firstRowOfTheHeaderThatWasDoubleClickedOn = null;
}



private void ToggleSelected(HeaderBase header, UltraGridRow row)
{
  if (row == null)
    return;
  
  UltraGridBand band = header.Band;
  UltraGridRow childRow = row.GetChild(ChildRow.First, band);
  if (childRow == null)
  {
    // May be a leaf node
    UltraGridColumn colSelected = band.Columns["Selected"];
    if (colSelected == null)
      return;
    int idx = colSelected.Index;
    for (; row != null; row = row.GetSibling(SiblingRow.Next, false))
    {
      row.Cells[idx].Value = !((bool)row.Cells[idx].Value);
    }
  }
  else
  {
    // Is a non-leaf node
    this.ToggleSelected(header, childRow);
    this.ToggleSelected(header, row.GetSibling(SiblingRow.Next, false));
  }
}


private UltraGridRow CalcGridRowForDblClick(Infragistics.Win.UIElement uiElement)
{
  this.m_firstRowOfTheHeaderThatWasDoubleClickedOn = null;
  
  if (uiElement == null)
  return null;

  object elementContext = uiElement.GetContext(typeof(UltraGridRow));
  if (elementContext == null)
    return null;

  UltraGridRow row = elementContext as UltraGridRow;
  if (row != null)
    this.m_firstRowOfTheHeaderThatWasDoubleClickedOn = row;
  return row;
}

It works. The key was the UIElement.GetContext() function, which I finally found by reading a two sentence Knowledge Base article on the Infragistics website.

©2006 Marc Adler - All Rights Reserved

RIP: Doug Walker (Alien Planetscapes)

Doug Walker, long-time mainstay of the New York Spacerock scene and founder of the band Alien Planetscapes, passed away Tuesday April 4th at home. Doug had a heart condition for several years, and it finally caught up to him.

Doug was one of my oldest friends. I met Doug in 1975, and played in several seminal free-jazz bands with him in the mid to late 70's, including the great Third Sun.

Doug lost his wife Fran 2 years ago in a horrific car crash.

Doug leaves behind his son Evan, who was born three weeks after my own son.

God Bless You, Doug. Now you can jam with Robert Moog.





©2006 Marc Adler - All Rights Reserved

What is the World Coming To?

First I take a full-time job, then DonXML does!

http://donxml.com/allthingstechie/archive/2006/03/20/2609.aspx

Don and I are moving in parallel paths. We have both found small consulting companies that are staffed with people that we respect. We both feel, and been given the opportunities, to make a difference in our companies and to help grow the business.

From what I remember, Don does not like to cross the Hudson River, so I will leave him to the NJ Pharma companies while I go after the Financials!


©2006 Marc Adler - All Rights Reserved

Rules of Engagement

My Expectations from a Client

My first engagement with Finetix has just ended, and it was an eye-opener in certain respects.

I have been thinking about what I would like to include in a retrospective to the client in order to help them manage future projects successfully. Instead of enumerating what I consider to be the positives and negatives of the project, I would like to add some of the aspects to my master “Lessons Learned” list, and also to reiterate some of my long-standing project-management processes.

1) Clearly Define Roles
2) Always Be Engaged
3) Make Sure Everyone Knows Everything
4) Define A Methodology And Stick To It
5) Have The Environment Completely Set Up
6) Have Documentation Ready
7) Engage the Business Analysts and Business Users
8) Do Not Pair-Program Unless Absolutely Necessary
9) Always Have a Roadmap For Refactoring


Clearly Define Roles

If you are bringing in consultants to lead the technical team, then tell your team members and enforce the leadership. If you are expecting your consultants to be short-term body-shoppers who are coding, then tell them this up front. Do not allow incorrect perceptions to pervade the group.

There will always be bruised feelings when you bring consultants on to a project in a leadership role. We consultants completely understand this. Nobody wants their work to be reevaluated and critiqued. If the consultants are being brought in to do rearchitecture, then they must be given the power to lead the effort.

Make it clear as to who holds the trump card.


Always Be Engaged

As a project manager, make sure that you know what all of your team members are doing, and make sure that all team members know what each other is doing. Make sure that the server side and client side teams communicate. Many server-side devs have also done GUI work and vice-versa.

If you have to support a legacy version of your product, then make sure that you are able to delegate that task to certain members of the team. Have a plan to get these “legacy developers” involved with the new development. The legacy devs are subject matter experts who are important to the whole team.


Make Sure Everyone Knows Everything

As mentioned above, do not isolate the groups. Involve the Business Analysts and Business Users where appropriate. Unless there are well-defined isolation layers, make sure that you try to completely define the communication layers and formats between the client and server. If the BA’s are planning something new, have them run it by the entire team so that the team can discuss the ramifications on the architecture.


Define A Methodology And Stick To It

If you are going to do Agile or Scrum, then stick to it. If you are going to do Waterfall, then stick to it. If you are going to decide on daily standups, then stick to it.


Have The Environment Completely Set Up

Consultants get paid a lot of money, and their efforts are usually time-boxed. So, make sure that they are productive from day one. Have all user accounts set up. Have all of the software ready. If this software requires additional registration, then make sure that this is done. Make sure that there are enough licenses for the software, and make sure that support packages have been purchased.

Make sure that the computers have enough memory and enough free disk space. Developers need about 2gb of RAM and about 100gb of free hard drive space. Make sure that certain shared drives can be accessed.

Make sure that the security people in the building know that a team is starting, and to expect them. If a consultant is made to wait in the lobby while security tries to track down someone to sign them in, then this serves nobody.


Have Documentation Ready

Does the system have documentation? Use cases? Architecture docs? If not, then either prepare them or prepare for a steeper learning process.


Engage the Business Analysts and Business Users

I have talked about this before. If you are writing a trading system, then the developers should be able to have access to the traders. The developers should be able to peek over a trader’s shoulder while he is working. The developers should have a complete feel for the kind of system that a trader would like to have.

If there are BAs that the developers must interact with, then make sure that the devs and BAs have constant interaction. Make sure that the BAs know if some of their ideas will be difficult to implement.


Do Not Pair-Program Unless Absolutely Necessary

I continue to abhor pair programming. I think that it cuts down productivity in a big way. I think that the best development is done when the developer is “in the zone”, and does not have to expend energy explaining every move to someone else.

Instead, have design meetings and developer meetings. It does not have to be “Big Design Up Front”. However, if a developer is architecting a crucial module, then let him present the design to the entire team and get critiques before the effort is too far gone.


Always Have a Roadmap For Refactoring

Do you know why a product is being refactored? Does the business absolutely need it? Do you really need to rewrite an entire application from scratch?

Remember that, when you refactor an application, you need a plan to retain all of the business logic that is built in to the existing app.

Make sure that the code has comments. Make sure that there is plenty of technical documentation. Make sure that all of the business rules are documented.

Make sure that all dead code has been eliminated from the application. It is difficult to rearchitect a system if the hands of 30 different people have been in it over the years.


These are my ideal Rules of Engagement for any project that I will be on, either as a leader or as a hired hand. I would love to hear your comments.


©2006 Marc Adler - All Rights Reserved

Transition #1

My last day in the land of the Freeborn is today. Good exposure to Credit Derivs.

Starting Monday at the home of the Big Mack. Enhancing an FX system.

©2006 Marc Adler - All Rights Reserved

Garden Leave

I rode on the train this morning with a neighbor who is a Managing Director at a Wall Street firm. He is switching companies, and just handed in his resignation. However, he does not start his new position for another 90 days.

It seems that most Wall Street companies are imposing a minimum number of days that you have to stay with a firm when you give your notice. In his case, he had to give 90 days notice. However, his current employer must pay him for the entire 90 days.

His current employer wants him out of the office as fast as possible, so that he does not have the chance to do any internal recruiting. So, he gets to sit at home for 3 months and collect a paycheck.

From what he told me, this practice has been prevalent in Europe, and is just starting to make the rounds in the United States. He told me that it is referred to as "Garden Leave", because the only thing you can do for 3 months is putt around in your garden.

©2006 Marc Adler - All Rights Reserved

"On Becoming a Quant" by Mark Joshi

This was an interesting post that I found on a financial forum. For all of you who have harbored the fantasy of becomign a quant, here is a treatise on what it is like in the real world. Thanks to Mark Joshi for writing this.


ON BECOMING A QUANT
MARK JOSHI

What does a quant do?

A quant designs and implements mathematical models for the pricing
of derivatives.

What sorts of quants are there?

(1) Front office/desk quant
(2) Model validating quant
(3) Research quant
(4) Quant developer

A desk quant implements pricing models directly used by traders.
Main plusses close to the money and opportunities to move into trading.
Minuses can be stressful and depending on the outfit may not involve
much research.

A model validation quant independently implements pricing models
in order to check that front office models are correct. Plusses more
relaxed, less stressful. Minusses model validation teams can be uninspired
and far from the money.

Research quant tries to invent new pricing approaches and sometimes
carries out blue-sky research. Plusses it’s interesting and you learn a
lot more. Minusses sometimes hard to justify your existence.
Quant developer – a glorified programmer but well-paid and easy to
find a job.

All forms of quants spend a large amount (i.e. more than half) their
time programming. However, implementing new models is interesting
in itself. The standard programming approach is object-oriented C++.

A wannabe quant must learn C++.

In the UK standard sources for job adverts are www.jobserve.com
search under ”quant”, the FT appointments section on a Thursday,
and www.wilmott.com has a jobs board. A lot of adverts are from
recruitment consultants rather than from banks. It’s important to
realize that the job may not even exist – the consultant wants to get
decent candidates that he can then try to place them in banks. The
consultant gets a commission from the bank if he can place you. They
tend to have short attention spans. If you do well at the first couple of
interviews then they will work hard to get you a good job but if you
don’t they will quickly lose interest. Also, be aware their agenda is to
get a good commission rather than to help you so they will push you
at jobs on that basis.

In fact, going via a recruitment consultant is the standard way to
get a job. Quants are generally not hired as a part of the on campus
recruitment process but instead hired as they are needed by the team.
Because of this it’s not a great idea to start applying a long time before
you want to start. Obviously personal contacts should be exploited
as much as possible. Banks tend not to be into paying expenses for
interviews. One therefore needs to go to London or New York and
attempt to get as many interviews as possible as quickly as possible.
What should one learn? Standard books are

• Hull - Options future and other derivatives – comprehensive
but low level mathematically and can be frustrating for pure
mathematicians

• Baxter and Rennie – accessible introduction to martingale approach
but oriented towards theory rather than practicalitues

• Wilmott (Derivatives) – good on the PDE approach but not so
good on other approaches.

My book will be published in early 2003 and I like it, of course.
Stochastic calculus is useful but not as important as it at first appears.
Standard texts are Oksendal, and Karatzas and Shreve. It’s
hard to find the time to pick it up on the job so it’s worth learning in
advance. It’s also worth spending some time going over basic probability
theory – eg Chung’s books.

Interviewers tend to care more about understanding the basics well
than on knowing a lot. It’s also important to demonstrate genuine
interest in the field. Read the Economist and the FT or Wall Street
Journal comprehensively. It’s not unusual to ask basic calculus or analysis
questions e.g. what is the integral of log x. Asking for a derivation
of the Black-Scholes equation is very common too. They always ask
you to explain your thesis so be prepared to be able to do this.

Generally, a PhD (or almost a PhD) is a necessity to get a quant
job. I would advise against starting before it’s awarded as it tends to
be hard to get it done whilst doing a busy job.

The main challenge for a pure mathematician is to be able to get
one’s hands dirty and learning to be more focussed on getting numeric
results than on fancy theories. The main way to do this is to implement
pricing models for practice. If this doesn’t appeal you aren’t suited to
being a quant. There are quite a few ex-pure mathematicians working
in the city so it can certainly be done but there is some prejudice
towards applied maths and physics people.

How much does a quant earn? A quant with no experience will
generally get between 35 and 50k pounds. This will generally go up
fairly rapidly. Bonuses are generally a large component of total salary.
How hard does a quant work? This varies a lot. At RBS we get in
between 8.30 and 9 and go home around 6pm. The pressure varies.
Some of the American banks expect much longer hours. Wall St tends
to be more demanding than the City. In London 5 to 6 weeks holidays
is standard. In the US 2 to 3 is standard.

©2006 Marc Adler - All Rights Reserved

The Wall Street Stack Revisited

Chris pointed me to an article on the MSDN/Smart Client page. A startup company called IdeaBlade is offering much of the business object and form-binding backbone that all of us have developed in the past.

I would be interested to hear your opinions of it.

©2006 Marc Adler - All Rights Reserved

Lepus Reports

Allow me to give a shout out to Lepus Reports. I just discovered an archive of past Lepus Reports on my current client's Intranet, and have been devouring each issue with relish.

They release two reports per month; a Research Report that contains about 10 articles dealing with a specific area of financial IT, and a briefer Management Report. For instance, one article might give an overview of ECNs, talk about the current trends in ECN usage, and give a brief overview of the various vendors in that space. They will also survey a number of financial firms to find out what they are currently using.

The last few pages of each report are devoted to news, so you can find out about new products, who has changed companies, and who has bought out whom.

If I were a financial IT consultant who changed clients every few months (which I am) and needed to get up to speed on a certain domain, I would certainly reach for a Lepus Report to get a quick backgrounder.

Note: I have no connection with Lepus, other than being a satisfied reader.

©2006 Marc Adler - All Rights Reserved

Mooney N9078V

I am flying the Mooney again!

©2006 Marc Adler - All Rights Reserved

A Data Point - Senior .NET Arch/Dev Salary

Even though my resume is not in the "active" state on Monster and Dice, I still get a ton of calls from headhunters. Since coming out of my New Jersey territory, and working in New York, I now pay attention to what Wall Street dev managers, architects, and developers are making. After all, when we try to recruit candidates for Finetix, we need to be a little more than competitve with the current batch of companies.

Perhaps everyone have just gotten their new budgets approved ... perhaps it is all of the annual February job movement taking place, and new hires want to re-stock their technical teams .... but this week has been ridiculous in the number of calls and emails that I have been getting for Wall Street firms looking for solid technical talent. The same thing must be happening over in the City of London.

On Wall Street, it seems that the base salary for a senior .NET and Java architect/developer is $150,000. This must mean that the salary for a City of London position is around £100,000. (Can any London-based developer confirm this?)

Bonus is everything on Wall Street. It was shocking to hear that the bonuses for average developers at my current client around around 10%. Lehman's bonus are much, much higher, but you have to work every Saturday. UBS is around 40%.

I would love to get more data points. Please feel free to comment.



©2006 Marc Adler - All Rights Reserved