Sunday, November 13, 2005

Full-Layer vs Bottom-Up Reachitecture and Scrum

To reengineer a credit derivates trading system using Scrum. (30-day sprints)

The existing application is somewhat monolithic, with direct database access from the UI layer, and business rules embedded in the UI.

Things to consider
1) The users want to see visible functionality after each sprint. We need to rearchitect, but our refactoring must not affect the current UI.
2) We need to implement new structured products after the 3rd sprint.

Would you rearchitect from the bottom up and devote each sprint to deliver a fully-tested foundation layer, working your way up until you hit the UI layer? Or, would each sprint consist of implementing the entire stack with ever-increasing functionality?

I would love to hear your opinions.

©2005 Marc Adler - All Rights Reserved


Len Holgate said...

It depends ;)

I expect it would be a bit of a mix of both. What I tend to do is try and locate the areas where you get the most value and target those first. That might mean that you refactor within a layer in one phase and across layers in later phases. Likewise it depends on time constraints and how much visible change you want/need to show your users during each sprint.

The users may not be able to see visible changes after each sprint. It's best to tell them that up front. Sometimes they will, sometimes you may have to go for several sprints before they actually see anything but then they should see more... I find it best to be clear about this kind of thing from the start.

So, for example, I might decide that the first course of action is to decouple the UI from the database code. That might require building a new data access library/layer and moving code from the UI to the new layer and allowing the code to communicate via an interface. You might not be able to do ALL of this in one go, so you might aim to get it so that you just do the stuff around the areas that will require new functionality to be developed in the 4th sprint, etc.

I wrote about my experiences refactoring an FX trading system here

Good luck!

marc said...

Thanks, Len. I have sent a link to your article to all of the folks in my group. A good read!

Anonymous said...

Get your customers used to Sprints. Give them small initial feature releases and focus on the infrastructure.

Slowly, change from infrastructure focused to user focused over 2 to 3 sprints.

John S Nolan said...

I'd go for the single feature all the way through approach. In fact why not implement the new structured products first?

The state of the infrastructure adds no value - it may incur cost - but it has no value to the end user. Surely the priority for them is the new structured products?

I am slightly confused how you would show visible value without changing the UI though? If there is no visible difference, what are the measures? Performance?

marc said...

> I am slightly confused how you would show visible value without changing the UI though? If there is no visible difference, what are the measures? Performance? <

1) Quality - lots of bugs in the current release

2) Performance - a bit, but the performance is OK right now.

3) Real Reason - the business is paying for the new functionality. There will be an interim release that will contain the refactored code, but the business has to be 100% hidden from the refactoring effort. So we have been asked to maintain 100% backward pixel compatibility with the current release.

Len Holgate said...

I'm not so sure that it's actually a good idea to hide the refactoring from the business. I know why you might want to do that (the business doesn't want to pay for the refactoring). But without the refactoring the new functionality may not even be possible... We didn't own up to refactoring phases at the start but we changed our ideas quite quickly and the business were happy as long as we still managed to deliver the new functionality in a timely manner. In fact it was much easier once we told the business that some of our releases would contain no new functionality but were simply working towards making the functionality available. We wanted them live so that we could catch regressions quicker (the team had no formal testing in place at the start and each release was plagued by regressions). Once the business realised that quality was going up and so was speed of delivery they didn't care that we were refactoring as well as providing new functionality. Then again we were doing two releases per week at one point...