Tuesday, November 29, 2011

I am going to be in Sao Paolo for Two Weeks

The market in Brazil seems to be exploding, and I am going to be in Sao Paolo from December 4 to December 16  to kick off work for a major new client.

If there are any Brazilian (or ex-pats living in Brazil) that might be interested in a coffee, please feel free to contact me.

©2011 Marc Adler - All Rights Reserved. All opinions here are personal, and have no relation to my employer.

Saturday, November 26, 2011

HFT Trading Company Fined $850K

(Thanks to John Bates for the pointer)

According to this article, Infinium Capital Management was fined $850,000 for three separate incidents related to High Frequency Trading. It seems that they lost control of their algos at various times over the past 2 years.

I was thinking about this article in relation to a friend of mine who was interviewing at various HFT firms over the past few months (no, it's not me, and yes, he found a great job). He was asked every single little question about C++ minutiae, questions about manhole covers, questions about networking trivia, and questions about the timing of instructions. Maybe valuable stuff for a HFT developer to know. But not once did these interviews concentrate on testing and quality assurance and development practices.

This is probably the first step by the exchanges into monitoring their HFT partners. As technology improves, and as it becomes easier to recognizes errant algos (probably, in part, to the same kind of monitoring that Nanex uses), you will see these kinds of fines become more commonplace. It will also become necessary for HFT firms to come up with better simulation scenarios for their algo testing.

It would be interesting to me if Infinium hired Nanex to do a post-mortem on these three situations so that Infinium can start fixing their algos. Maybe this could grow into a sideline business for a Nanex-like startup, where other HFT firms could outsource the post-mortem and QA work. At the very least, the exchanges would be able to outsource this kind of real-time surveillance work.

©2011 Marc Adler - All Rights Reserved. All opinions here are personal, and have no relation to my employer.

Friday, November 18, 2011

There seems to be a shortage of .NET Developers


As an aside, my company is looking for a few great .NET developers who can work at one of our hedge fund clients, so if you are really good and you are available, let me know. The skills required are WPF, WF, WCF, and Entity Framework.

©2011 Marc Adler - All Rights Reserved. All opinions here are personal, and have no relation to my employer.

Wednesday, November 16, 2011

Microsoft Droping Dryad and Concentrating on Hadoop

Read about it here.

From what I have seen from my new vantage point in the consulting world, there are a number of Wall Street companies who are making investments into the Hadoop infrastructure. Good move by Microsoft.

©2011 Marc Adler - All Rights Reserved. All opinions here are personal, and have no relation to my employer.

Wednesday, November 09, 2011

Time Series Foundation released by Microsoft Research

Time Series Foundation

Time Series Foundation (TSF) is an open, .NET platform for exploring and prototyping new algorithms in time series analysis and forecasting. TSF is based on state space model methodology that includes all types of exponential smoothing, some autoregressive algorithms, and innovative algorithms for event detection and calendar event impact prediction.

TSF relies on Excel charting and presentation APIs by implementing an Excel interop layer. Numerical and graphical results of time series analysis and forecasting can be put in programmatically generated workbooks with the help of this layer. TSF also offers an Excel add-in that exposes a large subset of the platform's functionality through the Excel ribbon UI.

©2011 Marc Adler - All Rights Reserved. All opinions here are personal, and have no relation to my employer.

Saturday, November 05, 2011

Caching for .NET Apps

The goal ... find a simple distributed cache that can be used by a .NET enterprise app, supports the storage of large objects, and will result in no additional cost to the client. We don't care about cache notifications, and it's OK (but not desirable) to incur the cost of serialization from native .NET objects to XML strings.

Memcached for Windows is here. I used the Linux version at Citadel for our new multi-asset trading platform, and it was fairly easy to use, except it doesn't support the enumeration of cached objects.

A management console GUI for Memcached is here.

It has been over 2 years since I considered Microsoft Velocity (see past blog entries here). However, it seems that Microsoft's Velocity Cache has been put into the Azure AppFabric, and the client is not considering the cloud at this time. Does anyone know if you can use Velocity (and the new Azure Service Bus) independent of Azure?

It seems that the new System.Runtime.Caching namespace in .NET 4 doesn't support distributed caching.

There is a project on Codeplex called SharedCache, but it looks like it has not been updated in a while. In addition, some of the comments seem to indicate that it hogs the CPU.

NCache has a free version.

Has anyone used any other solution? So far, it looks like NCache or Memcached is the way to go.

©2011 Marc Adler - All Rights Reserved. All opinions here are personal, and have no relation to my employer.

Friday, November 04, 2011

Installing RabbitMQ on Your PC

For the past 7 years, I have been running an old copy of Tibco EMS on my laptop, and I've been using this whenever I need to do some prototyping and development of new message-based apps. But in these cost-conscious times, many of my clients are looking for low-cost solutions. So, time to seriously consider RabbitMQ.

This is a quick guide that I wrote to get RabbitMQ up and running on a local PC in preparation for some .NET development.

Download the RabbitMQ Bits

1) RabbitMQ requires the Erlang runtime. You can download the Erlang/OTP runtime from http://www.erlang.org/download.html. You need to install Erlang before downloading and installing the RabbitMQ server.

2) Download rabbitmq-server-2.6.1.exe for Windows from the RabbitMQ site, http://www.rabbitmq.com/download.html.

2a) Install RabbitMQ. It will install a Windows Service and will automatically start it.

2b) From the Windows Start Menu, run services.msc and verify that the RabbitMQ service has started.

3) All of the .NET download for RabbitMQ are located at http://www.rabbitmq.com/dotnet.html

3a) Download the .NET 3.0 version of the RabbitMQ client libraries and samples. The file is called rabbitmq-dotnet-client-2.6.1-dotnet-3.0.zip

3b) Extract the ZIP file to C:\Program Files\RabbitMQDotNetClient (you can pick any location that you want).

3c) In C:\Program Files\RabbitMQDotNetClient\bin, there is the main assembly called RabbitMQ.Client.dll. In the same directory is the assembly for WCF integration called RabbitMQ.ServiceModel.dll.

4) From the Windows Start Menu, go to the RabbitMQ Server item, and then run RabbitMQ Command Prompt

4a) rabbitmqctl.bat is the command-line utility that lets you control RabbitMQ and list various objects. A sample command to list the RabbitMQ exchanges is
 rabbitmqctl list_exchanges
However, we will use the management console, not the command line.

5) Download and install the RabbitMQ management plugins from http://www.rabbitmq.com/plugins.html

5a) Download all of the EZ files associated with rabbitmq_management_visualiser

5b) Drop these files into the RabbitMQ Server\plugins directory, which on my system is C:\Program Files (x86)\RabbitMQ Server\rabbitmq_server-2.6.1\plugins

5c) Restart the RabbitMQ server. You can do this through the Services.msc applet on your Windows machine.

Test It Out Using the Management Console

Now we will test out RabbitMQ by sending a message into an exchange and have a wildcard subscriber read that message.

6) As a test of the RabbitMQ management console, try this URL in your browser:

Use the user id "guest" and password "guest"

7) Go to the Exchange tab and add a new exchange called CalcExchange that is non-durable (transient) topic. Fill in the name, type and durability as follows:
Name: CalcExchange
Type: topic
Durability: Transient

8) Go to the Queues tab and add a new durable queue called Calc.Queue.1

9) Stay in the Queues tab. In the table of queues, click on Calc.Queue.1

9a) Add a binding. In the Exchange and Routing Key fields, add the following:
Exchange: CalcExchange
Routing Key: Calc.Data.*.1

This binds any messages that have the routing key Calc.Data.*.1 to the Calc.Queue.1 message queue. So, if you publish a message with the key Calc.Data.Foobaz.1, it will be routed to this queue, where the subscriber will pick up the message for processing.

Now we will try to send a sample message using the management console.

10) Go back to the Exchanges tab and click on the CalcExchange item

10a) Go down to the Publish Message section.

10b) Type in the following entries for the Routing Key and Payload fields:
Routing Key: Calc.Data.ABC123.1,
Payload: This is a test message for Calc Node 1

10c) Press the Publish button. When the status message pops up, just close it. The status message should have a green background, indicating success.

11) Go back to the Queues tab, and click on Calc.Queue.1

11a) Go down to the Get Messages section and press the "Get Message(s)" button. You should see the message that you just sent.

12) You can also go to the last tab of the management page, which should be the Visualizer tab, and view the topology.

As an aside, the sample programs that come with the .NET Client need a bunch of mods before you can get them to load into Visual Studio 2010.

©2011 Marc Adler - All Rights Reserved. All opinions here are personal, and have no relation to my employer.