My Battle With The Market

Quick Recollection:

When I ran Americas Data Collection Development for Reuters I had a run-in with the feed which is the grand-daddy of them all throughput-wise. That would be OPRA. OPRA aims to soon have the capacity to send one million messages per second. This is a phenomenal amount of data, all of which relates to options pricing.

Back up to Feb 2007, the market rates began accelerating. Reg NMS, the big regulatory change in the U.S., had gone live and market participants were beginning to comply. Then the Chinese Market had a large sell off, and the Dow Industrials were off 416 points.

In the options market, record message rates were being processed. Our systems handled it rather well, considering. However, we did hit a snag. OPRA is broken into 24 data lines, one for each letter (almost). At that time, the splits were pure. Since then, the split is less formal, and can get rebalanced.

However, on February 27, prior to rebalancing, the "W" line, which had the highest peaks, was going nuts. Our OPRA W line handler crashed suddenly near the end of the day. My OPRA developer began to look into the problem and did the usual replay of the data after the market closed.`

After a bit, he came into my office, excited. He had it nailed.

"Jeff, I figured out the reason for the crash of OPRA W...Annnd its going to crash tomorrow too...Annnnnd I'll know ahead of time when it will happen.

This certainly got my interest. What could this have been to make him so confident so fast? Its never this easy.

He explained that this was the first time OPRA had reached a 9-digit sequence number. The W line was so overloaded, it reached 9 digits. Think about that - one hundred million messages on a single line in one day. Sometime soon, we'll probably laugh at "such a low rate".

Anyhow, the 9-digit number causes OPRA to send a sequence reset message. This was the first time this had been sent. And the sequence reset message did not match their specification.

Our Business Analyst immediately got on the phone with OPRA, bringing their problem to their attention. We waited, and prepared for tomorrow, so that our operations team would be aware of the issue and have a plan in place.

In the mean time, OPRA got back to us. They decided to issue a new specification, rather than fix the problem. Exchanges behaving badly, I called it. Handling that was my job. Any wonder why I am a consultant now?