Google Bigtable in Financial Markets

This week Google entered the financial software market. Capitalizing on its deep experience with ”Big Data,” the company will provide SunGard Data Systems with new software that will help to process over 6 billion trades an hour. SunGard will use Google's cloud database service (Bigtable) to maintain records of every order, trade and quote captured from the market, and will use business intelligence software designed to query information from that database environment.

The initiative comes from SunGard's work in the Consolidated Audit Trail (CAT) space. SEC Rule 613 requires "... a comprehensive consolidated audit trail that would allow regulators to efficiently and accurately track all activity throughout the U.S. markets in National Market System (NMS) securities." NMS securities are mainly exchange-traded equities and standardized options contracts, which represents a substantial amount of the everyday market. What this means in practical terms is that everything related to each "...quote and order, such as origination, modification, cancellation, routing, and execution" must be captured. The idea is to give regulators a way to track market events (and meltdowns like the Flash Crash of 2010) in real time. Using this database, regulators could reconstitute the market at any point in time, and also conduct surveillance on market activity with a brief turnaround time. Using data that captures the state of the market at the time of a disruptive event is front and center in the CFTC and DOJ's current cases against a UK trader. For regulators, the applications of this platform for ongoing monitoring and enforcement are quite extensive.

SunGard is currently one of six bidders to build and control CAT for regulators. The cloud database environment that Google is providing, in conjunction with SunGard's own systems, helps to sidestep a key issue in this project. Fixed-hardware solutions are inefficient for this type of work, wherein a calm day could produce only 30 billion trade events, but a volatile day could produce nearly four times that volume. The scalability of the system will allow for both fast and slow days while maintaining efficient use of resources. On Wednesday, SunGard announced it had completed two key tests of its new system. Emulating a sudden spike in market activity, SunGard processed 2.5 billion trade records in 15 minutes, and, in a sustained market test, SunGard processed six billion trade events in a single hour.

For Google, the Beta release of its Bigtable system may now be attractive to an entirely new industry. The company published SunGard's results on their own website, which emphasizes the cost savings that using Bigtable could bring to exchanges and other market participants who will be required to fund the CAT program.