Big Data Services

Business decision making at all levels within an organization is becoming increasingly data driven. 90% of electronic data in existence today was created within the last two years, and the amount of new data generated in a single day is staggering. The era of data creation is over; the task at hand is the ability to turn that data into actionable information.

Bates Group provides cutting-edge analytic services in support of data analysis, synthetization, and presentation. We are driven by data analytics, and have been since our founding. Our strengths lie in our ability to identify relevant data, clean and standardize the needed fields, analyze the data available, and distill the salient information present within the data into easy-to-understand takeaways.

Our practitioners are well seasoned in analyzing both structured and unstructured data sets, and are very familiar with current best practices in the field of data science.

Litigation and Investigations

In today’s data-driven world, the ability to manage and analyze large, complex sets of data is critical to the success of litigation matters and internal and regulatory investigations. Bates Group is highly experienced in providing consulting services involving large and complex datasets, some containing billions of records.

Utilizing state-of-the-art technology, we have developed an efficient and cost-effective approach to assist in navigating the discovery process, performing data diagnostics, and conducting statistical and empirical analysis. Through our engagements, we have collected and aggregated data from multiple sources within our clients’ businesses, and, in doing so, have gained the experience to help minimize the efforts required from our clients’ data personnel and to ensure data integrity throughout the process.

Bates Group is also equipped with proprietary software to provide an efficient in-house solution for potential data entry exercises. We understands the importance of protecting client data and take steps to ensure that all data at Bates is protected and safe from unauthorized access. Bates Group vigilantly updates its systems to ensure they are secured against vulnerability, based on the latest computer security industry protocols and announcements. We also undertake regular SOC 2 audits to ensure the proper controls around security and confidentiality. For more on our data security measures, please give us a call or visit our Data Security page. For examples of work we have performed in support of litigation or investigation matters, scroll to the bottom of this page.

Capital Markets Data Surveillance: Abnormal Performance and Market Abuse Data Detection Services

Regulators are dedicated to using large scale market activity data to monitor financial institutions and to bring enforcement actions against them. Against that backdrop, it is crucial that firms understand how a regulator might view their data in case they are targeted for an investigation.

Bates Group has a substantial, proven track record assisting clients in two key areas: abnormal performance indicators and market abuse detection. Both areas are key concerns of regulators and areas in which they are using machine learning and other data analysis techniques to detect illegal behavior (see here for abnormal performance, beginning at slide 13; see here for market abuse and the Consolidated Audit Trail).

Our data-driven detection services can be used to screen for abnormal performance in a fund, manager, or asset’s prior returns. This service is invaluable for financial institutions, family offices, or even individual investors in uncovering and avoiding fraudulent schemes. We evaluate the quality of the reported prior performance from multiple angles, looking for indications that the reported results may not be accurate. Our in-depth understanding and experience in financial markets then allows us to examine each flag or exception generated by our data-driven review, discounting flags and exceptions that may have a reasonable explanation, and thoroughly investigating those that do not.

Bates Group has also developed a number of proprietary programs and platforms for identifying instances of market abuse in both equity and futures markets. Our analysis can be used to identify spoofing, layering, iceberging, momentum ignition, wash sales, painting the tape, insider trading, cherry picking, and pump and dump behavior. Our data-driven capabilities are augmented by our deep experience in trading and trading platforms, allowing us to thoroughly investigate each instance of potentially illegal behavior and to provide concrete guidance back to our clients on whether or not such activity has taken place.

For example reporting from our work in this space, please contact us.

Independent Data Validation Services

Corporations are increasingly reliant on analysis conducted in a big data environment to drive key decision making, often as a factor in determining the positioning and strategy of the business itself. Frequently the work performed is done with a high degree of care, but it is inevitable that errors will occur in any analysis environment. We have found this to be especially true in instances where the client is utilizing a programming language in which only a few employees are fluent, or when the same team performing the analysis also reviews their own results. Unfortunately, this is a fact of life for even the largest organizations, as they are often building and growing their internal data science teams and may not have standard practices or institutional knowledge developed to conduct a rigorous independent review process.

In these instances, Bates can be an invaluable resource. We have assisted clients by reviewing work that has been performed internally, or by a third-party, replicating the results and verifying the conclusions. Before making a major decision that could impact the future of the business, it pays to ensure that the analysis driving that decision is error free. Bates Group provides its corporate clients with that assurance by validating the work performed by others.


Examples of Bates Groupā€˜s Capabilities

Big Data Services

Our staff has experience that covers a wide variety of disciplines and includes litigation matters and regulatory inquiries:

We have performed an internal investigation on behalf of a large financial institution into the trading activity across the entire trading platform of a financial instrument. The dataset spanned multiple years, included over 6,500 FAs and covered tens of thousands of clients (all equating to 25+ million transactions). During the course of the investigation, we partnered with counsel and utilized our industry expertise to determine the appropriate metrics and presentations that allowed them to reach the necessary conclusions. We also played an instrumental role working with both the IT and the product (at issue) branch of the firm to extract the necessary data; which included performing diagnostics on the data to confirm accuracy/completeness. Our product and services were extremely influential as the investigation transitioned into responding to the regulatory inquiries.


We have analyzed real estate investment trust (REIT) transactions covering a five-year period of time to determine whether the transactions met various suitability guidelines. In performing our analysis on millions of records we created testing protocols that allowed us to determine which, if any, of these transactions violated the individual requirements of the suitability guidelines. The results of our analyses were provided to the broker-dealer and state regulators.


Our staff has performed complex regression analysis to investigate the impact of trades and orders on stock prices. Prior to performing such analyses, we collected and standardized millions of records of orders and activity data from the client. Data were processed in a cost-efficient manner such that any additional analysis for a particular stock was automated.


We have researched and analyzed various open-end fund complexes to deduce investment strategy, trading patterns, alleged mispricing, and expense to shareholders. Our staff reviewed a dataset of portfolio trades by the portfolio managers as well as trades by investors.


We also have experience performing a microeconomic study that utilized billions of records including trade and order data for all companies listed on an exchange. Data were also merged with market maker quote data to conduct a comprehensive study on how market makers reacted to investor activity.

Contact Bates Group

Bates Group is with you every step of the way. Contact us today for more information on how our End-to-End Solutions can help your firm.

Contact Bates Group

Automat:ee