Art of Analytics: Deep Sea Monster

The best minds from Teradata, our partners, and customers blog about relevant topics and features.
Teradata Employee



The Insights

Fraud costs organizations, including government agencies, billions of dollars each year. With fraud becoming increasingly sophisticated and taking many forms, companies need new ways to identify and mitigate the threat.


'Deep Sea Monster' looks like a mystical creature, but in reality it is a visual representation of potentially fraudulent government procurement networks in Russia. Today, government procurement data in Russia is open and can be easily accessed by anyone. The bidding process for procuring goods and services is designed to be open and transparent. As part of the process, each step and procurement action taken by the government and contractors leaves a digital footprint.


'Deep Sea Monster' offers a detailed view of business relationships between contractors and government agencies. Each node represents either a contractor or a government body. The size of the node is based on the total amount of federal money received by the contractor. Each line represents a government contract.


Initially, identifying an umbrella-like relationships between 'elite' group of contractors and government organizations was difficult, which is why only companies potentially affiliated with fraud were chosen for the analysis. The hypothesis that contractors were trying to hide their identities by creating dummy entities and were actually involved in suspicious activities proved to be true. A short list, prioritized by revenue, of these companies and their dummy aliases could then be created for investigations.


The Analytics

Teradata Aster was used to perform the analytics on open government financial data. We worked with detailed contract data from 2015 from one of the central regions of the Russian Federation. A string of company data such as their full name and address was leveraged to identify potentially affiliated companies using a Jaccard distance algorithm. The Jaro-Winkler distance was used for the same purpose for separate words, including first names, last names and other single word entities. Sigma chart in Teradata AppCenter was used to visualize the results.


The Benefits

The Accounts Chamber of the Russian Federation estimated that nearly $2 billion (in U.S. dollars) in contracts in 2015 were connected with fraud. This shows that in spite of open tender procedures being a requirement in the several hundred thousand public organizations in Russia, fraud is still a major problem.


The Teradata team won the People's Choice Award on Russian Open Government Financial Data Contest BudgetApps 2016 by demonstrating a number of analytic tools and techniques for extracting value from government data. The C-level management of the Ministry of Finance of the Russian Federation emphasized that these tools should be used for government portals.


The analytics can be used by the public as an instrument of social control, by procurement process regulators to fight fraud, by internal control departments of government organizations to prevent corruption, and by government audit institutions to estimate and act on unresolved problems. The outcome of the analytics leads to a smarter government that serves its people by using public resources in the most efficient way.