Data scientists can easily find and access the data they need when they need it — regardless of its format or location.
Many large companies have vast amounts of data flowing around their organization, but they cannot detail what they have, where it is, what it contains, and how it relates to other datasets in the company. This is the information a data scientist needs. Without it, progress will be slow.
As the central clearinghouse for all data in your ecosystem, Ab Initio’s data cataloging technology provides a single portal to all your enterprise data. Ab Initio accelerates the discovery, cleansing, integration, governance, and enrichment of datasets across your organization. As new data and metadata are generated, Ab Initio makes that information available throughout the enterprise. With Ab Initio, you can quickly identify data that is trusted and appropriate for use, significantly decreasing the time required for data analytics and model training. You can easily perform semantic searches over metadata to find the most relevant datasets and then browse and filter the results.
When the best case is that you can run your audit rules only once a year, it’s time to find a better case.
Waiting six months to determine if you have a discrepancy in your data is a problem, particularly when regulators come to visit.
For one major financial firm, a six-month delay would have been wonderful — they faced twelve-month delays for their most critical reports, and less critical reports might run only once every two years. Their systems could not support running their audit rules more frequently than once every 12 to 24 months. They didn’t have the capacity to access the data they needed in a timely manner.
Developing new audit rules and reports was a very slow, time-consuming process. To develop new rules, a business expert would specify them and then a developer would code them. If the implementation had problems — for example, if the expert didn’t clearly specify the rule or the developer misunderstood the specification — debugging was slow and torturous. By the time a business expert determined that the rule was incorrect, the developer was already assigned to another task. As a result, it typically took six months to create and debug new rules and reports.
Further complicating the process, auditors needed to be able to review the rules. Since the auditors couldn’t read the code, the financial institution had to maintain detailed documentation for each rule. Because they had thousands of rules, this task became very large and prone to error. Of course, errors in the documentation then led to more scrutiny from the auditors, creating a vicious circle. This was not a good situation.
Fortunately, they asked Ab Initio for help.
By using Ab Initio, the financial services company completely transformed rule development. With Ab Initio, they could easily define rules to extract data from virtually any type of data store, transform it into the form they needed, and load it into any destination. This simple process enabled business experts to write, test, and debug the new rules without involving developers. In the first month, a handful of non-technical experts put a dozen new rules into production, a feat that would previously have been impossible.
Ab Initio provided significant improvements in data access and performance. Using Ab Initio software, the company could run their rules daily rather than once every one to two years. Business and regulatory exposure plunged. To top it all off, Ab Initio’s intuitive interface made the rules self-documenting. Rather than trying to maintain reams of documentation about the rules, the company could just let the auditors read the rules.
In the end, the company went from running its rules yearly to daily and decreased the time required to create new rules from months to days. The savings in terms of reduced financial exposure were huge. The savings in terms of not having auditors breathing down their necks were incalculable.