Delve deeper into Ab Initio’s governance, quality, and catalog capabilities:
Highlights
Data Catalog
Enable a data-driven organization — discover, catalog, enrich, curate, and access your enterprise data.
Ab Initio provides end-to-end data cataloging, regardless of whether the data resides in the lake, the cloud, an operational system, a data warehouse — or another data catalog. From identifying datasets and enriching them with meaning, to curation and access, Ab Initio automates many key steps in the process. As a result, when users search for data and explore it, the data’s meaning is clear from semantically generated links to business terms. Users determine whether data is fit for purpose by exploring profiles, models, data formats, data quality, lineage, and reviews. A user can even display access-controlled live data that is masked on the fly, based on that user’s permissions.
Ab Initio’s data catalog provides the necessary foundation to enable data-driven processes and decisions. However, rather than being a passive repository used only for reference, Ab Initio’s data catalog is an active repository; it is used to drive operational processes. For example, because the data catalog knows both the physical data and the associated logical meaning of the data (for example, “this is a country field”), it can automatically generate the operational data quality rules to process the data. The productivity benefits of such an approach can be significant. Numerous automation opportunities exist in areas such as PII data security, building data lakes and data marts, simplifying data wrangling and data access, self-service provisioning, and so forth.
Case Studies
Enterprise Data Quality Management
Sometimes, it’s not enough to just control the data!
For one large US financial institution, unmanageable data was a problem. Their data management was scattered across the enterprise, and different parts of the bank used different systems.
Many of these systems were homegrown using open source and various data management products. One area had a completely separate IT system. With no standard system or method of managing data quality, the company couldn’t trust the results and they couldn’t answer questions from regulators. That made the regulators unhappy, and when regulators are unhappy, no one is happy.
Working with Ab Initio, the financial institution began standardizing their operations and eliminating all point-to-point interfaces and legacy datastores. They built a data lake using Ab Initio’s data governance, data quality, and data lineage capabilities to govern the data and track data quality. Ab Initio’s intuitive, spreadsheet-like interface for rule development greatly simplified the process of developing data quality rules and putting them into production.
Becoming skilled at finding bugs in data leads to a new set of challenges. Financial institutions have a lot of data. Even if only a small percentage of that data has issues, that’s still a lot of issues. It doesn’t take much before the people who fix those issues are overwhelmed with the number of bugs that need attention.
Once again Ab Initio was able to help. With Ab Initio, the financial institution built a complete data quality management solution. More than simply executing data quality rules, the new system let the company manage the entire process of identifying, tracking, and fixing problems with their data and data quality rules without overwhelming anyone with lots of defects.
When a record fails a data validation rule, the system checks to see if there is already an open ticket for that specific failure. If there is, the record is added to the existing ticket. If no ticket exists, a new one is created. That ticket will end up being composed of all the records that fail a specific validation test. Ab Initio assigns that ticket to the appropriate person based on stored reference data, who then drills down into the attached records and determines whether there is a data problem or a rule error.
If the problem is with the data quality rule, the ticket is reassigned. If the problem is with the data, the data owner is responsible for fixing it and is immediately notified. Fixes need to be logged and verified. In the days before the financial institution adopted Ab Initio, assigning responsibility at such a granular level would have been impossible. It was impossible no longer.
With their new system in place, the financial institution can track down data quality issues, fix them, and then verify the fix, all without overwhelming the people responsible. Data issues are routed to data owners, and rule errors are routed to the appropriate business experts who test and debug the rules. This is data quality management at the enterprise level, powered by Ab Initio.