SmallNet currently support a leading insurer operating in the Lloyd’s, UK, Continental Europe, and Bermudian markets. Specialising in providing insurance cover to commercial enterprises and offering reinsurance protection to other insurance companies around the world.
Initially the client started the dialogue with the Enterprise Architects around conducting a presentation of Data Models and data modelling tools; conceptual, physical, and logical modelling, with the potentially the IBM Insurance Models.
However, it became clear that the client wanted to not only model the data but click on the data or data model and understand the business term and make this available to the end user community, after careful due diligence they selected the IBM InfoSphere Governance Catalog (IGC) to achieve their initial goals/aims by providing a platform to create a meta data Catalog populated with Terms, mappings (lineage) and data models (Logical and Physical).
The primary data source initially comes from Microsoft’s SSIS ETL tool (company standard) and via a IBM third party a connector provides all the ‘operational’ metadata so ultimately the end user can drill down from a report to see not only the business term (meaning) but could see how (the lineage) the data was processed plus when, where and by whom providing trusted data across the enterprise enabling the analysts/users make confident and accurate decisions.
The data is reported in a multi-faceted Business Intelligence tools for which the IBM tool (IGC) Information Governance Catalog which has numerous metadata bridges to present this to the BI layer.
Today they have some 1,600 end users accessing the business data via their BI reporting tools and a number of admin/authorised business architects enabled to edit/amend and publish business terms/glossaries and policies in line with the business.
Next steps and Dynamic changes in growth – Due to organisational changes the provisional of Data Governance services is changing to reflect changes in the organisations business structure. The business has grown through acquisition over the years, initially these new businesses were incorporated into the organisation and a central data governance team was responsible for all data governance tooling and activity.
The business has been re-structured into a number of functionally focussed legal business entities with their own balance sheet and P&L. The centralised model for Data Governance has changed to become a Data Governance platform that the legal entities can used for their own purposes. The entities will be responsible for their own implementation of Data Governance using a common platform of tools and services provided by the parent organisation.
Requirements / Projects
- To extend their Governance capability the next phases of development are to extend and improve the Lineage capability to cover assets across the organisation and better manage and administer it.
- To monitor and manage data quality, linking results and activities to business assets in IGC.
- To move from a central Data Governance office to a Data Governance platform and services to be consumed by the legal entities.
SmallNet was engaged to help with all of these objectives, to be run as separate projects. Both projects would follow the same pattern of an initial pilot to help firm the requirements, proof of capability, and understand the potential of the tooling.
What we did
Data Lineage
Detailed analysis of the lineage requirement, to capture lineage from many different sources including models and spreadsheets and other sources not explicitly covered by standard IGC bridges or connectors. In addition, analysis of how the business wanted to use and exploit / consume the lineage information. Flow diagrams can look impressive but do not necessarily fit with the day to day operational benefits lineage has to offer.
How?
- Created and implemented a data lineage architecture that enabled the organisation to capture lineage from any source as mappings using familiar tools such as excel.
- Created tools to apply and maintain the mappings directly into IGC.
- Provided an SmallNet virtual data mart for lineage that provided familiar ODBC/SQL based access to the lineage information stored.
- Created lineage reports through their reporting tool of choice, Power Bi.
Delivered as a pilot project that delivered a working model for a small lineage sample, which could then be adopted and owned / used by the organisation.
