The Challenge
With their more than 3,000 customers, ekz is the leading service provider for libraries in the German-speaking regions. To increase efficiency in the procurement management of its library customers, an Approval Plan solution was to be provided. This solution needed to shorten delivery times for libraries and streamline the customers’ process for building inventories. ekz had already made a decision for the solution offered by NEWBOOKS Solutions from Cologne. A publication date had also already been determined. The only remaining challenge for the desired solution was the provision of product metadata from differing sources in the ONIX 2.1 data format – a standard in the book trade. This meant consolidating source data supplied in a wide variety of formats so that it can then be exported into the target format in uniform quality. Within ekz, intellectually recorded catalogue entries were available in the standard MARCXML format. In addition, there were other data sources that provided data in proprietary formats including MARCXML, Onix 2.1 or Onix 3. There was no data converter between the exchange formats available on the market.
Before the start of the project with Avantgarde Labs, the editorial team already managed over 400,000 items of metadata, which were recorded intellectually in the data management by trained librarians in accordance with the RAK-ÖB rules or, since 2018, in accordance with RDA. Third-party data were used selectively, but could hardly be merged automatically or aggregated with the self-generated data. ekz’s goal was to find a solution that would, on the one hand, shift the ever-increasing burden of metadata processing due to the expansion of supplier relationships to automated processes and, on the other hand, not jeopardize the upcoming release of the Approval Plan solution.
The Avantgarde Labs solution
Since the existing concepts for mapping the MARC to ONIX formats did not produce reliable results and the release date of the Approval Plan solution was not to be jeopardized, Avantgarde Labs implemented a two-step process:
- Phase 1: Design of transformation logics from MARCXML to ONIX as input for the Approval Plan solution
- Phase 2: Implementation of a web-based self-service system to integrate data
Phase 1: Short-term immediate solution The ekz experts were provided with Excel sheets with a predefined syntax to design the transformation logics from MARC to ONIX. Meanwhile, the development team began automated reading of the defined rule sets from Excel and generated automated transformation processes from them. These were applied directly to the quality-assured ekz MARC data and the MARC XML data of the German National Library. The result was data that could be merged with ONIX data from VlB and Libri, two leading metadata suppliers in the book industry, and provided valid ONIX for the go-live of the Approval Plan solution.
Phase 2: Provision of a sustainable self-service solution Since Excel cannot be used as a generic mapping tool in the long term, a web-based self-service system – a PIM solution – was made available to the experts at ekz. With this custom PIM solution as a link between several systems, it was possible to implement the entire data workflow in the product data management – without developer support:
- Onboarding of new data suppliers, independently of formats and interfaces.
- Creation of a product data model that uses a set of rules to draw on the respective best information from the data sources available and generates a “golden record” as a single point of truth.
- Use of provenance information to ensure traceability of the data origin.
- The transformation processes as designed can be deployed easily for ongoing productive operations.
Project results & customer benefits
The extremely tight schedule up to the release of the new Approval Plan solution was met using an agile and pragmatic approach. The development team delivered Phase 1 on time. The Approval Plan solution went live without delay and provides ekz with a new form of sales access to its thousands of library customers. The first version of the PIM solution was put into operation by the responsible department after only 6 months. In the meantime, the entire ekz data aggregation runs on the new application. With the help of the new product information management system, a much larger volume of data can be processed with significantly less work and the intellectual refinement of the data can be supported efficiently. As a result, the available database has been expanded from 400,000 items then to over 21 million items, and numerous downstream systems have been connected with the processed data. Due to the flexibility of the new PIM solution, many other formats are now also supported in both import and export. The automation of the processes not only led to a noticeable reduction in costs, but also reduced the time to market while at the same time improving data quality.
The customized PIM solution also enables ekz to further establish itself as an innovative service provider in the library and publishing market. By providing the highest degree of autonomy and flexibility, the foundation for continuous further development of the digital business model has also been laid.
"The product information system created by Avantgarde Labs has become the central tool for our metadata management in a very short time. The PIM is a data hub for product data and, thanks to its flexibility, supports diverse automated data refinement processes. New data sources can be integrated and new data recipients can be served now very easily and with precision, which has resulted in a very significant optimization of metadata usage."
Tobias Schmid
Head of Processes and Projects, ekz.bibliotheksservice GmbH
