The Challenge
MISUMI Europa GmbH is a company active throughout Europe in the area of catalogue sales for mechanical engineering items. The field of trading in industrial supply products is very diverse and it is not possible for any single company alone to carry all the products required by their customers in their own assortment. Nevertheless, in an effort to reduce drop rates with a one-stop store, Misumi decided to move away from being a distributor to become a trading platform. To be able to offer the required third-party products in the company’s own store requires that the product data from other retailers must first be imported into Misumi’s product information system. To do so means that the source data structure must be converted to that of the target system.
This cleanup and transformation was initially carried out by Misumi in a manual process. Due to the increasing amount of data and the addition of new data suppliers, product data onboarding was outsourced to a service provider. However, the cost and time to deliver a complete import of new product data increased in an essentially linear relationship. In addition, errors kept appearing in the product data due to the manual work. The resulting incomplete product data had a negative impact on findability, which caused dissatisfaction among customers.
The Avantgarde Labs solution
The goal was to substantially reduce the costs as well as the time required for retailer onboarding through partial automation, while ensuring high data quality. The manual transfer of a single data record was to be replaced by the establishment and automated application of rules. In the first step, the BMEcat files of the suppliers were prepared for further processing. The Talend data integration tool was used for this ETL process. With its standard connectors and no-code approach, this tool enabled efficient standardization of the source data. However, since the actual catalogue integration could not be carried out on the basis of predefined rules, a user interface was required.
For maximum efficiency, a multi-stage mapping process between the raw data and Misumi’s target system was designed and implemented in cooperation with the respective departments. The procedure included the step-by-step mapping of categories, brands, series and product properties in self-service. To make the application as efficient as possible, the resulting mapping tables were held for the data transformation in a Mongo DB, which is particularly well suited for hierarchical data structures. Based on this information and the prepared source data, it was possible to create the CSV files for import into Misumi’s PIM.
Project results & customer benefits
With the support of Avantgarde Labs, Misumi was able to successfully implement a key component for the trade platform strategy – a scalable data sourcing and provisioning system. This meant the partial automation completely replaced the manual ad-hoc process. This greatly reduced the time to market for product data onboarding and decoupled the costs from the size of the product catalogues to be imported.
By automating data conversion, careless mistakes are avoided and data quality problems in supplier data can automatically be detected and corrected with standardized rule sets. The optimized product data helps to increase confidence in the purchasing process among demanding customers and creates a differentiator from the competition.
