Connected Vehicle Data Migration

SCENARIO

In a world where big data is an invaluable asset for organizations, a leading car manufacturer company decided to rely on Technology Reply's expertise to enrich its new analytics platform, with useful information for data analysis related to the areas of autonomous driving, smart cockpit and connected vehicle services.

The analytics platform will enable the study and analysis of driving style, through significant key indicators.


SOLUTION

Through the signals coming from sensors placed in the vehicles, a large amount of data can be collected, providing visibility of the use of the cars in terms of usage and activity times; relevant information for undertaking advanced predictive maintenance and driving behavior analyses. Through such analyses, the identification of the real needs of drivers can be identified in order to provide them with the most suitable services.

The solution implemented on the new platform includes, in addition to the process of Extraction, Transformation and Loading (ETL) of the data on various layers of the lakehouse, the enrichment of the information already presents on the new platform.

Until now in fact, the current platform had only the data related to the vehicle, without product and market type details.

Through the migration and exposure of data on the new platform, it will now be possible to integrate the vehicle data with the related product and market master data, in this way the end user could perform deeper analysis with greater levels of detail.

Moreover, in addition to the implementation of the back-end solution, Technology Reply will be responsible for designing and developing a front-end solution, for the creation of dashboards dedicated to the visualization of reports and key indicators useful for the business users analysis.

The project is divided into four macro areas:

  • Migration from the decommissioned system in favor of the new platform, of the entire ETL process and calculation of important KPIs related to the connected vehicle
  • Connecting and exposing large datasets to enrich the information already present on the new platform and facilitate more detailed business analyses
  • Porting and readjustment of current dashboards to highlight different driving styles, types of roads and trips driven, performance statistics of battery levels etc.
  • Optimizing the performance of current front-end dashboards by relegating complex logic to the Back-end, previously developed on the Front-end, with the idea of improving the flow of information through detailed levels of data aggregation


TECHNOLOGIES

The technologies used are the following ones:

  • A fast and collaborative analytics platform based on Apache Spark, supporting Python, R, Java, and SQL languages, in addition to the data science frameworks and libraries (e.g., TensorFlow, PyTorch, and scikit-learn). It is a Cloud-based tool created with the goal of processing and transforming massive amounts of data by leveraging a fast, reliable, scalable and easy-to-use work environment for all professionals.
  • A business intelligence tool that enables the generation of detailed dashboards and reports for analyzing, visualizing and sharing business data. Through this tool, data overviews, trends, problems identifications and timely decisions making can be performed quickly.


CHALLENGES

The main challenge concerned the management of a huge amount of data that is sent daily in near real time from the sensors placed inside the vehicles. Technology Reply was able to meet this challenge through the study of a solution based on data aggregation logics aimed to performance optimization of the ascent on the different layers, all leveraging the new Cloud-based tool, adopted specifically for large amount of data processing and transforming.