The process of billions of the records in banking sector by using Ab initio is very difficult and to process the data it will take lot of time.
The Solution
Expedux designed and implemented the strategy to migrate all the data from Ab Initio store in Hive using spark without any change in data flow. We scheduled Pyspark Job using Oozie and shell script and enabled query performance optimization and updated data using pyspark.