CBA Enhances Data Lake with Cloudera Upgrade

Cloudera Upgrade Powers CBA’s Enhanced Data Lake and AI Strategy

The Commonwealth Bank of Australia (CBA) is the nation’s largest bank, proudly serving over 17 million customers. It provides retail and commercial banking services predominantly in Australia and in New Zealand through its subsidiary, ASB. 

CBA has just completed a major upgrade to the latest version of Cloudera, marking a significant milestone in an ambitious, multi-year program designed to modernize and optimize CBA’s data infrastructure. This ambitious initiative lays the groundwork for the Bank's strategy to migrate the data lake to the cloud, ensuring CBA remains a data and AI leader in the financial service industry. 

Omnia: Elevating CBA's Data Lake to New Heights with Cloudera's Latest Innovations

CBA’s data lake, named Omnia, is the largest Cloudera installation in the Southern Hemisphere, with a staggering 24,000 pipelines powering critical business functions for the Bank’s retail and business customers. The sheer volume of pipelines, combined with a two-major-version in-place upgrade, made this a particularly complex undertaking. 

Given its scale and complexity, the upgrade project was a comprehensive collaboration between CBA and Cloudera. Cloudera's professional services team partnered closely with CBA, embedding Cloudera’s solution architects within CBA’s project team to provide continuous support and guidance.

The Cloudera team delivered invaluable insights to CBA’s Omnia platform team, covering not just the infrastructure and operating system, but also the critical technical components like Java and back-end databases. 

Cloudera's latest software release enabled CBA to perform an in-place upgrade with full rollback capabilities over the course of a weekend. This meant no additional environments were needed, and there was no need to duplicate any data or metadata. 

Following the upgrade of Cloudera’s core elements, pipelines were systematically released and data validation was conducted to support data delivery for critical business functions.

As a result of the collaboration and extensive project planning, the upgrade was successfully completed. This upgrade unlocks the full potential of Cloudera's latest innovations, offering improved cost management and enhanced security features, while bolstering CBA’s operational stability and cyber security. This not only future proofs and safeguards the Bank’s invaluable data assets but also ensures that CBA stays at the forefront of data and AI.