EY is orchestrating a data and AI ecosystem of partners to accelerate end-to-end enterprise transformation

PAC recently had the opportunity to attend the EY Global Data and AI Summit in Spain. The event provided a range of interesting and thought-provoking sessions led by a combination of representatives from EY, independent software vendors (ISV), and enterprise organisations from across the world. It was an opportunity to bring together people from a wide range of backgrounds to discuss how organisations must continue to evolve to maximise their potential through innovative data use cases supported by ever more sophisticated artificial intelligence (AI) capabilities. EY assembled a diverse range of international enterprise speakers spanning automotive retailing, mining and metals, insurance, and banking to demonstrate data and AI-led industry transformation.

It was clear from those speaking that organisations continue to be challenged in the transition from legacy data management approaches to contemporary techniques supported by ever more ubiquitous and sophisticated uses of AI. Organisations find themselves in a place where not only is data as valuable a commodity as the products and services they provide or sell, but the ever-growing expectation for deep and rapid insight from data at scale is a critical business differentiator.

From both listening to sessions and speaking with attendees, it was clear to PAC that the exponential global demand for data and AI skills combined with an ever-growing diversity of ISVs is, at least in the near term, creating a challenging environment for enterprises to meet their data demands. As PAC can attest, the ever-growing breadth and depth of ISVs in this space is challenging at best to keep abreast of. This is where the experience and expertise of EY provide a crucial role in building relationships across the data and AI ISV ecosystem to ensure their clients are exposed to partners best suited to their needs. EY rightly used the phrase “better together” to describe the ecosystem of partner relationships needed for an enterprise organisation to be successful at scale regarding their data and AI strategy.

There were many impressive ISVs at the global summit, but PAC considered one, in particular, to provide some really compelling capabilities to address the complexities enterprise organizations are navigating regarding data management and governance. The company in question is Databricks, and they are of particular interest due to the conversations PAC is having with CIOs regarding their data and AI strategies.

CIOs continue to migrate their compute and data workloads to a variety of different types of cloud platforms. However, it is all too common for enterprise organizations to be burdened by the complexity of the business operating models that have not been designed in a manner that makes the transition to contemporary data and AI strategies a relatively smooth task. The cliché of multiple silos of data within one system and across many is still as much a reality today as it was a decade ago sadly. Migrating to a modern data strategy has historically been a time-consuming, highly complex, and costly programme of work. CIOs that PAC speaks with commonly indicate a hesitancy in moving from their existing complexity into what may become a modern cloud-based version of the same challenge over the long term if not planned for correctly. PAC also continues to see enterprise organisations spend significant amounts of time and resources trying to triage data into a perfect form before migrating it.

This is where the session by Databricks was of particular interest to PAC. The company was founded in 2013 by the original creators of Apache Spark. They have popularised the term data lakehouse to describe how their platform combines the best elements of data warehouses and lakes to provide an open-source-based data engineering platform to process and transform large volumes of data ideal for AI models and general data processing. The architecture of Databricks separates the storage and compute layers which helps with scaling and provides operational flexibility across multiple suppliers. They operate atop the Amazon, Google, and Microsoft hyper-scale cloud compute platforms and can currently be included as part of a Microsoft Enterprise License Agreement (ELA). This last element, regarding the Microsoft ELA, is a particularly advantageous relationship for Databricks as it lowers the barrier of entry by opening a much broader potential customer base through an enterprise organisations relationship with Microsoft. Databricks also integrates across an extensive ecosystem of data sources, developer tools, and partner solutions.

However, despite all these valuable capabilities, PAC considers the Databricks medallion architecture a significant differentiator in addressing how an enterprise organisation drives data insight at scale whilst ensuring quality. Whether data sources are internally or externally created, enterprise organisations continue to see the volume, velocity, variety, and complexity of ingested data grow exponentially with every year that passes. PAC considers managing data quality at scale to be one of the fundamental challenges a CIO faces that, if not addressed, will profoundly impede the success of an enterprise organisation. This is where the medallion architecture, as part of a data lakehouse, provides a three-tiered system (i.e., bronze, silver, gold) to filter unstructured, dirty, and diverse data at scale using schemas. Such an approach provides a scalable and secure foundation that identifies data not suitable to be promoted to the next tier without it being corrected or disposed of. This ensures AI models and business services are provided with data that meets the quality an enterprise organisation needs at scale.

At the summit Databricks founder Ali Ghodsi reinforced the “better together” EY theme by discussing why the partnership with EY is essential in combining the technology innovation of Databricks with the implementation innovation of EY. PAC can attest that enterprise organisations continue to be challenged with access to data and AI management expertise that can accelerate their path to value realisation and creation. PAC considers the partnership between EY and Databricks to be a compelling combination to support enterprise organisations navigate their data and AI needs. However, the technology and services landscape is highly competitive and continually evolving, so both companies need to keep “evolving together”.