“Organisations have oodles of data floating around, but not all of it is absolutely key to operating successfully. “It’s the art of understanding that there are few principles you really need to adhere to, and understanding which data is key to the organisation,” says Tim Bowes, associate director for data engineering at data consulting firm Dufrain. As organisations store and process more information, this becomes ever more important. This, in turn, relies on a good understanding of the data being collected and held, the systems it is held in, and the regulatory, compliance and security regimes that apply to the data.įirms also need to understand which data is critical to operations, and which delivers the most value. Certainly, an effective data architecture needs to map the flow of information through the organisation.

What is a data architecture?ĭata architecture is often described as a data management blueprint. “If it doesn’t come with an articulated business problem, then that’s the next place to go.” Physical data architecture, data flows and integration of data sources and applications come after that. “I like to ask the question, ‘what are we able to do with better data, what is it that could be different?” says PA Consulting’s Garrood. Building a data architecture should be more than just a housekeeping or compliance exercise. The challenge for CIOs and CDOs is to tie that opportunity back to the needs of the business. The move away from traditional relational database systems to much more flexible data structures – and the ability to capture and process unstructured data – gives organisations the potential to do far more with data than ever before. As management consultancy McKinsey put it in a 2020 paper: “Technical additions – from data lakes to customer analytics platforms to stream processing – have increased the complexity of data architectures enormously.” This is making it harder for firms to manage their existing data and to deliver new capabilities. However, part of the challenge for CIOs and CDOs is that technology is driving complexity in both data management and how it is used. Firms need to ensure that data architecture projects deliver value to the business, he adds, and this needs knowledge and skills, as well as technology. “Data architecture is many things to many people and it is easy to drown in an ocean of ideas, processes and initiatives,” says Tim Garrood, a data architecture expert at PA Consulting.

Each organisation will need to build a data architecture that works for its own needs.

However, given the vast range of data sources and ways that data can be manipulated and used, there is no single blueprint for doing this. At the simplest level, a data architecture is about knowing where the organisation’s data is, and mapping how data flows through it. To address this complexity and make data “work” for the business, companies need to look at their data architecture. At the same time, organisations face regulatory and compliance risks if they are unclear about what data they hold, and where. As a result, firms are forgoing the advantages that data should offer. The situation is made more complex still by the range of tools available for storing and manipulating data, from data lakes and data hubs, to object storage, and machine learning (ML) and artificial intelligence (AI).Īccording to a survey by storage manufacturer Seagate, as much as 68% of business data goes unused.

Datum architecture