Home TECHNOLOGY Data Warehouse And Data Lake: Definition And Differences

Data Warehouse And Data Lake: Definition And Differences

But where is the data in the company? The question introduces us to two other terms, Data Warehouse and Data Lake, on which it is necessary to make a minimum of clarity , especially in order not to run into the typical, and superficial, approach that declares certain technologies dead because they do not fully meet the needs of the “digital “(and I am thinking of mainframes, ERPs, etc.). The ease with which, sometimes also for marketing reasons, certain technologies are defined as “rock” leads us to think that the “others” must be quickly dismissed.

The Data Warehouse, according to some, should follow this fate. Its construction results from heavy investments, but certain types of analysis play well its function and, therefore, is often defining hybrid architectures that see the most traditional data Warehouse work alongside the smartest Data Lakes.

The term Data Lake was minted in 2010 and is attributed to James Dixon, CTO of Pentaho (a BI company later acquired by Hitachi Data Systems), who used the metaphor of water (data) and of the lake or bottle ( basin and container) to exemplify the concept:

Data Warehouse

The water is bottled in separate containers according to the type of source that generated them. The destination and use for which they are intended; the data is then saved in homogenous formats, refined, and ready to be distributed operators and systems with specific functions.

The ETL process is typical of Data Warehouses. At the base of the so-called ” schema on write ” where the structure of the database is defined a priori, the data is written to this defined structure and then analyzed.

The process is complex and expensive considering future integrations and implementations with new sources, which is essential if you do not want to build a rapidly obsolete container.

Its features and access modes are perfect for users who need relatively simple analysis and, above all, who know what to look for, what type of relationship to analyze. They are business users who find in these “containers” and in the business intelligence systems that query them excellent answers in terms of reporting and understanding the trend of known phenomena.

In the first decade of the 2000s, the evolution of these solutions was concentrated on the user interface and on the possibility for users to build their dashboards for analysis of specific interest to the user.

Data Lake

Data, like water, flows freely into the “basins” from the sources that generate them, and in these basins, they are examined and sampled in their original formats:

only when applications and operators interrogate them are they converted into readable formats business systems and can be compared with other information.

A Data Lake can include structured data from relational databases (rows and columns), semi-structured data (CSV, log, XML, JSON), unstructured data (emails, documents, PDFs), and binary data (images, audio, video).

Each element is uniquely identified through a series of tags that correspond to its metadata; this allows, when a Data Lake is queried about a specific problem, to extract the relevant data and then submit them to analysis.

According to a specific model, the so-called “on read scheme” is where the data is read without being “written.” The Data Lake, compared to the Data Warehouse, has the advantage of being very flexible and does not require lengthy implementation;

It has simpler management and is reversible without problems (different types of data can be entered and deleted without applying changes to the structure).

Data Lakes are ideal for carrying out ever deeper analyses, discover unimaginable relationships that can lead to the identification of new businesses, but require specialized figures in the analysis. Data Scientists excel, but of which there is a significant lack.

According to Forrester, the results were below expectations: the data lakes proved too expensive, slow to update and what was done was useless, retracing the failures already seen in the past with data warehouses and BI platforms.

Tech Buzz Reviews
Techbuzzreviews are a team full of web designers, freelancers, marketing experts, bloggers. We are on a mission to provide the best technology-related news with passion and tenacity. We mainly focus on the areas like the latest technology news, upcoming gadgets, business strategies and many more upcoming trends which are trending all over the world.

Most Popular

Optimizing Your Home Network for Better Performance

Most of our day-to-day activities depend on the internet and having a sluggish home network can be a source of constant frustration. Whether it's...

QWERTY: The Origins Of The Keyboard We Use Every Day

The historical backdrop of the QWERTY console: where the format we use on workstations, notepads, and cell phones comes from. We have it before...

The Painter’s Guide to Estimates: Unlocking Insights for Successful Projects

Introduction: Estimating the cost of a painting project is an intricate art form that demands a keen eye for detail, a deep understanding of materials...

Realme C67 – The Smartphone With A Killer Price Arrives

Designed for the needs of younger people, realm's new smartphone offers a premium experience in a low price range. The new Realme C67 is...

An Overview of APA Dissertation Format Requirements with Essential Writing Tips

An APA dissertation format communicates scientific communication by standardising research work. It promotes clarity of expression. APA style makes it easier for the reader...

Reducing Overhead Costs with LED Warehouse Lighting

In the vast expanse of a modern warehouse, lighting isn't just a matter of flipping a switch; it's an ongoing operational cost impacting everything...

The Power Of AI In Cybersecurity: 3 Questions CISOs Should Ask Themselves

To protect businesses against growing threats, it is essential to carefully review new technologies before deploying them. Even more precisely, the year of generative...