Home TECHNOLOGY Data Warehouse And Data Lake: Definition And Differences

Data Warehouse And Data Lake: Definition And Differences

But where is the data in the company? The question introduces us to two other terms, Data Warehouse and Data Lake, on which it is necessary to make a minimum of clarity , especially in order not to run into the typical, and superficial, approach that declares certain technologies dead because they do not fully meet the needs of the “digital “(and I am thinking of mainframes, ERPs, etc.). The ease with which, sometimes also for marketing reasons, certain technologies are defined as “rock” leads us to think that the “others” must be quickly dismissed.

The Data Warehouse, according to some, should follow this fate. Its construction results from heavy investments, but certain types of analysis play well its function and, therefore, is often defining hybrid architectures that see the most traditional data Warehouse work alongside the smartest Data Lakes.

The term Data Lake was minted in 2010 and is attributed to James Dixon, CTO of Pentaho (a BI company later acquired by Hitachi Data Systems), who used the metaphor of water (data) and of the lake or bottle ( basin and container) to exemplify the concept:

Data Warehouse

The water is bottled in separate containers according to the type of source that generated them. The destination and use for which they are intended; the data is then saved in homogenous formats, refined, and ready to be distributed operators and systems with specific functions.

The ETL process is typical of Data Warehouses. At the base of the so-called ” schema on write ” where the structure of the database is defined a priori, the data is written to this defined structure and then analyzed.

The process is complex and expensive considering future integrations and implementations with new sources, which is essential if you do not want to build a rapidly obsolete container.

Its features and access modes are perfect for users who need relatively simple analysis and, above all, who know what to look for, what type of relationship to analyze. They are business users who find in these “containers” and in the business intelligence systems that query them excellent answers in terms of reporting and understanding the trend of known phenomena.

In the first decade of the 2000s, the evolution of these solutions was concentrated on the user interface and on the possibility for users to build their dashboards for analysis of specific interest to the user.

Data Lake

Data, like water, flows freely into the “basins” from the sources that generate them, and in these basins, they are examined and sampled in their original formats:

only when applications and operators interrogate them are they converted into readable formats business systems and can be compared with other information.

A Data Lake can include structured data from relational databases (rows and columns), semi-structured data (CSV, log, XML, JSON), unstructured data (emails, documents, PDFs), and binary data (images, audio, video).

Each element is uniquely identified through a series of tags that correspond to its metadata; this allows, when a Data Lake is queried about a specific problem, to extract the relevant data and then submit them to analysis.

According to a specific model, the so-called “on read scheme” is where the data is read without being “written.” The Data Lake, compared to the Data Warehouse, has the advantage of being very flexible and does not require lengthy implementation;

It has simpler management and is reversible without problems (different types of data can be entered and deleted without applying changes to the structure).

Data Lakes are ideal for carrying out ever deeper analyses, discover unimaginable relationships that can lead to the identification of new businesses, but require specialized figures in the analysis. Data Scientists excel, but of which there is a significant lack.

According to Forrester, the results were below expectations: the data lakes proved too expensive, slow to update and what was done was useless, retracing the failures already seen in the past with data warehouses and BI platforms.

Tech Buzz Reviews
Techbuzzreviews are a team full of web designers, freelancers, marketing experts, bloggers. We are on a mission to provide the best technology-related news with passion and tenacity. We mainly focus on the areas like the latest technology news, upcoming gadgets, business strategies and many more upcoming trends which are trending all over the world.

Most Popular

Top 10 E-Commerce Website Development Companies

We are living in times smitten and overtaken by technology. What we couldn't think of getting done without moving mountains a decade ago can...

Changing Your Google Workspace Plan Type

Google applications are everywhere, and chances are, you've used at least one of them, whether for personal or professional use. There's Gmail, the very...

How To Use The Skyscraper Technique: A Beginner’s Guide

Those in the business of content creation are competing to be ahead in terms of ranking in search engines. The so-called Skyscraper Technique is...

How Real-Time Tracking Systems Improve Safety In Industrial Companies

In industrial companies, the risk of injury for employees is high compared to other workplaces. Real-time location systems increase employee safety and reduce responder...

Employee Engagement Ideas For Remote Workers

With the outburst of the pandemic COVID-19, businesses had no choice but to embrace the work-from-home lifestyle. But, now two years later, its’ not...

How To Implement Attraction Marketing

The main thing you can do with fascination advertising is sharing your story. You can use this procedure to genuinely interface with a lead...

Social Agency Or Do It Yourself? Advantages And Disadvantages

Everyone has a Facebook page or an Instagram profile, but how many can manage these tools when a business is involved? After all, creating...