Extract, Transform and Load

ETL - Introduction

etl_-_introduction

ETL-process

ETL (Extract, Transform and Load) is a process in data warehousing responsible for pulling data out of the source systems and placing it into a data warehouse.

The data is loaded in the DW system in the form of dimension and fact tables.

ETL involves the following tasks:

Extraction

Extracting the data from source systems (SAP, ERP, other operational systems) and then data is converted into one consolidated data warehouse format which is ready for transformation processing.

Data is extracted from the source system and kept it in the staging area.

Staging area is required when you want to get the data from multiple data sources together or if you want to join two or more systems together. For example, you will not be able to perform a SQL query joining two tables from two physically different databases.

Transform

In data transformation, you apply a set of functions on extracted data to load it into the target system. Data, which does not require any transformation is known as direct move or pass through data.

Transforming the data may involve the following tasks:

  • Applying business rules (so-called derivations, e.g., calculating new measures and dimensions)
  • Cleaning leaning (e.g., mapping NULL to 0 or "Male" to "M" and "Female" to "F" etc.).
  • Filtering (e.g., selecting only certain columns to load).
  • Splitting a column into multiple columns and vice versa.
  • Joining together data from multiple sources (e.g., lookup, merge).
  • Transposing rows and columns.
  • Applying any kind of simple or complex data validation (e.g., if the first 3 columns in a row are empty then reject the row from processing).

Load

During Load phase, data is loaded into the end-target system and it can be a flat file or a Data Warehouse system.

See also Dimension and Fact Tables


If you like dEexams.com and would like to contribute, you can also write your article here or mail your article to admin@deexams.com . See your article appearing on the dEexams.com main page and help others to learn.


Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above.


Python if , elif and else

Python Conditions and If statements

  • 0
Python for beginners

Learning Python Part 1

  • 3
Struct Alignment and Padding

Struct Alignment and Padding in C++ And C

  • 0
Friend function

Friend function C++

  • 0
Pointers

C++ Pointers

  • 0
Structures

C++ Structures

  • 0
Types of Inheritance in C++

Inheritance and access specifiers C++

  • 0
Java date pattern

Java Date Pattern Syntax

  • 0
Java Date and Calendar

Java Date formats

  • 0
JAVA Data Type

Data types in Java

  • 0
Java unreachable code

Unreachable Code Error in Java

  • 0

Post Comment

Comments(0)

WEB TECHNOLOGY

Articles

×

Forgot Password

Please enter your email address below and we will send you information to change your password.