Data loading

The loading of the data in a warehouse is one of the most important process in developing a business intelligent software. In the staging area after the data has been extracted from the source system and transformed according to the target system the next step comes is to load the data in the warehouse.

There are three common techniques for data loading: 1. Full data refresh. 2. Incremental Load / refresh. 3. Trickle Feed.

Full Data Refresh:

In full data refresh, following steps are taken on target system(table):
a) Target table data is deleted.
b) All new data is loaded in the target from the source.

Incremental Load/Refresh:

In incremental refresh:
a) New and updated data is acquired from the source system.
b) The new data is appended to the existing data in the target system and the updated data is updated in the target system's existing data.

Trickle Feed:

In trickle feed: When ever the source system is updated, the changing are made in the target system immediately.

---

Things to Consider:

Main Objectives under consideration while deciding the loading strategies are:
a) Increase data freshness.
b) Increase data loading performance.