Skip to content
AIV

Airbyte description

Airbyte Introduction

Airbyte is the new open-source data integration platform that runs in the safety of your cloud and replicates data from applications, APIs & databases to data warehouses, lakes and other destinations. Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Overview

This document contains the process to get the data from the various vendors to the AIV application. Let’s understand it with the small diagram: -

Image

The Airbyte will collect the Detected datasets and provide curated datasets to our application. For more detail understanding, go to Airbyte documentation: https://docs.airbyte.com/

Airbyte Characteristics

  • The Airbyte Auto Sync the updates of the data from the vendor as per the selected time for example from once in 5 minutes to every 24 hours.

  • User can sync the data manually every time they update from the vendor.

  • User can add a prefix to streamed sources.

FAQ

  1. What if we connect the source and fetch the data to make datasets in our application then the datasets seem empty? Ans: Yes, it may seem empty, due to not having the data or it may show you the Sample data provided by the vendor. to bypass that situation, try to make an app, project on the vendor’s side.
  2. Why the sync takes much time to get the data? Ans: It depends on the vendor’s logs and its loading time.
  3. What if we have all the properties but the airbyte connection gets failed to connect us with the vendor. Ans: If you have all the properties and the connection gets failed, then you can raise the Question to the Airbyte developers on https://discuss.airbyte.io/new
  4. Can we add a specific time to sync the data on airbyte? Ans: No, There is no option available right now on airbyte, you need to add a specific time duration to sync the data like once a day(Every 24 hours) or every 5 minutes.