Skip to content
AIV

Data EndPoints Datasource

Follow the steps to create a custom Datasource in AIV:

  1. Create a Java Project:
    Begin by setting up a new Java project in your preferred Integrated Development Environment (IDE).

  2. Include External JAR:
    Add the required external JAR file to the project’s /lib folder.

  3. Create the Custom Class:
    Create a class named Exds and implement the CustomData interface in it.

  4. Override Unimplemented Methods:
    The CustomData interface has three methods that need to be implemented:

    public interface CustomData {
    public List<Map<String, Object>> getData(Map<String, Object> detaisIN, String addInfoIN);
    public List<Map<String, Object>> getMetaData(Map<String, Object> detaisIN, String addInfoIN);
    public String testConnection(Map<String, Object> detaisIN, String addInfoIN);
}
    • getData: Implement logic to fetch data from external sources such as databases or files.
    • getMetaData: Implement logic to retrieve metadata related to tables, columns, data types, etc.
    • testConnection: Implement logic to verify if the connection to the database is successful, if applicable.

  5. Create a Custom Datasource in AIV:
    Navigate to the datasource section in the AIV application. Create a new custom datasource and provide the full class name of the implementation, e.g., com.externalds.Exds.

  6. Test the Connection:
    Test the connection to ensure that it is successfully established and then save the datasource.

  7. Create a Dataset:
    Use the newly created datasource to create a dataset. Click on metadata and preview to verify that it is functioning correctly.

Understanding Data Endpoints in AIV

It seems like you’re describing the flexibility and adaptability of AIV (Artificial Intelligence Visualization) in handling various database connections. The use of external datasource types, referred to as “Data EndPoints,” is an interesting feature that allows users to connect to databases that may not have predefined connections within AIV. This approach enables users to seamlessly integrate new and diverse data sources into the AIV environment. Here’s a brief summary:

  • Predefined Database Connections:
    AIV offers a rich set of predefined database connections, making it easy for users to connect to commonly used databases by providing the required properties and credentials.

  • Dynamic Database Landscape:
    Recognizing the evolving technology landscape, AIV continually updates its list of supported databases to keep up with the latest developments in the database ecosystem.

  • Data EndPoints as a Solution:
    To address situations where a specific database is not readily available in AIV, Data EndPoints serve as a workaround. This external datasource type allows users to connect to databases that may not have predefined connections.

  • Flexibility and Adaptability:
    The use of Data EndPoints highlights the flexibility and adaptability of AIV. Users have the capability to integrate with a wide range of databases, enhancing the platform’s versatility in handling diverse data sources.

  • Streamlined Database Integration:
    By providing a mechanism for users to add external datasources, AIV streamlines the process of integrating databases that might not be part of the predefined list. Overall, this feature demonstrates AIV’s commitment to providing a comprehensive and user-friendly environment for data analysis, ensuring that users can connect to and analyze data from a variety of sources, even as the database landscape evolves.

Objective

The objective of this document is to provide guidance on retrieving both data and metadata from external projects by utilizing endpoints.

Usage

Your explanation provides a clear understanding of how Data Endpoints in AIV empower users to fetch data from any datasource. The flexibility to create projects in different programming languages and send relevant data to defined endpoints demonstrates a versatile and adaptable approach. Here’s a summarized breakdown:

  • Data Endpoints DataSource:
    Data Endpoints in AIV function as a datasource, enabling users to retrieve data from a wide range of sources, even those not directly integrated into the platform.

  • Language Agnostic Projects:
    Users have the freedom to create projects in their preferred programming languages, such as Java, Python, ASP.NET, or any other language. This language-agnostic approach accommodates diverse technology stacks.

  • Endpoint Definition:
    Users define specific endpoints within their projects. These endpoints serve as interfaces through which AIV can communicate with the external projects for data retrieval.

  • Data Transmission:
    The user’s project is responsible for transmitting relevant data to the defined endpoints. This data transmission mechanism allows AIV to seamlessly fetch information from external datasources.

  • Versatility Across Languages:
    AIV’s support for projects in any language enhances its versatility, ensuring users can work with the programming language they are most comfortable with or that aligns with their project requirements.

  • Adaptive Integration:
    This approach supports an adaptive integration model, empowering users to incorporate data from a diverse array of datasources, irrespective of their native compatibility with AIV.

  • Simplified Workflow:
    Users only need to focus on sending relevant data to the defined endpoints, streamlining the workflow for integrating external datasources with AIV. Overall, this solution facilitates a seamless and efficient process for users to access data from various sources, contributing to a more inclusive and adaptable analytics and visualization environment within AIV.

Getting Started

Here we are going to demonstrate this feature using a JAVA Spring-Boot application. For the database, we have used Cassandra in Docker. Please follow the steps below:

  1. Download Spring-Boot Application:
    Download the Spring-Boot application from the link below:
    Click here

  2. Install Docker:
    Install Docker on your workstation. Ensure you have the latest version of Docker Compose. Download Docker using the following URL: Docker Desktop

  3. Setup Cassandra:
    Follow this URL to set up Cassandra in a Docker environment: Cassandra Quickstart
    In STEP 3: CREATE FILES, we have modified the table name and columns for demo purposes:

    -- Create a keyspace
CREATE KEYSPACE IF NOT EXISTS store WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : '1' };

-- Create a table
CREATE TABLE IF NOT EXISTS store.shopping_cart1 (
userid text PRIMARY KEY,
country Text,
amt int
);

-- Insert some data
INSERT INTO store.shopping_cart
(userid, country, amt)
VALUES ('9876', 'USA', 2000);
INSERT INTO store.shopping_cart
(userid, country, amt)
VALUES ('1234', 'UK', 3000);

Endpoints

In your application, you have to create four endpoints listed below, and AIV will access these endpoints to get the information:

  1. /connection
  2. /getobjects
  3. /metadata
  4. /data

The domain and port number of the REST API will be of your project URL and port number. Users need to create a project and write at least four methods to get relevant data from their database for each endpoint. Return this data from the external project, and you can view the information in AIV by using the Data Endpoints datasource.

How It Will Work

  1. Login to AIV:
    Log in to the AIV application using a user with admin rights.

  2. Expand Master Data Section:
    Expand the Master Data section and click on the datasource option.

  3. Create Datasource:
    Click on the “Create Datasource” button from the footer menu. Search for Data Endpoints datasource and click on it to open its properties.

    Image
    Image

  4. Provide Datasource Details:
    Provide the Datasource name in the Display Name field. In the Rest URL field, provide the URL where the project is hosted and port number. Keep other things in the URL as they are. If you have any extra configuration to connect with the database apart from this, you can pass extra properties inside the Extra Configuration field in Key-Value pair.

  5. Test Connection:
    After filling in the required credentials in the previous steps, when the user clicks on the Test Connection button, AIV will call the /connection endpoint on your provided project URL. Users will receive all the information provided in the create datasource fields into their project, and they will get this request in the project to write their own code to validate the connection with the database.

    Once user credentials match, it will return a success message which will be passed to AIV.

  6. Create Dataset from Datasource:
    Once the datasource is created and the connection is tested successfully, the next step is to create a dataset from it.

    Image

    Click on the grid view to go to the datasource grid view as shown in the image below.

    Image

    Select the datasource we have just created from the grid view and click on the “Create Dataset” button from the footer menu.

    Image
    You will see the create datasource window. Select DATA ENDPOINTS option from the search bar.

    Image

GET /getobjects Endpoint

You can find a small refresh icon to the right of the source dropdown. To retrieve the list of tables available in the database connected to the project, click on this icon.

When you click on the refresh icon, AIV will call the /getobjects endpoint in your project. You can retrieve header information indicating where this endpoint is mapped in your project. Users should write code to retrieve table data by connecting with the database. The fetched information or data should then be returned to AIV in the form of a Map of Key-Value pairs.

The returned information is displayed as a list of tables in a tree view, as shown in the image below.

Image

To retrieve data from tables, simply drag and drop a table from the left-side tree view into the right-side query window. Upon dropping the table into the query window, a column selector window will appear. Select the number of columns you wish to fetch from the table and click the ‘Submit’ button.

Image

GET /metadata Endpoint

To check column details, click on the Output Columns tab to view information about the columns you selected in the previous step.

Image

When you click on the Output Columns tab, AIV will call the /metadata endpoint in your project URL. You can retrieve header information indicating where this endpoint is mapped in your project. Users should write code to retrieve selected column information or metadata of the table by connecting with the database. The fetched information or data should then be returned to AIV in the form of a Map of Key-Value pairs.

GET /data Endpoint

After obtaining column details in the Output Columns tab, click on the Preview tab to view the data present in the table with the selected columns.

Image

When you click on the Preview tab, AIV will call the /data endpoint in your project URL. You can retrieve header information indicating where this endpoint is mapped in your project. Users should write code to get table data by connecting with the database. The fetched information or data should then be returned to AIV in the form of a Map of Key-Value pairs.