Skip to content

How to install and use xData jobs

This document has been machine translated.

Overview

When using Talend to process xData data, you may need to

  • Retrieve data from xData tables using the Event Data Distribution API
  • Insert data into xData's tables using the Data Loader API

Since these APIs are provided as REST APIs compliant with the JsonRPC standard, they are not easy to use from Talend.

Therefore, we have prepared a set of jobs ("xData jobs") that serve as wrappers for each of the xData APIs. By placing an xData job in a job you are creating via drag & drop and specifying the necessary parameters, you can easily use the API.

Below are instructions for using xData jobs and a list of available jobs.

Importing an xData job

Before you can start using xData jobs, you need to import them by following the steps below.

  1. after launching Talend, right-click in the tree on the left and select Import items.
  2. Select the separately distributed(If you use xData Edge, it exists in the xdata-edge-base-X.X/supplements directory) talend_xdata_jobs-2.0.zip in the Select archive file, check all items (Select All), and press the Finish button.
  3. After successful import, the xData job will be added under Job Designs in the tree.

Using xData Jobs (Basic)

The basic usage of an xData job is as follows.

Placing and connecting xData jobs

  1. select the xData job you want to use from the tree of xData jobs (under xdata_jobs), and drag & drop it onto the job you are creating.
  2. To connect xData jobs to each other or xData jobs to other components, you must use Trigger On Component Ok. (Right click on the job you are connecting from > Trigger > On Component Ok)

Specifying Parameters

  1. click on the deployed xData job to make it selected.
  2. Select the Component tab at the bottom of the screen and add as many items as necessary to the Context Param.
  3. Select a key for each parameter from the pull-down menu and enter a value for each. When entering the values, keep the following in mind.
    • Enclose string parameters in " and escape them appropriately (" -> \", \ -> \, etc.)
    • Structured parameters should be entered as JSON strings (e.g. "{\"key\"\"value\"}")

How to use xData jobs (data input/output)

In the case of an xData job with record data input/output, such as retrieving data from a table or inserting data into a table, the data is temporarily stored in a local Json format file, which is used as an intermediary to pass data to/from other Talend components.

Therefore, you must specify the destination path for the Json file as follows

xData job with output data (e.g. xdata_mapserv_eventdata)

  1. Specify the destination path for the output Json file in the parameter output_file_path of the target xData job.
  2. Place tFileInputJSON and connect to it with trigger On Component Ok from the previous xData job.
  3. In tFileInputJSON, set it to read data from the same destination path as the output_file_path specified earlier.
  4. After that, enter "*" in Loop Json query, set the schema of the output data in Edit schema, and specify the key name corresponding to each column in Mapping.

xData job with input data (e.g. xdata_loaderwebapi_add_record)

  1. Save the input data as a Json format file with tFileOutputJSON.
  2. Connect from tFileOutputJSON to the target xData job with trigger On Component Ok.
  3. Specify the same path as the Json file destination specified above for the input_file_path parameter of the xData job.

Note that (input) and (output) are given to xData jobs that correspond to input and output in the list below.

List of xData jobs

The following is a list of xData jobs.

The ones with the name *_init must be called first, because they set the connection information (endpoint, API Key, API Secret) required to use other xData jobs in the same lineage.

Otherwise, it is a wrapper for the corresponding named xData API. Please refer to the reference of each xData API for a description of the functions and parameters.

Event Data Distribution API (mapserv)

  • xdata_mapserv_init
  • xdata_mapserv_eventdata (output)
  • xdata_mapserv_riskmap (output)
  • xdata_mapserv_predictmap (output)
  • xdata_mapserv_contour (output)

Route discovery API (routing)

  • xdata_routing_init
  • xdata_routing_dijkstra (output)

Data Loader API (loaderwebapi)

  • xdata_loaderwebapi_init
  • xdata_loaderwebapi_add_record (input)
  • xdata_loaderwebapi_register_loader
  • xdata_loaderwebapi_init_record

Provenance API (xdata_prov)

  • xdata_prov_init
  • xdata_prov_begin_session
  • xdata_prov_begin_or_resume_session
  • xdata_prov_resume_session
  • xdata_prov_end_session
  • xdata_prov_get_session
  • xdata_prov_get_session_list
  • xdata_prov_get_session_token
  • xdata_prov_dump_session
  • xdata_prov_commit
  • xdata_prov_rollback
  • xdata_prov_new_ddc
  • xdata_prov_delete_ddc
  • xdata_prov_resolve_ddc
  • xdata_prov_get_ddcs
  • xdata_prov_set_ddc
  • xdata_prov_get_ddc_info
  • xdata_prov_get_ddc_columns
  • xdata_prov_get_ddc_records (output)
  • xdata_prov_put_ddc_records (input)
  • xdata_prov_exec_ddc
  • xdata_prov_cancel_ddc_activity
  • xdata_prov_count_ddc_records
  • xdata_prov_join_ddc
  • xdata_prov_merge_ddc
  • xdata_prov_ddc_exists
  • xdata_prov_get_ddc_activities
  • xdata_prov_ddc_processing_status
  • xdata_prov_get_ddc_last_activity
  • xdata_prov_process
  • xdata_prov_wait_ddc
  • xdata_prov_get_running_processes
  • xdata_prov_stop_running_process