Introduction of the Module System

Note

This document has been machine translated.

Machine Configuration for the Module System

To use the module system, both xData Edge and a module worker must be operational.
Possible machine arrangements include:

• Deploying xData Edge and the module worker on the same machine
• Deploying xData Edge and the module worker on separate machines

This guide describes installation when both components are placed on a single machine.

Module Worker Machine Environment

The module worker host must satisfy the following requirements:

• SSH access (password or public‑key authentication) enabled
• If GPU usage on containers is required, NVIDIA Container Toolkit and NVIDIA GPU drivers must be installed

Module Worker Installation

Navigate to the xdata-edge-base/tools directory and execute module-exec-configure.sh:

sudo ./module-exec-configure.sh

During execution, the script prompts for several inputs.

1. Verification of IP Address or Hostname

Script prompt:

Auto detected IP address of this host: <detected_IP>
It must be an IP address or hostname that can be resolved from the network to which this host belongs.
Is the above IP address/hostname correct? [y/n]:

• The script automatically detects the host’s IP address.
• If the detected value is correct, type y.
• If incorrect, type n and manually enter the proper IP address or hostname.

Example of manual entry:

Enter the IP address or hostname of this host: 192.168.254.1
Confirmation: 192.168.254.1

2. Entry of Module Executor Username

Script prompt:

Enter the module executor username (new or existing user): 

• Provide a username for executing modules.
• If an existing user is entered, that user will be used.
• If a new username is entered, the script will create it (recommended unless otherwise required).

Confirmation example:

Confirmation: <entered_username>
Is the above username correct? [y/n]:

• If the username is acceptable, type y.
• To correct it, type n and re‑enter.

When creating a new user:

User <entered_username> does not exist. Are you sure you want to create it? [y/n]:

• If the username is absent, confirmation is requested.
• Type y to create it.
• Type n to cancel creation.

Module Worker Functionality Verification

Use the calculate-module from the Information Asset Repository to confirm correct installation of the module system.
After obtaining calculate-module, follow the “Module Installation” section in its README to install the module.

The module system and installed module can be verified with curl commands.
The following steps cover creation of input data, module execution, and result retrieval.

1. Creation of input table
2. Registration of two records
3. Creation of a citation for the input dataset
4. Execution of the add method
5. Creation of a citation for the output dataset
6. Retrieval of output table data

All steps, including citation creation, are mandatory; skipping any step causes errors.
When executing the curl commands, replace APIKey and Secret with actual values.
If the API host differs from the default, modify http://localhost/api/v2/dataset/jsonrpc accordingly.

1. Creation of Input Table

curl -X POST http://localhost/api/v2/dataset/jsonrpc \
  -H "Content-Type: application/json" \
  -H "APIKey: <API_KEY>" \
  -H "Secret: <API_SECRET>" \
  -d '{
    "jsonrpc": "2.0",
    "method": "dataset.create",
    "params": {
      "dataset_name": "in_sample_dataset",
      "columns": [
        {"column_name": "id", "data_type": "int"},
        {"column_name": "num1", "data_type": "int"},
        {"column_name": "num2", "data_type": "int"}
      ]
    },
    "id": "1"
  }'

A successful execution returns a JSON response containing dataset information.

2. Registration of Two Records

curl -X POST http://localhost/api/v2/dataset/jsonrpc \
  -H "Content-Type: application/json" \
  -H "APIKey: <API_KEY>" \
  -H "Secret: <API_SECRET>" \
  -d '{
    "jsonrpc": "2.0",
    "method": "dataset.record.put",
    "params": {
      "dataset_name": "in_sample_dataset",
      "records": [[1, 3, 5], [2, 12, 88]]
    },
    "id": "2"
  }'

3. Creation of Citation for Input Dataset

curl -X POST http://localhost/api/v2/dataset/jsonrpc \
  -H "Content-Type: application/json" \
  -H "APIKey: <API_KEY>" \
  -H "Secret: <API_SECRET>" \
  -d '{
    "jsonrpc": "2.0",
    "method": "dataset.citation.create",
    "params": {
      "citation_label": "in_sample_dataset",
      "source": "in_sample_dataset"
    },
    "id": "3"
  }'

A successful execution returns a JSON response containing citation information.

4. Execution of add Method

curl -X POST http://localhost/api/v2/module/jsonrpc \
  -H "Content-Type: application/json" \
  -H "APIKey: <API_KEY>" \
  -H "Secret: <API_SECRET>" \
  -d '{
    "jsonrpc": "2.0",
    "method": "module.exec",
    "params": {
      "module": "calculate-module",
      "method": "add",
      "input": {
        "numbers": {
          "data_ddc": "in_sample_dataset"
        }
      },
      "output": {
        "result": {
          "data_ddc": "out_sample_result",
          "output_mode": "append"
        }
      }
    },
    "id": "4"
  }'

A successful execution returns a response such as {"result": true, "id": "4", "jsonrpc": "2.0"}.

5. Creation of Citation for Output Dataset

curl -X POST http://localhost/api/v2/dataset/jsonrpc \
  -H "Content-Type: application/json" \
  -H "APIKey: <API_KEY>" \
  -H "Secret: <API_SECRET>" \
  -d '{
    "jsonrpc": "2.0",
    "method": "dataset.citation.create",
    "params": {
      "citation_label": "out_sample_result",
      "source": "out_sample_result"
    },
    "id": "5"
  }'

6. Retrieval of Output Table Data

curl -X POST http://localhost/api/v2/dataset/jsonrpc \
  -H "Content-Type: application/json" \
  -H "APIKey: <API_KEY>" \
  -H "Secret: <API_SECRET>" \
  -d '{
    "jsonrpc": "2.0",
    "method": "dataset.record.get",
    "params": {
      "citation_label": "out_sample_result"
    },
    "id": "6"
  }'

A successful execution returns:

{"result":[{"id":1,"result":8},{"id":2,"result":100}],"id":"6","jsonrpc":"2.0"}

The two records registered in step 2 yield results 3 + 5 = 8 (id = 1) and 12 + 88 = 100 (id = 2).

id	num1	num2
1	3	5
2	12	88

For detailed API specifications, refer to the following xData documentation:

• Dataset API
• Module API