Skip to content

Install Module Mechanism

Note

This document has been machine translated.

To use the module mechanism, it is necessary to configure the following settings in the xData Edge installer before installing xData Edge.

Example of machine configuration to install the module mechanism

As an example of the configuration of a machine running the module mechanism and xData Edge

  • Both the Module Organization and xData Edge are deployed on the same machine.
  • Place the Module Organization and xData Edge on separate machines.

These are two possible configurations.

In either pattern, SSH connection is made from the machine running xData Edge to run the module. Even if the module and xData Edge are deployed on the same machine, SSH connection will be made via localhost to run the module. In this case, you need to configure the local machine where xData Edge is installed to allow SSH connections.
Also, SSH connection does not support public key authentication, so you need to configure SSH connection with password authentication.

Environment of the machine where the module mechanism is installed

The machine on which the module is to be executed (the machine to which SSH connection is to be made) must meet the following requirements

  • SSH connection must be available with password authentication
  • Docker and Docker Compose must be installed.
  • To use a GPU on the container, the NVIDIA Container Toolkit and NVIDIA GPU drivers must be installed.

The xData Edge installer will install Docker, Docker Compose, NVIDIA Container Toolkit, and NVIDIA GPU drivers in the deployment environment.
If you wish to run the Algorithm Module on other execution environments, you will need to install them separately.

Install xData Edge with module mechanism

Configuration for dataproc api

To run the dataproc api for use with the module mechanism, you will need to configure an SSH worker. Please add or change the environment variable settings in manifest/base/evwh/provenanceapi/deployment.yaml of the installer according to your machine configuration. Please refer to the section Setting Environment Variables for the environment variables to be set.
After completing the settings, install xData Edge.

Checking for dataproc api communication

The following commands can be used to check the communication with the dataproc api (apart from checking that the SSH worker is properly configured). (This is separate from checking if the SSH worker is properly configured.) 

curl -H "Content-type:application/json" -H "APIKey: 14139d71-8ca2-4c35-88a2-66d3fc77bd4c" -H "Secret: iBbBVb83iSdWP8muFAAcwlK1OsJosYnWHAKE" -X POST -d '{"id":1, "jsonrpc": "2.0", "method": "dataproc.get_server_version","params": {}}' http://localhost/api/v1/dataproc/jsonrpc
If the following response is returned, you can confirm that the dataproc api is working. 
{"result": "0.0.2", "id": 1, "jsonrpc": "2.0"}

Others

If you wish to use xData Edge by updating existing xData Edge instead of installing a new one, please contact us separately as updating the DB is required.

Environment variable settings

The environment variables to be set when operating the module execution mechanism are as follows.

  • WORKER_SSH_HOST
    • Summary: Hostname to connect to on the machine running the Docker module.
    • Example: localhost
    • Default value:
  • WORKER_SSH_PORT
    • Summary: Port number to connect to on the machine running the Docker module.
    • Example: 22
    • Default value: 22
  • WORKER_SSH_USER
    • Summary: User name of the machine running the Docker module to connect to.
    • Example: ubuntu
    • Default value:
  • WORKER_SSH_PASS
    • Summary: Connection password for the machine running the Docker module
    • Example: password
    • Default value:
  • WORKER_ROOT_DIR
    • Summary: The full path of the directory containing the modules to run (in the case of this Example, xdata-algo-modules contains the set of modules you want to run)
    • Example: /home/ubuntu/work/xdata-algo-modules
    • Default value: .
  • WORKER_DATA_DIR
    • Summary: Full path of the directory where temporary data is stored when the module is executed (in the case of this Example, a directory with UUID notation is created under runs)
    • Example: /home/ubuntu/work/runs
    • Default value:
  • WORKER_MOUNT_DIR
    • Summary: Path of the directory to mount on the Docker module
    • Example: /work/runs
    • Default value: /work/runs
  • WORKER_SESSIONS_DIR
    • Summary: Path of the session directory
    • Example: /home/ubuntu/work/sessions
    • Default value:
  • WORKER_APP_DIR
    • Summary: APP directory path
    • Example: /home/ubuntu/work/app
    • Default value: