Skip to content

Deploy Edge Environment

This document has been machine translated.

Operating environment

Prepare a VM or a physical server as the environment in which Edge itself will be installed (hereinafter referred to as "Edge VM").

Among the functions provided by Edge, the CRNN/LGBM analysis API requires a GPU, so in addition to the configuration in which the Edge VM itself is equipped with a GPU, a configuration in which a separate host with a GPU (hereinafter referred to as "GPU host") is used is supported.

The Edge VM must meet the following requirements:

  • CPU: Intel or AMD x86_64 architecture CPU with 2 cores/4 threads or more, generally within 5 years old
  • Memory: 4GB or more (8GB or more recommended)
  • Storage: 100GB or more
  • Network environment with HTTP/HTTPS connection to the Internet (proxy environment is acceptable)
  • OS: Ubuntu 18.04 LTS Server or Ubuntu 20.04 LTS Server
  • The user who installs the software must have root privileges on the OS.
  • Do not run services that have HTTP server functions (LISTEN to HTTP/HTTPS ports) such as Apache/Nginx, etc.

Restrictions

  • When using Ubuntu 20.04 LTS Server: If you are using dynamic IP address assignment by DHCP instead of a fixed IP address, the IP address will be changed during Edge installation (during the execution of environment-setup.sh below). If the installation is being performed on a terminal via an ssh connection, it appears that the installation is stopped midway on the terminal due to the IP address change (in reality, the installation process continues). Normally, dynamic IP address operation should not be used for actual operation purposes, but please be careful when using dynamic IP address operation for testing purposes, etc.

In addition, Edge VMs or GPU hosts with GPUs must meet the following requirements when using the CRNN/LGBM analysis API:

  • GPU: NVIDIA GPU (Linux x86_64 Driver Version >= 460.27.03)

In addition, when configuring the Edge VM and GPU host, the Edge VM must be able to connect to port number 30122 on the GPU host.

What to provide

The following files are provided.

Installer package

  • xdata-edge-base-X.X.tar.gz (replace X.X hereafter with the version number)

Setup procedures per configuration

For Edge VM stand-alone configuration, perform the following setup:

  • Common setup for Edge VM/GPU host
  • Edge VM setup

For Edge VM and GPU host configurations, perform the following setup:

  • Common setup for Edge VM/GPU Host (performed for both Edge VM and GPU Host)
  • Edge VM setup
  • GPU host setup
  • Edge VM - GPU host connection setup

Common setup for Edge VM/GPU host

Extract the installer package

Extract the installer package. In this example, the installer package is placed in the home directory of the installation user.

tar zxf xdata-edge-base-X.X.tar.gz

Navigate to the install directory under the extracted directory.

cd ~/xdata-edge-base-X.X/install

Setting up the base software

Install the various software required for xData Edge (snap, docker, microk8s). Run the following script.

sudo . /environment-setup.sh

If you are installing on a host under an HTTP/HTTPS proxy environment, enter the proxy settings for the questions during script execution.

Do you need HTTP/HTTPS proxy settings? [y/n]: y
Proxy's hostname/IP address: XXX.XXX.XXX.XXX
Proxy's port number: XXXX

Be sure to log out and log in once after script execution is complete.

Edge VM setup

Deploy Edge components

Run the following script to configure and deploy xData Edge components. Answer the questions during script execution according to the desired configuration.

./edge-configure.sh

The following is an example of input when "EvWH DB is used, GPU is not used on this host, and MMS DataServer is not used".

Do you need the EvWH DB?[y/n]: y
1: Use the GPU analysis feature on this host.
2: Use the GPU analysis feature on another host. Or do not use the GPU analysis feature.
Select a number [1-2]: 2
Do you want to use GPUs in the development environment?[y/n]: n
Do you need the MMS DataServer?[y/n]: n

Running verification

Enter the following command to verify that the xData Edge containers (microk8s Pod) are running.

kubectl get pods

If the STATUS of all Pods displayed are Running as shown below, they are up and running (it may take a few minutes to get up and running). Note that the list of Pods below may increase or decrease depending on the configuration you have selected.

NAME                                   READY   STATUS    RESTARTS   AGE
evwhapi-7694689789-2xn4h               1/1     Running   0          69d
dataloaderapi-7bf4744d65-9tpbv         1/1     Running   0          55d
webapi-endpoint-ffc7bf666-5cf4r        1/1     Running   0          54d
analysis-cpu-69c496fcc9-h4lfk          1/1     Running   0          54d
provenanceapi-6f9f7b65dd-29zss         1/1     Running   0          54d
jupyter-7645d64955-5cmhl               1/1     Running   0          54d
evwhdb01-7555c94c98-w7r2j              1/1     Running   0          13d
analysis-gpu-79d9bc9db-wjz58           1/1     Running   0          13d
nginx-796d7fdb6d-88jrm                 1/1     Running   0          18s
getfileapi-ff59c8bdc-2g879             1/1     Running   0          18s
filemanipulationapi-6b4d66568b-vc8bz   1/1     Running   0          18s

GPU host setup

Deploy Edge components

Run the following script to deploy components for xData Edge GPU host.

./gpu-analysis-configure.sh

Running verification

Enter the following command to verify that the xData Edge containers (microk8s Pod) are running.

kubectl get pods

If the STATUS of all Pods displayed are Running as shown below, they are up and running (it may take a few minutes to get up and running).

NAME                                   READY   STATUS    RESTARTS   AGE
analysis-gpu-79d9bc9db-wjz58           1/1     Running   0          13d

Edge VM - GPU host connection setup

Configure the connection settings on the Edge VM side: Set the hostname/IP address of the GPU host to host in xdata-edge-base-X.X/config/evwhdb01-deployment.yaml on the Edge VM. 

        env:
        - name: REMOTE_HOST
          value: 'root@host'

After configuration, run reconfigure.sh to reflect the settings.

cd ~/xdata-edge-base-X.X/install
./reconfigure.sh

Web API communication confirmation (for the configuration that selected the use of EvWH DB)

The Web API is exposed via HTTP/HTTPS on the Edge VM in the state up to the above section.
If you have selected the EvWH DB configuration, you can check the operation of the Web API with the following command.

curl -H "Content-type:application/json" -H "APIKey: 0df23f0d-44cb-4085-8ac9-158e6ced3056" -H "Secret: 4AzbY3pvfes33aow49nhjn3aUphv9zFWctcT" -X POST -d '{"id":1, "jsonrpc": "2.0", "method": "prov.get_session_list", "params": {"show_all":false}}' http://localhost/api/v1/provenance/jsonrpc

The Web API on xData Edge is working correctly if the following response is returned.

{"result": [], "id": 1, "jsonrpc": "2.0"}

Initial Setup

Create JupyterHub user

Run the following script to create a JupyterHub user.

cd ~/xdata-edge-base-X.X/tools
./jupyter-adduser.sh <username>

You will be asked for a password, set the user's password.

From a browser on another host with HTTPS access to the Edge VM, access the following URL and confirm your JupyterHub login with the Username/Password you created above.

https://<xData Edge IP address>/jupyter

Confirmation of initial installation API key (for the configuration that selected the use of EvWH DB)

When using EvWH DB, the initially installed tenant database evwh has API keys initially installed for each API that can be used during development. To check the API key/Secret, run the following script.

cd ~/xdata-edge-base-X.X/tools
./apikey-list.sh