Running a full DigiAssets stack

From DigiByte Wiki
Jump to: navigation, search

The full DigiAssets stack will include:

  • Minio
  • Postgresql
  • Node.js
  • nginx
  • DigiAssets-Metadata-Server
  • DigiAssets-Block-Explorer
  • DigiAssetsd
  • pm2 (optional)
  • certbot (LetsEncrypt)

 

Step 1: OS, presumptions and initial preparation

This guide presumes a completed installation on Ubuntu 18.04 (LTS) with the user "digibyte" that you have sudo access from.
It also presumes you are moderately familiar with server administration through SSH, and have full access to either the VM on which this will be all running or the bare-metal system, with a dedicated static IP address. If you are doing this from behind a corporate firewall or at home, it presumes you have the technical knowledge and the access to be able to forward the relevant ports.

There are no minimum CPU requirements, though you'll want 8GB of RAM (Core Wallet will use ~4GB on it's own) with 16GB+ being good for future-proofing, as well as 1TB drive-space free, despite it currently only using ~250GB for a full sync (prior to a Postgresql vacuum) as of December 2019.

This whole process is expected to only require around an hour in front of a computer (excluding sync time), but will likely take ~4-5 hours for DigiByte Core to sync, and then a few days for the DigiAssets-MetaData-Server to sync (Depending on your CPU etc).

If you have not installed Ubuntu, go through with it now. There is no need for any GUI, so the basic Ubuntu 18.04 server install is sufficient.

Once you have completed the installation, ssh in to the system as "digibyte" user.
If required, you may need to:

sudo apt-get update && sudo apt-get install openssh-server
sudo ufw allow 22


Step 2: Domain Name setup

First things first, to give time for everything to propogate, you'll want to create a domain name with your hosting provider. In this case we are using digiassets.digibyteservers.io
Create an A-Record and point it to the IP address of your Ubuntu server

You'll also want to create the following additional 3x additional A-records:
explorer.domainname.tld
metadata.domainname.tld
digiassetsd.domainname.tld

If you want to host the varying parts of your DigiAssets stack, you will want to ensure that relevant domain records are setup now for each of the above parts, though the scope of this guide will only account for it occurring on one system

You can also go through this document now and find / replace "domainname.tld" with yourdomain.com that you will be hosting it under.


Step 3: Installing digibyted

We're going to use the pre-made binaries, as these have all been through the gitian-build process. If you would like to compile your own, you may do-so following the instructions in src/build/unix.md, however that is outside of the scope of this document

We're first of all going to create the DigiByte config file:

mkdir ~/.digibyte
cat <<EOF > ~/.digibyte/digibyte.conf
rpcuser=user
rpcpassword=password
server=1
listen=1
daemon=1
txindex=1
rpcallowip=127.0.0.1
EOF

We will then download and run DigiByte version 7.17.2 (The current version at the time of writing, December 2019)

wget -c [https://github.com/digibyte/digibyte/releases/download/v7.17.2/digibyte-7.17.2-x86_64-linux-gnu.tar.gz https://github.com/digibyte/digibyte/releases/download/v7.17.2/digibyte-7.17.2-x86_64-linux-gnu.tar.gz] -O - | tar xz
./digibyte-7.17.2/bin/digibyted

You can follow the logs with:

tail -f ~/.digibyte/debug.log

This step will likely take ~2 hours on a moderately specced machine
When the "block height for reward is...." matches the latest block on DigiExplorer.info, you know it's fully sync'd.



Step 4: Installing node.js

We'll be using nvm as the Node Version Manager to install a specific version of node.js:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.2/install.sh | bash
export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"

We then want to install and use version 10.17.0 (LTS):

nvm install 10.17.0

We then run the following in case this isn't a fresh server that had an older version:

nvm use 10.17.0


Step 5: Installing postgres

Postgresql is what we will be using for the database, so we'll create a user and password. You should not use this default password, please for the love of all things secure change it.
Database Name: digiassets
Database User: digiassetsuser
Database Pass: ********

So first we will install postgres:

sudo apt install postgresql postgresql-contrib

We're then going to allow anything locally to authenticate (This needs to be locked down more later):

cat <<EOF > ~/pg_hba.conf
# Database administrative login by Unix domain socket
local   all             postgres                                peer
#"local" is for Unix domain socket connections only
local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            trust
# IPv6 local connections:
host    all             all             ::1/128                 trust
# reject all other connection attempts
host    all         all         0.0.0.0           0.0.0.0           reject
EOF
sudo mv ~/pg_hba.conf /etc/postgresql/10/main/

Then restart the postgres service:

sudo /etc/init.d/postgresql restart

Now run the following line-by-line to log you in as the postgres user, and create the database + permissions. Change only the password '******' (Just type what you want as a secure password instead into it):

sudo -i -u postgres
psql
create database digiassets;
create user digiassetsuser with encrypted password ‘******’;
grant all privileges on database digiassets to digiassetsuser;
\q
exit

You should now be logged out of postgres and logged in as the digibyte user again.


Step 6: Installing Minio

Rather than using Amazon S3 storage, we're instead going to run this all locally:

curl -O [https://dl.minio.io/server/minio/release/linux-amd64/minio https://dl.minio.io/server/minio/release/linux-amd64/minio]
chmod +x minio
sudo mv minio /usr/local/bin
sudo useradd -r minio-user -s /sbin/nologin
sudo chown minio-user:minio-user /usr/local/bin/minio

We will then create the relevant directories to hold our data buckets. You can adjust the location of this yourself if required.

sudo mkdir /usr/local/share/minio
sudo chown minio-user:minio-user /usr/local/share/minio

Next we will make a config file for minio. Replace the IP address on the MINIO_OPTS line and the password on the MINION_SECRET_KEY line with something more secure:

sudo mkdir /etc/minio
sudo chown minio-user:minio-user /etc/minio
cat <<EOF > minio-conf
MINIO_VOLUMES="/usr/local/share/minio/"
MINIO_OPTS="-C /etc/minio --address 192.168.0.0:9000"
MINIO_ACCESS_KEY="minio"
MINIO_SECRET_KEY="*****"
EOF
sudo mv minio-conf /etc/default/minio

We want minio to start on boot:

curl -O [https://raw.githubusercontent.com/minio/minio-service/master/linux-systemd/minio.service https://raw.githubusercontent.com/minio/minio-service/master/linux-systemd/minio.service]
sudo mv minio.service /etc/systemd/system
sudo systemctl daemon-reload
sudo systemctl enable minio
sudo systemctl start minio
sudo ufw allow 9000

Check and ensure that minio has started correctly:

sudo systemctl status minio

(Push "q" to stop seeing the status)

You should see "Active: active (running)"
We will temporarily be using a browser now to complete the minio setup. Go to http://192.168.0.0:9000 and log in with the access key and secret key you defined in your minio conf above.
Once successfully logged in, click on the "+" down the bottom-right, click on "Create bucket" and then call it "digiassets". It should immediately appear on the left-hand column.
We are now done with our browser, back to the ssh session.

We're now going to edit the minio opts and change the IP address:

sudo nano -w /etc/default/minio

Set it to:

MINIO_OPTS="-C /etc/minio --address 127.0.0.1:9000"

Ctrl + X, y, <enter>

sudo systemctl restart minio


Step 7: Installing DigiAssets-Metadata-Server

cd ~/
git clone [https://github.com/DigiByte-Core/DigiAssets-Metadata-Server https://github.com/DigiByte-Core/DigiAssets-Metadata-Server]
cd DigiAssets-Metadata-Server
npm install
nano -w config/properties_development.conf

You will want to enable minio accessKey, secretKey, change the bucket to "digiassets", and the http_port to 9002


Step 8: Installing DigiAssets-Block-Explorer

cd ~/
git clone [https://github.com/DigiByte-Core/DigiAssets-Block-Explorer https://github.com/DigiByte-Core/DigiAssets-Block-Explorer]
cd DigiAssets-Block-Explorer
npm install
nano -w config/properties_development.conf

You will want to set the http_port=9003, as well as the [db] leaving #debug commented out

dir=node_modules/digiasset-block-parser/models/
[bitcoin_rpc]
ssl=false
url=localhost
path=/
username=user
password=password
port=14022
timeout=3000

We also need to ensure that we're going to correctly handle Burn instructions, so run:

nano +16 ~/DigiAssets-Block-Explorer/node_modules/digiasset-block-parser/models/assetstransactions.js

Change from (IF yours still has the older types):

type: DataTypes.ENUM('issuance', 'transfer'),

to:

type: DataTypes.ENUM('issuance', 'transfer', 'burn'),

(This will be committed in a future update)


Step 9: Installing digiassetsd

cd ~/
git clone [https://github.com/DigiByte-Core/digiassetsd https://github.com/DigiByte-Core/digiassetsd]
cd digiassetsd
npm install
nano -w config.js

You will want to modify the machineurl, bitcoind-port (14022) blockexplorer and metadataserver URLs


Step 10: Installing pm2

We will use pm2 to look after our node.js services for logging / control etc

cd ~/
npm install -g pm2
cd ~/DigiAssets-Metadata-Server
pm2 start npm --name "metadata" -- start
cd ~/DigiAssets-Block-Explorer
pm2 start npm --name "explorer" -- start
cd ~/digiassetsd
pm2 start npm --name "digiassetsd" -- start

 


Step 11: Installing nginx webserver

We will then install nginx and allow it out the firewall

sudo apt install nginx
sudo ufw allow 'Nginx Full'

Then lots of copy / paste for each of the instances, you should find / replace "domainname.tld" with your own domain if you haven't already:

cat <<EOF > ~/digiassetsd.nginx
server {
        listen   80;
        server_name digiassetsd.domainname.tld;
        location / {
            proxy_pass [http://127.0.0.1:8080 http://127.0.0.1:8080];
            proxy_http_version 1.1;
            proxy_set_header Upgrade \$http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header Host \$host;
        }
}
EOF

cat <<EOF > ~/metadata.nginx
server {
        listen   80;
        server_name metadata.domainname.tld;
        location / {
            proxy_pass [http://127.0.0.1:9002 http://127.0.0.1:9002];
            proxy_http_version 1.1;
            proxy_set_header Upgrade \$http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header Host \$host;
        }
}
EOF

cat <<EOF > ~/explorer.nginx
server {
        listen   80;
        server_name explorer.domainname.tld;
        location / {
            proxy_pass [http://127.0.0.1:9003 http://127.0.0.1:9003];
            proxy_http_version 1.1;
            proxy_set_header Upgrade \$http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header Host \$host;
        }
}
EOF

sudo mv digiassetsd.nginx /etc/nginx/sites-available/digiassetsd.domainname.tld
sudo mv metadata.nginx /etc/nginx/sites-available/metadata.domainname.tld
sudo mv explorer.nginx /etc/nginx/sites-available/explorer.domainname.tld
sudo ln -s /etc/nginx/sites-available/* /etc/nginx/sites-enabled/

 


Step 12: Installing certbot for LetsEncrypt

When prompted, select option 2 to Redirect all requests to HTTPS:

sudo add-apt-repository ppa:certbot/certbot
sudo apt install python-certbot-nginx
sudo certbot --nginx -d digiassets.domainname.tld
sudo certbot --nginx -d explorer.domainname.tld
sudo certbot --nginx -d metadata.domainname.tld
sudo service nginx restart