Running a full DigiAssets stack

From DigiByte Wiki
Revision as of 01:47, 29 February 2020 by Dgb chilling (talk | contribs) (Fixed up code blocks)
Jump to navigation Jump to search

The full DigiAssets stack will include:

  • Minio
  • Postgresql
  • Node.js
  • nginx
  • DigiAssets-Metadata-Server
  • DigiAssets-Block-Explorer
  • DigiAssetsd
  • pm2 (optional)
  • certbot (LetsEncrypt)

Step 1: OS, presumptions and initial preparation

This guide presumes a completed installation on Ubuntu 18.04 (LTS) with the user "digibyte" that you have sudo access from.It also presumes you are moderately familiar with server administration through SSH, and have full access to either the VM on which this will be all running or the bare-metal system, with a dedicated static IP address. If you are doing this from behind a corporate firewall or at home, it presumes you have the technical knowledge and the access to be able to forward the relevant ports.There are no minimum CPU requirements, though you'll want 8GB of RAM (Core Wallet will use ~4GB on it's own) with 16GB+ being good for future-proofing, as well as 1TB drive-space free, despite it currently only using ~250GB for a full sync (prior to a Postgresql vacuum) as of December 2019.This whole process is expected to only require around an hour in front of a computer (excluding sync time), but will likely take ~4-5 hours for DigiByte Core to sync, and then a few days for the DigiAssets-MetaData-Server to sync (Depending on your CPU and *especially* if you have a high-speed SSD etc).If you have not installed Ubuntu, go through with it now. There is no need for any GUI, so the basic Ubuntu 18.04 server install is sufficient.Once you have completed the installation, ssh in to the system as "digibyte" user.If required, you may need to:

sudo apt-get update && sudo apt-get install openssh-server<br  />sudo ufw allow 22

Step 2: Domain Name setup

First things first, to give time for everything to propogate, you'll want to create a domain name with your hosting provider. In this case we are using digiassets.digibyteservers.io


Create an A-Record and point it to the IP address of your Ubuntu server


You'll also want to create the following additional 3x additional A-records:
explorer.domainname.tld
metadata.domainname.tld
digiassetsd.domainname.tld


If you want to host the varying parts of your DigiAssets stack, you will want to ensure that relevant domain records are setup now for each of the above parts, though the scope of this guide will only account for it occurring on one system


You can also go through this document now and find / replace "domainname.tld" with yourdomain.com that you will be hosting it under.

Step 3: Installing digibyted

We're going to use the pre-made binaries, as these have all been through the gitian-build process. If you would like to compile your own, you may do-so following the instructions in src/build/unix.md, however that is outside of the scope of this document


We're first of all going to create the DigiByte config file:


mkdir ~/.digibyte<br  />cat <<eof> ~/.digibyte/digibyte.conf<br  />rpcuser=user<br  />rpcpassword=password<br  />server=1<br  />listen=1<br  />daemon=1<br  />txindex=1<br  />rpcallowip=127.0.0.1<br  />EOF


We will then download and run DigiByte version 7.17.2 (The current version at the time of writing, December 2019)


wget -c https://github.com/digibyte/digibyte/releases/download/v7.17.2/digibyte-7.17.2-x86_64-linux-gnu.tar.gz -O - | tar xz<br  />./digibyte-7.17.2/bin/digibyted


You can follow the logs with:


tail -f ~/.digibyte/debug.log


This step will likely take ~4 hours on a moderately specced machine


When the "block height for reward is...." matches the latest block on DigiExplorer.info, you know it's fully sync'd.



Step 4: Installing node.js

We'll be using nvm as the Node Version Manager to install a specific version of node.js:


curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.2/install.sh | bash<br  />export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"<br  />[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"


We then want to install and use version 10.17.0 (LTS):


nvm install 10.17.0


We then run the following in case this isn't a fresh server that had an older version:


nvm use 10.17.0



Step 5: Installing postgres

Postgresql is what we will be using for the database, so we'll create a user and password. You should not use this default password, please for the love of all things secure change it.


Database Name: digiassets
Database User: digiassetsuser
Database Pass: ********


So first we will install postgres:


sudo apt install postgresql postgresql-contrib


We're then going to allow anything locally to authenticate (This needs to be locked down more later):


cat <<eof> ~/pg_hba.conf<br  /># Database administrative login by Unix domain socket<br  />local all postgres peer<br  />#"local" is for Unix domain socket connections only<br  />local all all trust<br  /># IPv4 local connections:<br  />host all all 127.0.0.1/32 trust<br  /># IPv6 local connections:<br  />host all all  ::1/128 trust<br  /># reject all other connection attempts<br  />host all all 0.0.0.0 0.0.0.0 reject<br  />EOF


sudo mv ~/pg_hba.conf /etc/postgresql/10/main/


Then restart the postgres service:


sudo /etc/init.d/postgresql restart


Now run the following line-by-line to log you in as the postgres user, and create the database + permissions.


Change only the password '******' (Just type what you want as a secure password instead into it):


sudo -i -u postgres<br  />psql<br  />create database digiassets;<br  />create user digiassetsuser with encrypted password ‘******’;<br  />grant all privileges on database digiassets to digiassetsuser;<br  />\q<br  />exit


You should now be logged out of postgres and logged in as the digibyte user again.



Step 6: Installing Minio

Rather than using Amazon S3 storage, we're instead going to run this all locally:


curl -O https://dl.minio.io/server/minio/release/linux-amd64/minio<br  />chmod +x minio<br  />sudo mv minio /usr/local/bin<br  />sudo useradd -r minio-user -s /sbin/nologin<br  />sudo chown minio-user:minio-user /usr/local/bin/minio


We will then create the relevant directories to hold our data buckets. You can adjust the location of this yourself if required.


sudo mkdir /usr/local/share/minio<br  />sudo chown minio-user:minio-user /usr/local/share/minio


Next we will make a config file for minio. Replace the IP address on the MINIO_OPTS line and the password on the MINION_SECRET_KEY line with something more secure:


sudo mkdir /etc/minio<br  />sudo chown minio-user:minio-user /etc/minio<br  />cat <<eof> minio-conf<br  />MINIO_VOLUMES="/usr/local/share/minio/"<br  />MINIO_OPTS="-C /etc/minio --address 127.0.0.1:9000"<br  />MINIO_ACCESS_KEY="minio"<br  />MINIO_SECRET_KEY="*****"<br  />EOF<br  />sudo mv minio-conf /etc/default/minio


We want minio to start on boot:


curl -O https://raw.githubusercontent.com/minio/minio-service/master/linux-systemd/minio.service<br  />sudo mv minio.service /etc/systemd/system<br  />sudo systemctl daemon-reload<br  />sudo systemctl enable minio<br  />sudo systemctl start minio<br  />sudo ufw allow 9000


Check and ensure that minio has started correctly:


sudo systemctl status minio


(Push "q" to stop seeing the status)


You should see "Active: active (running)"


We will temporarily be using a browser now to complete the minio setup. Go to http://127.0.0.1:9000 and log in with the access key and secret key you defined in your minio conf above.


Once successfully logged in, click on the "+" down the bottom-right, click on "Create bucket" and then call it "digiassets". It should immediately appear on the left-hand column.


We are now done with our browser, back to the ssh session.


We're now going to edit the minio opts and change the IP address:


sudo nano -w /etc/default/minio


Set it to:


MINIO_OPTS="-C /etc/minio --address 127.0.0.1:9000"


Ctrl + X, y, <enter>


sudo systemctl restart minio



Step 7: Installing DigiAssets-Metadata-Server

cd ~/<br  />git clone https://github.com/DigiByte-Core/DigiAssets-Metadata-Server<br  />cd DigiAssets-Metadata-Server<br  />npm install<br  />nano -w config/properties_development.conf


You will want to enable minio accessKey, secretKey, change the bucket to "digiassets", and the http_port to 9002



Step 8: Installing DigiAssets-Block-Explorer

cd ~/<br  />git clone https://github.com/DigiByte-Core/DigiAssets-Block-Explorer<br  />cd DigiAssets-Block-Explorer<br  />npm install<br  />nano -w config/properties_development.conf


You will want to set the http_port=9003, as well as the [db] leaving #debug commented out


<pre>dir=node_modules/digiasset-block-parser/models/<br  />[bitcoin_rpc]<br  />ssl=false<br  />url=localhost<br  />path=/<br  />username=user<br  />password=password<br  />port=14022<br  />timeout=3000


We also need to ensure that we're going to correctly handle Burn instructions, so run:


nano +16 ~/DigiAssets-Block-Explorer/node_modules/digiasset-block-parser/models/assetstransactions.js


Change from (IF yours still has the older types):


type: DataTypes.ENUM('issuance', 'transfer'),


to:


type: DataTypes.ENUM('issuance', 'transfer', 'burn'),


(This will be committed in a future update)



Step 9: Installing digiassetsd

cd ~/<br  />git clone https://github.com/DigiByte-Core/digiassetsd<br  />cd digiassetsd<br  />npm install<br  />nano -w config.js


You will want to modify the machineurl, bitcoind-port (14022) blockexplorer and metadataserver URLs



Step 10: Installing pm2

We will use pm2 to look after our node.js services for logging / control etc


cd ~/<br  />npm install -g pm2<br  />cd ~/DigiAssets-Block-Explorer<br  />pm2 start npm --name "explorer" -- start<br  />#you may want to pause here for a week while explorer syncs<br  />cd ~/DigiAssets-Metadata-Server<br  />pm2 start npm --name "metadata" -- start<br  />cd ~/digiassetsd<br  />pm2 start npm --name "digiassetsd" -- start



Step 11: Installing nginx webserver

We will then install nginx and allow it out the firewall


sudo apt install nginx<br  />sudo ufw allow 'Nginx Full'


Then lots of copy / paste for each of the instances, you should find / replace "domainname.tld" with your own domain if you haven't already:


cat <<eof> ~/digiassetsd.nginx<br  />server {<br  />listen 80;<br  />server_name digiassetsd.domainname.tld;<br  />location / {<br  />proxy_pass [http://127.0.0.1:8080];<br  />proxy_http_version 1.1;<br  />proxy_set_header Upgrade \$http_upgrade;<br  />proxy_set_header Connection "upgrade";<br  />proxy_set_header Host \$host;<br  />}<br  />}<br  />EOF


cat <<eof> ~/metadata.nginx<br  />server {<br  />listen 80;<br  />server_name metadata.domainname.tld;<br  />location / {<br  />proxy_pass [http://127.0.0.1:9002];<br  />proxy_http_version 1.1;<br  />proxy_set_header Upgrade \$http_upgrade;<br  />proxy_set_header Connection "upgrade";<br  />proxy_set_header Host \$host;<br  />}<br  />}<br  />EOF


cat <<eof> ~/explorer.nginx<br  />server {<br  />listen 80;<br  />server_name explorer.domainname.tld;<br  />location / {<br  />proxy_pass [http://127.0.0.1:9003];<br  />proxy_http_version 1.1;<br  />proxy_set_header Upgrade \$http_upgrade;<br  />proxy_set_header Connection "upgrade";<br  />proxy_set_header Host \$host;<br  />}<br  />}<br  />EOF


sudo mv digiassetsd.nginx /etc/nginx/sites-available/digiassetsd.domainname.tld<br  />sudo mv metadata.nginx /etc/nginx/sites-available/metadata.domainname.tld<br  />sudo mv explorer.nginx /etc/nginx/sites-available/explorer.domainname.tld<br  />sudo ln -s /etc/nginx/sites-available/* /etc/nginx/sites-enabled/



Step 12: Installing certbot for LetsEncrypt

When prompted, select option 2 to Redirect all requests to HTTPS:


sudo add-apt-repository ppa:certbot/certbot<br  />sudo apt install python-certbot-nginx<br  />sudo certbot --nginx -d digiassets.domainname.tld<br  />sudo certbot --nginx -d explorer.domainname.tld<br  />sudo certbot --nginx -d metadata.domainname.tld<br  />sudo service nginx restart</eof></eof></eof></enter></eof></eof>