Difference between revisions of "Running a full DigiAssets stack"

From DigiByte Wiki
Jump to navigation Jump to search
(Initial commit)
(No difference)

Revision as of 01:40, 29 February 2020

The full DigiAssets stack will include:

  • Minio
  • Postgresql
  • Node.js
  • nginx
  • DigiAssets-Metadata-Server
  • DigiAssets-Block-Explorer
  • DigiAssetsd
  • pm2 (optional)
  • certbot (LetsEncrypt)

Step 1: OS, presumptions and initial preparation

This guide presumes a completed installation on Ubuntu 18.04 (LTS) with the user "digibyte" that you have sudo access from.It also presumes you are moderately familiar with server administration through SSH, and have full access to either the VM on which this will be all running or the bare-metal system, with a dedicated static IP address. If you are doing this from behind a corporate firewall or at home, it presumes you have the technical knowledge and the access to be able to forward the relevant ports.There are no minimum CPU requirements, though you'll want 8GB of RAM (Core Wallet will use ~4GB on it's own) with 16GB+ being good for future-proofing, as well as 1TB drive-space free, despite it currently only using ~250GB for a full sync (prior to a Postgresql vacuum) as of December 2019.This whole process is expected to only require around an hour in front of a computer (excluding sync time), but will likely take ~4-5 hours for DigiByte Core to sync, and then a few days for the DigiAssets-MetaData-Server to sync (Depending on your CPU and *especially* if you have a high-speed SSD etc).If you have not installed Ubuntu, go through with it now. There is no need for any GUI, so the basic Ubuntu 18.04 server install is sufficient.Once you have completed the installation, ssh in to the system as "digibyte" user.If required, you may need to:zzzsudo apt-get update && sudo apt-get install openssh-server
sudo ufw allow 22

Step 2: Domain Name setup

First things first, to give time for everything to propogate, you'll want to create a domain name with your hosting provider. In this case we are using digiassets.digibyteservers.io


Create an A-Record and point it to the IP address of your Ubuntu server


You'll also want to create the following additional 3x additional A-records:
explorer.domainname.tld
metadata.domainname.tld
digiassetsd.domainname.tld


If you want to host the varying parts of your DigiAssets stack, you will want to ensure that relevant domain records are setup now for each of the above parts, though the scope of this guide will only account for it occurring on one system


You can also go through this document now and find / replace "domainname.tld" with yourdomain.com that you will be hosting it under.

Step 3: Installing digibyted

We're going to use the pre-made binaries, as these have all been through the gitian-build process. If you would like to compile your own, you may do-so following the instructions in src/build/unix.md, however that is outside of the scope of this document


We're first of all going to create the DigiByte config file:


zzzmkdir ~/.digibyte
cat <<eof> ~/.digibyte/digibyte.conf
rpcuser=user
rpcpassword=password
server=1
listen=1
daemon=1
txindex=1
rpcallowip=127.0.0.1
EOF


We will then download and run DigiByte version 7.17.2 (The current version at the time of writing, December 2019)


zzzwget -c https://github.com/digibyte/digibyte/releases/download/v7.17.2/digibyte-7.17.2-x86_64-linux-gnu.tar.gz -O - | tar xz
./digibyte-7.17.2/bin/digibyted


You can follow the logs with:


zzztail -f ~/.digibyte/debug.log


This step will likely take ~4 hours on a moderately specced machine


When the "block height for reward is...." matches the latest block on DigiExplorer.info, you know it's fully sync'd.



</eof>

Step 4: Installing node.js

We'll be using nvm as the Node Version Manager to install a specific version of node.js:


zzzcurl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.2/install.sh | bash
export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"


We then want to install and use version 10.17.0 (LTS):


zzznvm install 10.17.0


We then run the following in case this isn't a fresh server that had an older version:


zzznvm use 10.17.0



Step 5: Installing postgres

Postgresql is what we will be using for the database, so we'll create a user and password. You should not use this default password, please for the love of all things secure change it.


Database Name: digiassets
Database User: digiassetsuser
Database Pass: ********


So first we will install postgres:


zzzsudo apt install postgresql postgresql-contrib


We're then going to allow anything locally to authenticate (This needs to be locked down more later):


zzzcat <<eof> ~/pg_hba.conf
# Database administrative login by Unix domain socket
local all postgres peer
#"local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 trust
# reject all other connection attempts
host all all 0.0.0.0 0.0.0.0 reject
EOF


zzzsudo mv ~/pg_hba.conf /etc/postgresql/10/main/


Then restart the postgres service:


zzzsudo /etc/init.d/postgresql restart


Now run the following line-by-line to log you in as the postgres user, and create the database + permissions.


Change only the password '******' (Just type what you want as a secure password instead into it):


zzzsudo -i -u postgres
psql
create database digiassets;
create user digiassetsuser with encrypted password ‘******’;
grant all privileges on database digiassets to digiassetsuser;
\q
exit


zzzYou should now be logged out of postgres and logged in as the digibyte user again.



Step 6: Installing Minio

Rather than using Amazon S3 storage, we're instead going to run this all locally:


zzzcurl -O https://dl.minio.io/server/minio/release/linux-amd64/minio
chmod +x minio
sudo mv minio /usr/local/bin
sudo useradd -r minio-user -s /sbin/nologin
sudo chown minio-user:minio-user /usr/local/bin/minio


We will then create the relevant directories to hold our data buckets. You can adjust the location of this yourself if required.


zzzsudo mkdir /usr/local/share/minio
sudo chown minio-user:minio-user /usr/local/share/minio


Next we will make a config file for minio. Replace the IP address on the MINIO_OPTS line and the password on the MINION_SECRET_KEY line with something more secure:


zzzsudo mkdir /etc/minio
sudo chown minio-user:minio-user /etc/minio
cat <<eof> minio-conf
MINIO_VOLUMES="/usr/local/share/minio/"
MINIO_OPTS="-C /etc/minio --address 127.0.0.1:9000"
MINIO_ACCESS_KEY="minio"
MINIO_SECRET_KEY="*****"
EOF
sudo mv minio-conf /etc/default/minio


We want minio to start on boot:


zzzcurl -O https://raw.githubusercontent.com/minio/minio-service/master/linux-systemd/minio.service
sudo mv minio.service /etc/systemd/system
sudo systemctl daemon-reload
sudo systemctl enable minio
sudo systemctl start minio
sudo ufw allow 9000


Check and ensure that minio has started correctly:


zzzsudo systemctl status minio


(Push "q" to stop seeing the status)


You should see "Active: active (running)"


We will temporarily be using a browser now to complete the minio setup. Go to http://127.0.0.1:9000 and log in with the access key and secret key you defined in your minio conf above.


Once successfully logged in, click on the "+" down the bottom-right, click on "Create bucket" and then call it "digiassets". It should immediately appear on the left-hand column.


We are now done with our browser, back to the ssh session.


We're now going to edit the minio opts and change the IP address:


zzzsudo nano -w /etc/default/minio


Set it to:


zzzMINIO_OPTS="-C /etc/minio --address 127.0.0.1:9000"


Ctrl + X, y, <enter>


zzzsudo systemctl restart minio



Step 7: Installing DigiAssets-Metadata-Server

zzzcd ~/
git clone https://github.com/DigiByte-Core/DigiAssets-Metadata-Server
cd DigiAssets-Metadata-Server
npm install
nano -w config/properties_development.conf


You will want to enable minio accessKey, secretKey, change the bucket to "digiassets", and the http_port to 9002



Step 8: Installing DigiAssets-Block-Explorer

zzzcd ~/
git clone https://github.com/DigiByte-Core/DigiAssets-Block-Explorer
cd DigiAssets-Block-Explorer
npm install
nano -w config/properties_development.conf


You will want to set the http_port=9003, as well as the [db] leaving #debug commented out


zzzdir=node_modules/digiasset-block-parser/models/
[bitcoin_rpc]
ssl=false
url=localhost
path=/
username=user
password=password
port=14022
timeout=3000


We also need to ensure that we're going to correctly handle Burn instructions, so run:


zzznano +16 ~/DigiAssets-Block-Explorer/node_modules/digiasset-block-parser/models/assetstransactions.js


Change from (IF yours still has the older types):


zzztype: DataTypes.ENUM('issuance', 'transfer'),


to:


zzztype: DataTypes.ENUM('issuance', 'transfer', 'burn'),


(This will be committed in a future update)



Step 9: Installing digiassetsd

cd ~/
git clone https://github.com/DigiByte-Core/digiassetsd
cd digiassetsd
npm install
nano -w config.js


You will want to modify the machineurl, bitcoind-port (14022) blockexplorer and metadataserver URLs



Step 10: Installing pm2

We will use pm2 to look after our node.js services for logging / control etc


cd ~/
npm install -g pm2
cd ~/DigiAssets-Block-Explorer
pm2 start npm --name "explorer" -- start
#you may want to pause here for a week while explorer syncs
cd ~/DigiAssets-Metadata-Server
pm2 start npm --name "metadata" -- start
cd ~/digiassetsd
pm2 start npm --name "digiassetsd" -- start



Step 11: Installing nginx webserver

We will then install nginx and allow it out the firewall


zzzsudo apt install nginx
sudo ufw allow 'Nginx Full'


Then lots of copy / paste for each of the instances, you should find / replace "domainname.tld" with your own domain if you haven't already:


cat <<eof> ~/digiassetsd.nginx
server {
listen 80;
server_name digiassetsd.domainname.tld;
location / {
proxy_pass [1];
proxy_http_version 1.1;
proxy_set_header Upgrade \$http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host \$host;
}
}
EOF


cat <<eof> ~/metadata.nginx
server {
listen 80;
server_name metadata.domainname.tld;
location / {
proxy_pass [2];
proxy_http_version 1.1;
proxy_set_header Upgrade \$http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host \$host;
}
}
EOF


cat <<eof> ~/explorer.nginx
server {
listen 80;
server_name explorer.domainname.tld;
location / {
proxy_pass [3];
proxy_http_version 1.1;
proxy_set_header Upgrade \$http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host \$host;
}
}
EOF


sudo mv digiassetsd.nginx /etc/nginx/sites-available/digiassetsd.domainname.tld
sudo mv metadata.nginx /etc/nginx/sites-available/metadata.domainname.tld
sudo mv explorer.nginx /etc/nginx/sites-available/explorer.domainname.tld
sudo ln -s /etc/nginx/sites-available/* /etc/nginx/sites-enabled/



Step 12: Installing certbot for LetsEncrypt

When prompted, select option 2 to Redirect all requests to HTTPS:


sudo add-apt-repository ppa:certbot/certbot
sudo apt install python-certbot-nginx
sudo certbot --nginx -d digiassets.domainname.tld
sudo certbot --nginx -d explorer.domainname.tld
sudo certbot --nginx -d metadata.domainname.tld
sudo service nginx restart</eof></eof></eof></enter></eof></eof>