IoT Cloud

Privacy friendly framework for IoT Cloud.

Before Running

• Install Mosquitto broker sudo apt install mosquitto
• and make it listen on specified port e.g.: mosquitto -p 1883
• install docker-compose

Running using Docker

• from root directory run: CURRENT_UID=$(id -u):$(id -g) docker-compose up
  • NOTE: CURRENT_UID=$(id -u):$(id -g) is necessary because test container needs to run under current user to write reports
• to clean up _pycache_ and .pytest_cache directories created by docker use following commands:
  • sudo find . -path '*/__pycache__*' ! -path "./venv*" -delete
  • sudo find . -path '*/.pytest_cache*' ! -path "./venv*" -delete
  • NOTE: run commands first without -delete flag to test, to make sure you don't damage your system

Running using Flask

• export path to flask application factory method - export FLASK_APP="app.app_setup:create_app('development')"
• run from ./IoT-Cloud/app directory with command flask run

Running tests

• When running tests make sure you set environment variable TESTING_ENV to host_testing(defaults to testing), so the application uses config.env variables needed for running tests on docker host. If not set, tests will run as if they were inside docker container ( = with different URLs).

Testing communication between broker and app

• install sudo apt install mosquitto-clients
• publish to topics to which application is subscribed to by using mosquitto_pub -h <host> -p <port> -t <topic_name> -m "message"

Database

It is recommended to use dockerized Postgres database, which is included here (in ./postgres). You can bring it by running docker-compose up in that directory. It listens on port 5430 instead of 5432 to make sure it does not interfere with any host services, therefore you need to make sure that URLs in config.env correct this port.

NOTE: there is also another Postgres database inside main applications docker-compose, however that is used only for running tests.

Accessing database

• you can access database using pgadmin4
• login to psql using sudo -u postgres psql
• find postgresql.conf and pg_hba.conf using show config_file; and show hba_file; respectively
• in postgresql.conf set listen address to listen_addresses = '*' to allow remote access
• in pg_hba.conf add after localhost line: host    all             all             0.0.0.0/0            md5
• restart postgres service - sudo service postgresql stop and sudo service postgresql start
• steps based on This blog

Modifying database inside container

• docker exec -it $(docker-compose ps -q db ) psql -Upostgres -c '\z' to show all tables in the database. In another terminal, talk to the container's Postgres

• docker exec -it $(docker-compose ps -q db ) psql -Upostgres -c 'create table user()' to write queries directly

• docker exec -it $(docker-compose ps -q db ) pg_dump -Upostgres > backup.sql to dump (backup) data to host

• docker exec -i $(docker-compose ps -q db ) psql -Upostgres < backup.sql to restore backed up data

• based on StackOverflow answer

• Preferably access using pgadmin4 should be used with username postgres and exposed port from docker-compose.override (by default: 5431)

TLS

It's necessary to provide certificates to use application. When using Mosquitto, please use steps at Mosquitto website

• Files created in previous steps should be placed in certs folder both for Mosquitto and application, replacing *.dummy files
• Application currently does not require client certificates, to change that, you need to set require_certificate true in mosquitto.conf and provide client certfile and keyfile to client.tls_set in create_app.py through CLIENT_CERTFILE_PATH and CLIENT_KEYFILE_PATH config attributes
• in production SSL_INSECURE attribute in config should be set to False, so when generating certificates, make sure that broker name (hostname) matches name on certificate

Setting up HTTPS

• You need to provide certificate and key for Nginx server to be accessible - this should be done by replacing dummy files in ./webserver/ssl folder
• To generate self-signed certificate:
  • change directory to ./webserver/ssl
  • run sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./private/webserver.key -out ./certs/webserver.crt

Running Node-RED

• To start Node-RED with prepared settings.js file go to ./node-red directory and run in using:
  • ./node_modules/node-red/bin/node-red-pi -s ./settings.js if you have Node-RED installed in node_modules
  • node-red -s ./settings.js if installed globally

Troubleshooting

• If you have issues running tests inside container (there are some tests that need to be excluded when running inside container), try docker container prune and docker-compose up again
• If testing DB inside docker container is not being created by create_db.sh, you need to first remove persistent volume (docker volume rm iotcloud_data) and prune containers (docker container prune)
• if you encounter this error message: libpbc.so.1: cannot open shared object file: No such file or directory, make sure you run ldconfig after installing pbc, if that doesn't help:
  • check whether path to pbc is in LD_LIBRARY_PATH (echo $LD_LIBRARY_PATH )
  • if not, then run sudo find / -name libpbc.so
  • add path outputted by previous command to LD_LIBRARY_PATH - e.g. LD_LIBRARY_PATH=/usr/local/lib and export it
  • if that solves the issue, add LD_LIBRARY_PATH to ~/.bashrc and source it
• if you encounter error message when installing Charm (running ./configure.sh) stating that you don't have python3-dev or python3-config:
  • check whether you have any other version installed e.g python3.6-config, if yes, replace occurrence(s) of python3-config in ./configure.sh with the one you have installed and run it again