Design Status node instructions #40
Comments
cc for thumbs and rockets 🤗 @andmironov @oskarth @rachelhamlin @DanBigM |
|
This will be helpful, it is a WIP of my log so far with my node (it is a bit outdated as geth 1.9 came out: |
|
Issue Status: 1. Open 2. Started 3. Submitted 4. Done This issue now has a funding of 500.0 DAI (500.0 USD @ $1.0/DAI) attached to it.
|
|
@hesterbruikman I can't edit this repo - could you please add this to the Bounty Tracker project board? 🙏 (It's for accounting) |
|
@rachelhamlin added 👍 |
|
Hello everyone, I recently started working on this issue, until now I have all the process documented following instructions from this blog post, a bootnode configured on my Linux machine (Ubuntu 19.04) and am planning some research with users. For the research process, I intend to use initial documentation following similar steps as the blog post on the link above, but I have some questions about the process:
I intend to do user testing through the whole creation process, from the raw documentation to the site content with the UI design applied. Thank you. |
|
Hey @luisberns! Let me think of who can help you with this. Potentially @oskarth - do you think you could weigh in on this one? afaik there aren't any improvements planned to this build process. |
|
Thanks Luis! Some initial work on this was done before, don't know where to find this off the top of my head. Best person would probably be @adambabik @jakubgs or @andremedeiros (Igor was doing some work on this). IIRC we had some Dappnode integration/instructions before as well. As for improving this in general, I would love to see it done. Realistically it is up to Core and its priorities, but some things are likely to be low hanging fruit. |
|
Thanks for the replies! I understand that implementing new improvements could be hard due to the priorities. In respect of the documentation for the config files for the Status bootnode, is there any reference in that area? Or the config files are only necessary for the Cheers. |
|
Currently there's several Currently there exist an alternative that doesn't use Docker for
You can use these as a starting point. I should probably add a |
|
The reference for the configuration is the code itself: |
|
Thank you @jakubgs I'll check those links! |
|
I've implemented a way to run a bootnode via systemd in: status-im/status-go#1709 Please check out the |
|
@luisberns I have merged status-im/status-go#1709 so you can base your work off of those changes. The BOOTNODE.md file should be the starting point. |
|
Hello, @jakubgs I spend this weekend working with the systemd installation file that you included in the repository. I found that using the systemd method could make the flow way easier for the user, thank you for that! How I got to run the bootnode in my machineEven with the problems that I'll describe bellow I could get to run the bootnode in my machine, I changed the This would be enough to understand the installation process and proceed with the documentation and site design, but I thought it would be important to share the information about the errors that I encountered. What problems I encounteredI found some problems trying to run the command First, different directory for register the serviceThe systemd Makefile considers this variable to be the directory for storing/registering the service that will be called by the systemctl command When trying to run the service from this directory the service is not recognized by the systemctl. Second, the systemctl uses the
|
|
I'm not sure why you are trying to run that with Also, you said that:
What does that mean? Service isn't running from that directory, the current working directory of the service is defined by the And the STATUS = $(shell systemctl --user is-active $(SERVICE_NAME)) |
|
The reason why was running the process with sudo is, I'll state the steps that I followed to come to the actual moment:
Can you help me to understand what would be the correct way to go? Thank you! |
|
You're right, the But don't use it with sudo. Clean it up with |
|
@jakubgs I was trying to redo the process without sudo but once again I was getting permissions errors so I installed a new Ubuntu 16.04 LTS on a Virtual Machine for doing the process from a clean state. I want to confirm with you some things:
Also, I'm getting an error with the setup, I'll explain my process below. New distro for a bootnode:Install the requirements and clone the repo# Install systemd for a new installation of Ubuntu distro
$ sudo apt-get install systemd
# Install golang-go following this process https://github.com/golang/go/wiki/Ubuntu
$ sudo add-apt-repository ppa:longsleep/golang-backports
$ sudo apt-get update
$ sudo apt-get install golang-go
# Install and setup git for cloning the repository
$ sudo apt-get install git
$ git config --global user.name "username"
$ git config --global user.email "useremail"
# Clone the status-go git repository
$ git clone https://github.com/status-im/status-go $HOME/status-go
$ cd status-go
# Define GOPATH env variable
$ GOPATH=$HOME/status-go
$ echo $GOPATH
// [$HOME]/status-goMake a bootnode using systemd process# Still inside the status-go folder where was cloned
$ make run-bootnode-systemdI'm getting the following output: make[1]: Entering directory '/home/louis/status-go/_assets/systemd/bootnode'
cat: /var/tmp/status-go-boot/nodeaddr: No such file or directory
* Building bootnode binary...
make[2]: Entering directory '/home/louis/status-go'
go build -i -o /home/louis/status-go/build/bin/bootnode -v -tags '' -ldflags ' -X main.buildStamp=2019-12-11.16:17:13 -X github.com/status-im/status-go/params.Version=0.38.0 -X github.com/status-im/status-go/params.GitCommit=58fc7e4 -X github.com/status-im/status-go/vendor/github.com/ethereum/go-ethereum/metrics.EnabledStr=true' ./cmd/bootnode/
cmd/bootnode/main.go:12:2: cannot find package "github.com/ethereum/go-ethereum/crypto" in any of:
/usr/lib/go-1.6/src/github.com/ethereum/go-ethereum/crypto (from $GOROOT)
($GOPATH not set)
cmd/bootnode/main.go:13:2: cannot find package "github.com/ethereum/go-ethereum/log" in any of:
/usr/lib/go-1.6/src/github.com/ethereum/go-ethereum/log (from $GOROOT)
($GOPATH not set)
cmd/bootnode/main.go:14:2: cannot find package "github.com/ethereum/go-ethereum/p2p/discv5" in any of:
/usr/lib/go-1.6/src/github.com/ethereum/go-ethereum/p2p/discv5 (from $GOROOT)
($GOPATH not set)
Makefile:82: recipe for target 'bootnode' failed
make[2]: *** [bootnode] Error 1
make[2]: Leaving directory '/home/louis/status-go'
Makefile:96: recipe for target '/home/louis/status-go/build/bin/bootnode' failed
make[1]: *** [/home/louis/status-go/build/bin/bootnode] Error 2
make[1]: Leaving directory '/home/louis/status-go/_assets/systemd/bootnode'
Makefile:302: recipe for target 'run-bootnode-systemd' failed
make: *** [run-bootnode-systemd] Error 2It seems that the GOPATH is not defined, but I did define it and check before trying to create the bootnode, what I'm doing wrong in this process? Thank you! |
Wrong.
By default the And package sources are expected to be available under Should get you: |
This is just a warning caused by this line in the |
Why not? If it has either Docker or systemd it shouldn't matter. Part of the point of virtual machines is that on the inside they seem like a normal host on normal hardware.
Ones that have either Docker or systemd. |
|
@jakubgs @luisberns thank you both for all the progress on this issue! Per @luisberns suggestion, shall we have a call to go over the first version of documentation? Related development: a survey among core contributors showed that obtaining a static IP address from an ISP is a potential blocker for a number of people to run a node. Therefore, as a starting point, it would be great to complete a manual to I suggest we use the call to go over steps in the manual that are required for different node configurations. |
|
@jakubgs Thank you for your replies and recommendations! When I started working with the bounty I was using this Status blog article as a reference where changing the I repeated the installation process without changing the default I'm still having some errors with the process so I'll describe my process and outputs bellow. As the errors are the same for both versions of Ubuntu I'll be running in the 18.04 version for the log bellow. Run a Status BootnodeInstall required software and clone the git repository# Install required software
$ sudo apt-get install \
golang-go \
systemd \
make \
curl \
git
# Config git and clone the status-go repository
$ git config --global user.name "username"
$ git config --global user.email "useremail"
$ git clone https://github.com/status-im/status-go $HOME/status-go
$ cd status-goRunning a bootnodeSystemd method# Use make to run the setup process
$ make run-bootnode-systemdMy output: make[1]: Entering directory '/home/louis/status-go/_assets/systemd/bootnode'
cat: /var/tmp/status-go-boot/nodeaddr: No such file or directory
* Building bootnode binary...
make[2]: Entering directory '/home/louis/status-go'
go build -i -o /home/louis/status-go/build/bin/bootnode -v -tags '' -ldflags ' -X main.buildStamp=2019-12-15.23:42:40 -X github.com/status-im/status-go/params.Version=0.38.0 -X github.com/status-im/status-go/params.GitCommit=52dd8356 -X github.com/status-im/status-go/vendor/github.com/ethereum/go-ethereum/metrics.EnabledStr=true' ./cmd/bootnode/
cmd/bootnode/main.go:12:2: cannot find package "github.com/ethereum/go-ethereum/crypto" in any of:
/usr/lib/go-1.10/src/github.com/ethereum/go-ethereum/crypto (from $GOROOT)
/home/louis/go/src/github.com/ethereum/go-ethereum/crypto (from $GOPATH)
cmd/bootnode/main.go:13:2: cannot find package "github.com/ethereum/go-ethereum/log" in any of:
/usr/lib/go-1.10/src/github.com/ethereum/go-ethereum/log (from $GOROOT)
/home/louis/go/src/github.com/ethereum/go-ethereum/log (from $GOPATH)
cmd/bootnode/main.go:14:2: cannot find package "github.com/ethereum/go-ethereum/p2p/discv5" in any of:
/usr/lib/go-1.10/src/github.com/ethereum/go-ethereum/p2p/discv5 (from $GOROOT)
/home/louis/go/src/github.com/ethereum/go-ethereum/p2p/discv5 (from $GOPATH)
Makefile:82: recipe for target 'bootnode' failed
make[2]: *** [bootnode] Error 1
make[2]: Leaving directory '/home/louis/status-go'
Makefile:96: recipe for target '/home/louis/status-go/build/bin/bootnode' failed
make[1]: *** [/home/louis/status-go/build/bin/bootnode] Error 2
make[1]: Leaving directory '/home/louis/status-go/_assets/systemd/bootnode'
Makefile:302: recipe for target 'run-bootnode-systemd' failed
make: *** [run-bootnode-systemd] Error 2Docker method# Use make to run the setup process
$ make run-bootnode-dockerMy output: make[1]: Entering directory '/home/louis/status-go/_assets/compose/bootnode'
docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.40/containers/create: dial unix /var/run/docker.sock: connect: permission denied.
See 'docker run --help'.
Makefile:59: recipe for target 'keys/nodekey' failed
make[1]: *** [keys/nodekey] Error 126
make[1]: Leaving directory '/home/louis/status-go/_assets/compose/bootnode'
Makefile:305: recipe for target 'run-bootnode-docker' failed
make: *** [run-bootnode-docker] Error 2 |
|
I think you are using old go version. And your user is not in the |
|
I made a mistake in the process above that the git repository was cloned to the $ go get github.com/status-im/status-go
$ cd $HOME/go/src/github.com/status-im/status-go/
$ make run-bootnode-systemdSuccess, now I got to run the Status Bootnode with systemd! 🎉 I'll update the steps for installation and post here on the issue as well. For the docker installation I create the docker group following this doc and was able to run without a permission issue, but got this output: make[1]: Entering directory '/home/louis/go/src/github.com/status-im/status-go/_assets/compose/bootnode'
Unable to find image 'statusteam/bootnode:latest' locally
latest: Pulling from statusteam/bootnode
8e3ba11ec2a2: Pull complete
bd7c8273c1bf: Pull complete
5b11a42c09d3: Pull complete
Digest: sha256:[key-ommited]
Status: Downloaded newer image for statusteam/bootnode:latest
flag provided but not defined: -genkey
Usage of bootnode:
-addr string
listen address (default ":30301")
-keydata string
hex encoded private key
-n value
These nodes are used to connect to the network if the table is empty and there are no known nodes in the database.
-nodekey string
private key filename
-verbosity int
log verbosity (0-9) (default 3)
-vmodule string
log verbosity pattern
Makefile:59: recipe for target 'keys/nodekey' failed
make[1]: *** [keys/nodekey] Error 2
make[1]: Leaving directory '/home/louis/go/src/github.com/status-im/status-go/_assets/compose/bootnode'
Makefile:305: recipe for target 'run-bootnode-docker' failed
make: *** [run-bootnode-docker] Error 2 |
|
@jakubgs the problem that I comment previously it was indeed the port problem. I stopped the Docker container and restarted the Bootnode service and it was activated properly. |
|
You can setup the service on a different port by setting the make LISTEN_PORT=12345 |
Weird, because I thought the docs made it pretty clear, from
Maybe it should be:
|
|
Updated the wording in status-im/status-go#1755 which also allows for running the |
|
Great work @luisberns ! I went into the notes doc :
3 open points/questions @jakubgs , helpful if you can weigh in here as well:
Hope y'all are having a wonderful Christmas/New Years break! |
Doesn't sound like a good change to me. Status node and a bootnode are different things. Normal users that just want a service to relay their messages don't need a bootnode at all.
There are 2 ways available. Docker container and systemd service:
For example. When I run a History node(Mailserver) under systemd, I can see that the parent process of the node is But when I run the same History node(Mailserver) using Docker, I can see that the parent process is Docker: And in addition to being managed by Docker it is also limited by things like cgroups or namespaces.
If you run $ make run-mailserver-systemd
* Generating 'statusd' config...
Generating config at: /var/tmp/status-go-mail/config.json
* Generating 'statusd' service...
* Enabling 'statusd' service...
systemctl --user enable statusd
Created symlink /home/sochan/.config/systemd/user/default.target.wants/statusd.service → /home/sochan/.config/systemd/user/statusd.service.
* Restarting 'statusd' service...
systemctl --user restart statusd
* Your mailserver is listening on: 93.105.176.182:30304
* Make sure that IP and TCP port are available from the internet!
Your enode address is:
enode://be87d447b217901ab7e4417c92f61c55f4f33cb9089a172088b55741ae024b07aad677466aa69aab19fc54b5bed8bd00ffbc27f3b3e0c7f8fd109562b68a1379@12.34.56.78:30303Although I have thought that we could optionally provide a UTF QR code in the console, but that would require additional tools that most people wouldn't have installed.
The "nanoPC", as the name implies, is a nano... PC. It's a full fledged personal computer but slightly larger than a credit card when viewed from the top. It needs a screen, a keyboard, a mouse if you want GUI, and an ethernet or wifi connection to the internet, like any other PC would. Though it can be set up to just be accessible via SSH. See that it has an HDMI as well as USB and Ethernet: And I don't know about it being "recommended", it should be capable of running statusd just fine with plenty of resources left and it's also cute as a button. |
|
Oh yeah, and happy new year. |
Terminology
@jakubgs This is mostly why I want to remove the reference to bootnode I don't think it makes sense to beginners only wanting to fetch and send messages through their own node and get familiar. In this list of Small, Medium, Large, does either classify as a bootnode or is it a different category altogether?
1. systemd vs docker explanation
Thanks for clarifying! Is it correct to state that you can run 2. Configure node step@luisberns could you add this as a separate step? Something like
3. Recommended device✅ updated the table to include Ethernet outlet and Cable as well as Keyboard, mouse, display, cables ass optional and added a commend that the Nano PC can also be used through SSH. |
I don't think so. The only purpose of a bootnode is letting nodes know about other nodes. It's a digital matchmaker if you will. I'd agree with not posting instructions about running it yet, since it can only confuse people. And the
Probably. Dedicated devices have less resources, and avoiding additional layers like Docker can reduce the overhard for the software running, so in that sense yes. But in general the way you run a process is up to you and your operating system. If you look at systemd adoption you'll see that most major Linux distributions use it as their Init process, so it's the most sensible choice for most users. Using Docker requires installing additional packages, while for most using systemd doesn't. |
Added it to the
Added a note:
@jakubgs Assuming that people order the NanoPC, use it only to run a node, is systemd the recommended option in that case? |
|
fyi: added a reference to https://discuss.status.im/c/messaging-network as a forum to discuss issues during installation. |
|
Happy new year y'all! @hesterbruikman I added the instructions for adding the ENODE address in Status app. I think we could add some screenshots to make easier for people to configure the Bootnode on the app. But the most important thing on this process is the ENODE address, is it possible to collect the ENODE address besides when is logged on the terminal in the installation process? Also, I made a small adjustment on this summary:
|
|
Thanks for these updates @luisberns @jakubgs should the enode address be added to both Profile > Settings > Advanced > Bootnode as well as Profile > Settings > Sync settings > Mailserver I'm assuming that setting it as bootnode only makes sense if you have a static IP address, and else it's best left disabled in bootnode settings, is that right? |
|
Bootnode should go to Bootnodes and Mailserver(History node) should go to Mailservers. But as I said, normal users should never have to run a bootnode. We really should stop talking about that. |
|
fyi @luisberns trial of the documentation is underway while we smooth over the above details: https://discuss.status.im/t/get-your-node-running/1488/3?u=hester |
hesterbruikman
changed the title
|
@luisberns can you please check if there's any additional information in this notes doc that's worthwhile to incorporate or link to? https://notes.status.im/A0VSe1h2SD2ED_95Zy6AEA?view#Installing-Status Other than that I think we're as ready as can be to add the manual to status.im. @jakubgs do you think this could be an update of https://github.com/status-im/status.im/blob/develop/source/technical/run_status_node.md or rather a new file? I believe this was the workflow:
|
|
Yeah, the |
|
@hesterbruikman for our doc, as we also instruct how to add the ENODE addr in the Status app, I think would be interesting to add how to obtain aswell. I updated the content in the notes, can you check? - https://notes.status.im/WJqu-tZoRtqVRO61Ic4NxQ?view#Obtain-your-ENODE-address0 |
Or even simpler: It should show up right after setup too. Though now that I think about it we should include the password in the enode address. Here's the change: status-im/status-go#1811 |
|
@jakubgs I was waiting for the PR to be accepted before updating the docs. I just updated the description of how to generate the ENODE address as well as the QR Code. |
|
Thanks @luisberns ! Walked through the process over the weekend and the option to generate a QR code will make such a difference. Copying manually is a really error sensitive approach:S Cross-posting here what I wrote on our discuss forum. https://discuss.status.im/t/get-your-node-running/1488/19?u=hester I ran into issues at the last step and I think it has to do with setting a password, which is not included in the steps. Does either of you @jakubgs @luisberns have any thoughts on if, when this should happen? |
|
Latest version of the manual posted here for visibility: https://discuss.status.im/t/get-your-node-running/1488/22?u=hester Made a great step forwards in making it more accessible for the average user to set up a node. @luisberns Thanks for your work on this, and your patience! This issue spanned way beyond the Dec/Jan timeframe I had in mind due to shifting priorities. @rachelhamlin can you support in bounty payout? Sorry, I can't tag @StatusSceptre in here :/ |
|
⚡️ A tip worth 500.00000 SAI (500.0 USD @ $1.0/SAI) has been granted to @luisberns for this issue from @StatusSceptre. ⚡️ Nice work @luisberns! To redeem your tip, login to Gitcoin at https://gitcoin.co/explorer and select 'Claim Tip' from dropdown menu in the top right, or check your email for a link to the tip redemption page.
|
|
Issue Status: 1. Open 2. Started 3. Submitted 4. Done This Bounty has been completed. Additional Tips for this Bounty:
|
|
Thank you @hesterbruikman, it was nice doing this bounty and I'm available in case of other opportunities! |
Project
Objectives
Deliverables
UPDATE: It's important that all deliverables are editable.
Scope
Platforms
Linux system
Use cases
* (nano)PC with Ethereum fullnode (can be LES, ULC)Target users
Anyone with a dedicated computer that can run Linux
Background
Status currently hosts nodes that act as Whisper network bootnodes and Mailservers to enable its messaging functionality. These nodes are hosted in US, Asia and Europe and are paid for by Status. This introduces a dependency. There are several reasons why this is undesirable:
It's essential for the sustainability of the Status Network that more people are able to set up a node independently or with support of the community.
For more background info read this thread: https://discuss.status.im/t/increasing-bootstrap-node-diversity-kiss-edition/1138
Note 1: Status nodes are different from Ethereum bootnodes which people might already be running.Note 2: Once set up a node can have different configurations: a) Mailserver mode - around 600 MB per month b) Whisper Network bootnode modeIn addition you can set up an Ethereum full node, this takes significantly more diskspace 160-180GBSetting up an Ethereum full node is outside of the scope of this bounty.Contributor profile
To complete this bounty you ideally have the following skills and equipment:
Skills
Equipment
https://our.status.im/bulletproofing-against-chaos-unicorns-with-status-on-arm/
Collaboration
Online communication with:
The text was updated successfully, but these errors were encountered: