Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

node: refactor package node #21105

Merged
merged 160 commits into from
Aug 3, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
160 commits
Select commit Hold shift + click to select a range
3b2c927
created a new package node_2 where i am experimenting w/ interfaces a…
renaynay Apr 15, 2020
d44ac53
httpHandler -> httpServer
renaynay Apr 20, 2020
ec0c791
saving progress, working on ethstats impl
renaynay Apr 23, 2020
7f36098
committing progress, eth.ethereum + les.lightethereum impl backend
renaynay Apr 23, 2020
bc838a2
not working yet, but committing progress
renaynay Apr 23, 2020
a607196
fixed some registration for services
renaynay Apr 23, 2020
c842bf8
semi-functional impl
renaynay Apr 27, 2020
8dbba60
fixed dev mode start-up, should be working fine now
renaynay Apr 27, 2020
2b67abf
registerrpc and regiterprotocols
renaynay Apr 28, 2020
bf8a05b
fixed RegisterLifecycle to be generic
renaynay Apr 28, 2020
fc7bff8
reverted capitalization of endpoint -- accident
renaynay Apr 28, 2020
ca32adf
removed TODO
renaynay Apr 28, 2020
8290243
remove unnecessary constructor types
renaynay Apr 28, 2020
95bc256
fixes http/ws servers not being able to be started on given host/port
renaynay Apr 29, 2020
0ba43ec
registerHTTP but not w the signature felix suggested
renaynay May 3, 2020
4985505
enable rpc/ws
renaynay May 4, 2020
62ec847
changed RegisterHTTP and cleaned up some code, now gql can start on s…
renaynay May 4, 2020
104ca9c
some clean up
renaynay May 4, 2020
6e53f68
changed console http startup
renaynay May 4, 2020
deb1a8c
allow ws to be enabled on same port as http from console
renaynay May 4, 2020
06a8de8
added todos
renaynay May 7, 2020
4dd45c9
first steps implementing new design
renaynay May 12, 2020
c4f1d63
functional geth
renaynay May 12, 2020
e15007e
registering all apis
renaynay May 12, 2020
3993923
added todo
renaynay May 13, 2020
0a3f9a2
removed todo
renaynay May 13, 2020
2888ca4
Register http (#16)
renaynay May 15, 2020
1f1bc0d
remove unnecessary RegisterHTTP method
renaynay May 19, 2020
ae58c94
added docs, removed err return from register lifecycle
renaynay May 19, 2020
7acc43f
HTTPServer implements Lifecycle and can be started by the lifecycle l…
renaynay May 19, 2020
35460fd
removed unnecessary ListenerAddr
renaynay May 19, 2020
ccadac3
stopServer removes stopped server from list of httpservers on node an…
renaynay May 19, 2020
e3f2bd1
starting to remove use of node.Service throughout codebase
renaynay May 19, 2020
1f5f52e
WIP, trying to remove node.Service from tests
renaynay May 19, 2020
5866e01
fixes node tests (#18)
renaynay May 25, 2020
1badd5d
http server array now a map (#19)
renaynay May 25, 2020
fbc9a64
removed unnecessary change to pass host, port instead of endpoint
renaynay May 25, 2020
60ced1c
added some more tests for http servers
renaynay May 25, 2020
53a2c8f
moved some gql-specific tests to gql package
renaynay May 26, 2020
331d0a6
removed todo
renaynay May 26, 2020
45eecce
whisper tests working
renaynay Jun 3, 2020
a380ff6
create a standalone whisper service for wnode purposes
renaynay Jun 3, 2020
076a9a5
p2p/simulations uses `node.Lifecycle` instead of `node.Service` (#21)
renaynay Jun 4, 2020
3eb4fc8
fixed mailserver tests
renaynay Jun 5, 2020
6e41e41
les tests passing
renaynay Jun 5, 2020
2df58f7
changes after rebasing on master, likely bc of serverpool pr
renaynay Jun 5, 2020
fd6623e
fixed p2p testing package
renaynay Jun 5, 2020
ebde331
removing unnecessary methods and fixing todos
renaynay Jun 8, 2020
06bf009
removing more unnecessary methods
renaynay Jun 8, 2020
4ce4c40
instead of passing both les and eth explicitly, fetch backend from node
renaynay Jun 8, 2020
cc3daf6
typo
renaynay Jun 8, 2020
3f132b5
fetch backend from service context rather than node since node isnt s…
renaynay Jun 8, 2020
306f390
fixing some broken tests
renaynay Jun 8, 2020
fddc6bd
linted
renaynay Jun 8, 2020
6c91bb4
not passing backends to startnode
renaynay Jun 9, 2020
5d67845
fixing docs
renaynay Jun 9, 2020
b8c5dc4
fixed dao_test
renaynay Jun 16, 2020
885a7d4
lint
renaynay Jun 16, 2020
abbe6f8
fixed current tests, but need to add more
renaynay Jun 16, 2020
ae92dee
new test to check for proper creation of http servers on node
renaynay Jun 16, 2020
f2fcf69
added more test coverage, removed some useless code
renaynay Jun 17, 2020
204514c
minor changes, removing spaces, cleaninng up
renaynay Jun 17, 2020
41a25c3
some changes to graphql and other misc
renaynay Jun 18, 2020
666db04
linted
renaynay Jun 19, 2020
828646c
no need to pass gen state to makefullnode
renaynay Jun 19, 2020
3443202
missing register apis and protocols
renaynay Jun 22, 2020
9ed169e
register http before starting
renaynay Jun 22, 2020
fb8e5b0
added some documentation
renaynay Jun 22, 2020
e897df9
minor fixes
renaynay Jun 22, 2020
cfae6ed
Changes requested from PR review (#23)
renaynay Jun 29, 2020
d49a5be
attempt at fixing http test (#24)
renaynay Jun 30, 2020
52896ae
fixed broken http test, waiting whether it passes on all builds
renaynay Jun 30, 2020
dce4780
use int32 instead of bool when checking whether to handle requests (#25)
renaynay Jul 2, 2020
0e7b866
make sure to register all APIs when registering services apis on node
renaynay Jul 3, 2020
a5204a4
store int32
renaynay Jul 6, 2020
71d9590
remove P2PServer method from les, unnecessary
renaynay Jul 6, 2020
7661eb8
refactor http servers + graphql (#26)
renaynay Jul 7, 2020
d96b280
removed some TODOs
renaynay Jul 7, 2020
ff002b1
init leth handler
renaynay Jul 9, 2020
bafc12a
register protocols and apis of les on eth backend
renaynay Jul 9, 2020
975061c
node: make Node.Attach work before Node.Start
fjl Jul 10, 2020
8a568a4
all: excise SetContractBackend
fjl Jul 10, 2020
e930d38
revert blackbox test for ethclient
renaynay Jul 10, 2020
54b8389
les: fix setOracle for chains without checkpoint
fjl Jul 10, 2020
212bb78
les --> lifecycle decoupled from ethbackend (#29)
renaynay Jul 10, 2020
1844d22
remove Fatalf from node, should panic instead
renaynay Jul 10, 2020
a24c7b8
eth: remove AddLesServer
fjl Jul 10, 2020
aaee36b
cmd/utils: remove duplicate protocol/API registration
fjl Jul 10, 2020
ebc7061
remove newgqlhandler from rpcstack
renaynay Jul 10, 2020
5d2e755
node: stop lifecycles in reverse order
fjl Jul 10, 2020
1b50b55
node: simplify RegisterLifecycle
fjl Jul 10, 2020
489bb05
donnt check ls
renaynay Jul 10, 2020
b2931b6
Explicit type check for backend (#27)
renaynay Jul 10, 2020
e7602e3
go mod tidy
renaynay Jul 10, 2020
9ee9bbe
linted
renaynay Jul 13, 2020
b26ea83
changes requested by gary
renaynay Jul 15, 2020
dd06152
node: unexport CreateHTTPServer
fjl Jul 13, 2020
57ffd7c
graphql: don't check for HTTP server creation
fjl Jul 13, 2020
c8b01c4
node: unexport ExistingHTTPServer
fjl Jul 13, 2020
783ad61
node: add new lifecycle docs
fjl Jul 21, 2020
af82314
node: fix comment on Start
fjl Jul 21, 2020
33ffe5b
node: move datadir cleanup from Stop to Close
fjl Jul 21, 2020
767e3b1
node: simplify stop channel
fjl Jul 21, 2020
ce38ec6
node: remove Stop
fjl Jul 21, 2020
cd46fce
all: Node.Stop -> Node.Close
fjl Jul 21, 2020
d2c2474
mobile: add TODO for restart
fjl Jul 21, 2020
4fdd72b
node: add more runstate checks
fjl Jul 21, 2020
068fe78
node: ensure Register* can only be used in initializing state
fjl Jul 21, 2020
81ecb44
removed unnecessary use of fmt.Sprintf
renaynay Jul 21, 2020
0ea3bb4
revert accidental q
renaynay Jul 21, 2020
f3f1178
fixed ethstats backend interface
renaynay Jul 21, 2020
3c0fb99
node: whitespace changes
fjl Jul 21, 2020
7801fad
p2p/simulations/adapters: use Node.Attach where possible
fjl Jul 21, 2020
d4e2173
node: avoid setting server to nil
fjl Jul 21, 2020
dab27f6
node: use simple lock
fjl Jul 21, 2020
b8d7587
node: more cosmetic fixes
fjl Jul 21, 2020
b6c60cf
node: track open databases
fjl Jul 21, 2020
4ae4172
core/rawdb: make Freezer.Close idempotent
fjl Jul 21, 2020
8400a5d
node: avoid auto-closing DB if service closes it first
fjl Jul 21, 2020
fa88f30
p2p/simulations/adapters: delete snapshotService
fjl Jul 21, 2020
4da9ef7
p2p: delete Server.Running
fjl Jul 21, 2020
e1f4668
cmd/geth, cmd/utils: gofmt -w -s
fjl Jul 22, 2020
a30d401
p2p/simulations/examples: remove unused APIs callback
fjl Jul 22, 2020
5a3cf7a
p2p/testing: delete unused package
fjl Jul 22, 2020
e6cd0e3
node: unexport RegisterHTTPServer
fjl Jul 22, 2020
aba0667
node: clean up instrumented service tests
fjl Jul 22, 2020
a23b650
node: add test for database auto-closing
fjl Jul 22, 2020
0a5312a
node: WIP TestNodeOpenDatabaseFromLifecycle
fjl Jul 22, 2020
39dbf65
node: move Start up
fjl Jul 22, 2020
6a16cbe
node: simplify Start/Stop concurrency handling
fjl Jul 22, 2020
cc9fddd
node: unexport built-in API handler objects
fjl Jul 22, 2020
390e8a3
node: move RegisterApisFromWhitelist
fjl Jul 22, 2020
908268d
miner: fix test scripts
fjl Jul 22, 2020
891b286
miner: simplify test scripts
fjl Jul 22, 2020
c70d5e6
miner: simplify test scripts even more
fjl Jul 22, 2020
45f162c
node: remove atomic operations on n.state
fjl Jul 23, 2020
4a6faff
node: WIP rewrite HTTP server setup
fjl Jul 24, 2020
a1a4454
descriptive logging
renaynay Jul 27, 2020
2d81929
removed ws log
renaynay Jul 27, 2020
c985bd4
node: add test for StartRPC API
fjl Jul 27, 2020
f2dbf7e
node: fix test
fjl Jul 27, 2020
47315c0
node: delete redundant websocket enabling test
fjl Jul 27, 2020
9a1131c
node: fix all the HTTP server API tests
fjl Jul 27, 2020
6d544c6
node: improve comments
fjl Jul 27, 2020
d7c1258
node: fix starting RPC after it failed to listen
fjl Jul 27, 2020
b8b1046
changed node tests
renaynay Jul 27, 2020
8580962
rpcstack tests
renaynay Jul 27, 2020
68cc90c
linted and new tests
renaynay Jul 28, 2020
917135f
reshuffling some funcs in rpcstack
renaynay Jul 28, 2020
3664dd5
node: move disableWS down as well
fjl Aug 3, 2020
de11bb1
node: improve comments
fjl Aug 3, 2020
98cf053
go.mod: revert dependency updates
fjl Aug 3, 2020
b265e2c
eth: fix compile issue in backend.go
fjl Aug 3, 2020
1c639bd
node: fix IPC API registration
fjl Aug 3, 2020
e11bc43
graphql: fix test
fjl Aug 3, 2020
f1b1b33
node: remove blank line
fjl Aug 3, 2020
3001cf5
node: sort+dedup logged HTTP handlers
fjl Aug 3, 2020
4d42e9f
node: fix lint issue
fjl Aug 3, 2020
2b5bd5e
node: rename RegisterPath -> RegisterHandler
fjl Aug 3, 2020
74e2aa9
node: explain RegisterHandler 'name' parameter in docs
fjl Aug 3, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fixes node tests (#18)
  • Loading branch information
renaynay authored and fjl committed Aug 3, 2020
commit 5866e01c2b7cfcca4cc53405e07a1a1497029b4c
11 changes: 0 additions & 11 deletions node/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,17 +39,6 @@ func convertFileLockError(err error) error {
return err
}

// DuplicateServiceError is returned during Node startup if a registered service
// constructor returns a service of the same type that was already started.
type DuplicateServiceError struct {
Kind reflect.Type
}

// Error generates a textual representation of the duplicate service error.
func (e *DuplicateServiceError) Error() string {
return fmt.Sprintf("duplicate service: %v", e.Kind)
}

// StopError is returned if a Node fails to stop either any of its registered
// services or itself.
type StopError struct {
Expand Down
69 changes: 36 additions & 33 deletions node/node.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ type Node struct {

ServiceContext *ServiceContext

lifecycles []Lifecycle // All registered backends, services, and auxiliary services that have a lifecycle
lifecycles map[reflect.Type]Lifecycle // All registered backends, services, and auxiliary services that have a lifecycle

rpcAPIs []rpc.API // List of APIs currently provided by the node
inprocHandler *rpc.Server // In-process RPC request handler to process the API requests
Expand Down Expand Up @@ -107,8 +107,10 @@ func New(conf *Config) (*Node, error) {
accman: am,
ephemeralKeystore: ephemeralKeystore,
config: conf,
lifecycles: make(map[reflect.Type]Lifecycle),
ServiceContext: &ServiceContext{
Config: *conf,
Lifecycles: make(map[reflect.Type]Lifecycle),
},
httpServers: make([]*HTTPServer, 0),
ipc: &HTTPServer{
Expand Down Expand Up @@ -199,12 +201,12 @@ func (n *Node) Close() error {

// RegisterLifecycle registers the given Lifecycle on the node
func (n *Node) RegisterLifecycle(lifecycle Lifecycle) {
for _, existing := range n.lifecycles {
if existing == lifecycle {
Fatalf("Lifecycle cannot be registered more than once", lifecycle)
}
kind := reflect.TypeOf(lifecycle)
if _, exists := n.lifecycles[kind]; exists {
Fatalf("Lifecycle cannot be registered more than once", kind)
}
n.lifecycles = append(n.lifecycles, lifecycle)

n.lifecycles[kind] = lifecycle
}

// RegisterProtocols adds backend's protocols to the node's p2p server
Expand Down Expand Up @@ -265,7 +267,7 @@ func (n *Node) CreateHTTPServer(h *HTTPServer, exposeAll bool) error {

// running returns true if the node's p2p server is already running
func (n *Node) running() bool {
return n.server.Listening()
return n.server.Running()
}

// Start creates a live P2P node and starts running it.
Expand All @@ -289,9 +291,8 @@ func (n *Node) Start() error {

// TODO running p2p server needs to somehow be added to the backend

// Start the configured RPC interfaces
if err := n.startRPC(); err != nil {
n.stopLifecycles(n.lifecycles)
// Configure the RPC interfaces
if err := n.configureRPC(); err != nil {
n.server.Stop()
return err
}
Expand All @@ -301,8 +302,11 @@ func (n *Node) Start() error {
for _, lifecycle := range n.lifecycles {
if err := lifecycle.Start(); err != nil {
n.stopLifecycles(started)
n.server.Stop()
return err
}
started = append(started, lifecycle)
n.ServiceContext.Lifecycles[reflect.TypeOf(lifecycle)] = lifecycle
}

// Finish initializing the service context
Expand Down Expand Up @@ -478,15 +482,7 @@ func (n *Node) stopServer(server *HTTPServer) {

// removeLifecycle removes a stopped Lifecycle from the running node's Lifecycles
func (n *Node) removeLifecycle(lifecycle Lifecycle) {
remainingLifecycles := make([]Lifecycle, len(n.lifecycles)-1)
index := 0
for _, remaining := range n.lifecycles {
if remaining != lifecycle {
remainingLifecycles[index] = remaining
index ++
}
}
n.lifecycles = remainingLifecycles
delete(n.lifecycles, reflect.TypeOf(lifecycle))
}

// Stop terminates a running node along with all it's services. In the node was
Expand All @@ -496,7 +492,7 @@ func (n *Node) Stop() error {
defer n.lock.Unlock()

// Short circuit if the node's not running
if n.server == nil {
if n.server == nil || !n.running() {
return ErrNodeStopped
}

Expand All @@ -506,10 +502,11 @@ func (n *Node) Stop() error {
failure := &StopError{
Services: make(map[reflect.Type]error),
}
for _, lifecycle := range n.lifecycles {
for kind, lifecycle := range n.lifecycles {
if err := lifecycle.Stop(); err != nil {
failure.Services[reflect.TypeOf(lifecycle)] = err
}
delete(n.lifecycles, kind)
}
n.server.Stop()
n.server = nil
Expand Down Expand Up @@ -554,18 +551,6 @@ func (n *Node) Wait() {
<-stop
}

// Restart terminates a running node and boots up a new one in its place. If the
// node isn't running, an error is returned.
func (n *Node) Restart() error {
if err := n.Stop(); err != nil {
return err
}
if err := n.Start(); err != nil {
return err
}
return nil
}

// Attach creates an RPC client attached to an in-process API handler.
func (n *Node) Attach() (*rpc.Client, error) {
n.lock.RLock()
Expand Down Expand Up @@ -695,6 +680,24 @@ func (n *Node) ResolvePath(x string) string {
return n.config.ResolvePath(x)
}

// Lifecycle retrieves a currently running Lifecycle registered of a specific type.
func (n *Node) Lifecycle(lifecycle interface{}) error {
n.lock.RLock()
defer n.lock.RUnlock()

// Short circuit if the node's not running
if !n.running() {
return ErrNodeStopped
}
// Otherwise try to find the service to return
element := reflect.ValueOf(lifecycle).Elem()
if running, ok := n.lifecycles[element.Type()]; ok {
element.Set(reflect.ValueOf(running))
return nil
}
return ErrServiceUnknown
}

// apis returns the collection of RPC descriptors this node offers.
func (n *Node) apis() []rpc.API {
return []rpc.API{
Expand Down
7 changes: 1 addition & 6 deletions node/node_example_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ type SampleLifecycle struct{}
func (s *SampleLifecycle) Start() error { fmt.Println("Service starting..."); return nil }
func (s *SampleLifecycle) Stop() error { fmt.Println("Service stopping..."); return nil }

func ExampleService() {
func ExampleLifecycle() {
// Create a network node to run protocols with the default values.
stack, err := node.New(&node.Config{})
if err != nil {
Expand All @@ -50,15 +50,10 @@ func ExampleService() {
if err := stack.Start(); err != nil {
log.Fatalf("Failed to start the protocol stack: %v", err)
}
if err := stack.Restart(); err != nil {
log.Fatalf("Failed to restart the protocol stack: %v", err)
}
if err := stack.Stop(); err != nil {
log.Fatalf("Failed to stop the protocol stack: %v", err)
}
// Output:
// Service starting...
// Service stopping...
// Service starting...
// Service stopping...
}
Loading