Mentix service #755

Daniel-WWU-IT · 2020-05-18T14:04:27Z

This PR adds Mentix (Mesh Entity Exporter) as a Reva service. This includes:

Support for GOCDB as the data source
HTTP endpoint for getting the data in a standardized format (independent of the used source)
Exporting scrape configs for Prometheus
Example configuration file
Configuration documentation

For logging, I had to extend the rhttp NewService type to pass the global logger, as Mentix is actually a background service and needs logging outside of an HTTP request. I think that there should be services of a more general type so that services like Mentix can be added in a more elegant, streamlined way.

… message log

labkode · 2020-05-19T07:57:47Z

@Daniel-WWU-IT first of all, congrats for this astonishing piece of work, nice job!

For logging, I had to extend the rhttp NewService type to pass the global logger, as Mentix is actually a background service and needs logging outside of an HTTP request. I think that there should be services of a more general type so that services like Mentix can be added in a more elegant, streamlined way.

I couldn't agree more, let make sure is tracked in a Github issue for later.

One question, can you figure out the pat dependency where is used and from which pacakge? go mod why should help you.

I'm happy to merge after you make CI green (some lints are failling).

Daniel-WWU-IT · 2020-05-19T08:51:17Z

@labkode I just updated everything to make lint happy and added some comments as well. The path package was used in network.utils, but I removed that dependency already (though it's a standard package).

Daniel-WWU-IT · 2020-05-19T09:14:02Z

@labkode I created an issue for generic services: #757

labkode · 2020-05-19T12:11:39Z

@Daniel-WWU-IT thanks! I was refering to this package:
https://github.com/cs3org/reva/pull/755/files#diff-f949e2d81c8076ebbf8af38fcbb72c1fR48

Daniel-WWU-IT · 2020-05-19T13:12:44Z

I'm not using that package, neither directly nor indirectly. According to go mod why, it is used by the dataprovider service:

github.com/cs3org/reva/internal/http/services/dataprovider
github.com/tus/tusd/pkg/handler
github.com/bmizerany/pat

This reverts commit 9b9ecf1.

hound · 2020-05-20T08:22:19Z

pkg/mentix/exporters/webapi.go

+	return nil
+}
+
+func (exporter *WebAPIExporter) GetName() string {


exported method WebAPIExporter.GetName should have comment or be unexported

hound · 2020-05-20T08:22:19Z

pkg/mentix/exporters/webapi.go

+	return nil
+}
+
+func (exporter *WebAPIExporter) HandleRequest(resp http.ResponseWriter, req *http.Request) error {


exported method WebAPIExporter.HandleRequest should have comment or be unexported

hound · 2020-05-20T08:22:19Z

pkg/mentix/exporters/webapi.go

+	BaseRequestExporter
+}
+
+func (exporter *WebAPIExporter) Activate(conf *config.Configuration, log *zerolog.Logger) error {


exported method WebAPIExporter.Activate should have comment or be unexported

hound · 2020-05-20T08:22:19Z

pkg/mentix/exporters/webapi.go

+	"github.com/cs3org/reva/pkg/mentix/exporters/webapi"
+)
+
+type WebAPIExporter struct {


exported type WebAPIExporter should have comment or be unexported

hound · 2020-05-20T08:22:19Z

pkg/mentix/exporters/reqexporter.go

+	return strings.TrimSpace(endpoint)
+}
+
+func (exporter *BaseRequestExporter) WantsRequest(r *http.Request) bool {


exported method BaseRequestExporter.WantsRequest should have comment or be unexported

hound · 2020-05-20T08:22:21Z

pkg/mentix/exporters/exporter.go

+	return nil
+}
+
+func (exporter *BaseExporter) Stop() {


exported method BaseExporter.Stop should have comment or be unexported

hound · 2020-05-20T08:22:21Z

pkg/mentix/exporters/exporter.go

+	return nil
+}
+
+func (exporter *BaseExporter) Start() error {


exported method BaseExporter.Start should have comment or be unexported

hound · 2020-05-20T08:22:21Z

pkg/mentix/exporters/exporter.go

+	locker   sync.RWMutex
+}
+
+func (exporter *BaseExporter) Activate(conf *config.Configuration, log *zerolog.Logger) error {


exported method BaseExporter.Activate should have comment or be unexported

hound · 2020-05-20T08:22:21Z

pkg/mentix/connectors/gocdb.go

+	return properties
+}
+
+func (connector *GOCDBConnector) GetName() string {


exported method GOCDBConnector.GetName should have comment or be unexported

hound · 2020-05-20T08:22:22Z

pkg/mentix/connectors/gocdb.go

+	return nil
+}
+
+func (connector *GOCDBConnector) RetrieveMeshData() (*meshdata.MeshData, error) {


exported method GOCDBConnector.RetrieveMeshData should have comment or be unexported

hound · 2020-05-20T08:22:52Z

pkg/mentix/connectors/gocdb.go

+	gocdbAddress string
+}
+
+func (connector *GOCDBConnector) Activate(conf *config.Configuration, log *zerolog.Logger) error {


exported method GOCDBConnector.Activate should have comment or be unexported

hound · 2020-05-20T08:22:52Z

pkg/mentix/connectors/gocdb.go

+	"github.com/cs3org/reva/pkg/mentix/network"
+)
+
+type GOCDBConnector struct {


exported type GOCDBConnector should have comment or be unexported

hound · 2020-05-20T08:22:52Z

pkg/mentix/connectors/connector.go

+	return nil
+}
+
+func FindConnector(connectorID string) (Connector, error) {


exported function FindConnector should have comment or be unexported

hound · 2020-05-20T08:22:53Z

pkg/mentix/connectors/connector.go

+	log  *zerolog.Logger
+}
+
+func (connector *BaseConnector) Activate(conf *config.Configuration, log *zerolog.Logger) error {


exported method BaseConnector.Activate should have comment or be unexported

hound · 2020-05-20T08:22:53Z

pkg/mentix/config/ids.go

+)
+
+const (
+	ExporterIDWebAPI           = "webapi"


exported const ExporterIDWebAPI should have comment (or a comment on this block) or be unexported

hound · 2020-05-20T08:22:53Z

pkg/mentix/config/ids.go

+package config
+
+const (
+	ConnectorIDGOCDB = "gocdb"


exported const ConnectorIDGOCDB should have comment (or a comment on this block) or be unexported

Daniel-WWU-IT added 8 commits May 14, 2020 16:15

Added Mentix (Mesh Entity Exporter) as a Reva service

30038ce

Merged the engine into the main Mentix type

12d60e3

Cleanup

a6f68ba

Configuration names adjusted to Reva naming style

41e7f01

The HTTP service creator function now receives a handle to the global…

2d4a96e

… message log

Added logging

174dbb3

Improved logging slightly

47669d4

Added Mentix configuration to the docs

615cbd7

Daniel-WWU-IT requested a review from labkode as a code owner May 18, 2020 14:04

Daniel-WWU-IT mentioned this pull request May 18, 2020

Add Export script as a Reva background service sciencemesh/sciencemesh#83

Closed

mirekys mentioned this pull request May 18, 2020

Add driver to fetch information from GOCDB REST API for providers sciencemesh/sciencemesh#102

Closed

Daniel-WWU-IT added 3 commits May 19, 2020 10:15

Corrected lint errors and warnings

295d0d4

Added comments to most exported methods, functions and types

4d64f79

Removed dependency on the 'path' package

9b9ecf1

Daniel-WWU-IT added 2 commits May 20, 2020 10:17

Revert "Removed dependency on the 'path' package"

1ab39ef

This reverts commit 9b9ecf1.

The Mentix type wasn't commented

6218560

hound bot reviewed May 20, 2020

View reviewed changes

labkode merged commit 05e7746 into cs3org:master May 20, 2020

Daniel-WWU-IT deleted the mentix branch May 20, 2020 12:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mentix service #755

Mentix service #755

Daniel-WWU-IT commented May 18, 2020

labkode commented May 19, 2020 •

edited

Loading

Daniel-WWU-IT commented May 19, 2020

Daniel-WWU-IT commented May 19, 2020

labkode commented May 19, 2020

Daniel-WWU-IT commented May 19, 2020 •

edited

Loading

hound bot May 20, 2020

hound bot May 20, 2020

hound bot May 20, 2020

hound bot May 20, 2020

hound bot May 20, 2020

hound bot May 20, 2020

hound bot May 20, 2020

hound bot May 20, 2020

hound bot May 20, 2020

hound bot May 20, 2020

hound bot May 20, 2020

hound bot May 20, 2020

hound bot May 20, 2020

hound bot May 20, 2020

hound bot May 20, 2020

hound bot May 20, 2020

Mentix service #755

Mentix service #755

Conversation

Daniel-WWU-IT commented May 18, 2020

labkode commented May 19, 2020 • edited Loading

Daniel-WWU-IT commented May 19, 2020

Daniel-WWU-IT commented May 19, 2020

labkode commented May 19, 2020

Daniel-WWU-IT commented May 19, 2020 • edited Loading

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

hound bot May 20, 2020

Choose a reason for hiding this comment

labkode commented May 19, 2020 •

edited

Loading

Daniel-WWU-IT commented May 19, 2020 •

edited

Loading