Go Elasticsearch Connect Couchbase

This repository contains the Go implementation of the Couchbase Elasticsearch Connector.

Contents

---

• What is Couchbase Elasticsearch Connector?
• Why?
• Features
• Usage
• Configuration
• Examples

What is Couchbase Elasticsearch Connector?

Go Elasticsearch Connect Couchbase is a source connector. So it writes Couchbase mutations to Elasticsearch as documents.

---

Why?

• Build a Couchbase Elasticsearch Connector by using Go to reduce resource usage.
• Suggesting more configurations so users can make changes to less code.
• By presenting the connector as a library, ensuring that users do not clone the code they don't need.

---

Example

package main

import (
	goelasticsearchconnectcouchbase "github.com/Trendyol/go-elasticsearch-connect-couchbase"
	"github.com/Trendyol/go-elasticsearch-connect-couchbase/couchbase"
	"github.com/Trendyol/go-elasticsearch-connect-couchbase/elasticsearch/document"
)

func mapper(event couchbase.Event) []document.ESActionDocument {
	if event.IsMutated {
		e := document.NewIndexAction(event.Key, event.Value, nil)
		return []document.ESActionDocument{e}
	}
	e := document.NewDeleteAction(event.Key, nil)
	return []document.ESActionDocument{e}
}

func main() {
	connector, err := goelasticsearchconnectcouchbase.NewConnectorBuilder("config.yml").
		SetMapper(mapper).
		Build()
	if err != nil {
		return
	}

	defer connector.Close()
	connector.Start()
}

Custom log structures can be used with the connector

package main

import (
	goelasticsearchconnectcouchbase "github.com/Trendyol/go-elasticsearch-connect-couchbase"
	"github.com/Trendyol/go-elasticsearch-connect-couchbase/couchbase"
	"github.com/Trendyol/go-elasticsearch-connect-couchbase/elasticsearch/document"
	"log"
	"os"
)

func mapper(event couchbase.Event) []document.ESActionDocument {
	if event.IsMutated {
		e := document.NewIndexAction(event.Key, event.Value, nil)
		return []document.ESActionDocument{e}
	}
	e := document.NewDeleteAction(event.Key, nil)
	return []document.ESActionDocument{e}
}

func main() {
	logger := log.New(os.Stdout, "cb2elastic: ", log.Ldate|log.Ltime|log.Llongfile)

	connector, err := goelasticsearchconnectcouchbase.NewConnectorBuilder("config.yml").
		SetMapper(mapper).
		SetLogger(logger).
		SetErrorLogger(logger).
		Build()
	if err != nil {
		return
	}

	defer connector.Close()
	connector.Start()
}

---

Usage

$ go get github.com/Trendyol/go-elasticsearch-connect-couchbase

---

Dcp Configuration

Check out on go-dcp-client

Elasticsearch Specific Configuration

Variable	Type	Required	Default
elasticsearch.collectionIndexMapping	map[string]string	yes
elasticsearch.urls	[]string	yes
elasticsearch.bulkSize	int	yes
elasticsearch.bulkTickerDuration	time.Duration	yes

---

Examples

• example