Skip to content

Commit

Permalink
Use scylla driver for cassandra
Browse files Browse the repository at this point in the history
  • Loading branch information
@nivekuil
nivekuil committed Jun 27, 2021
1 parent 7ba75e3 commit 0a78a0d
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 1 deletion.
2 changes: 2 additions & 0 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -103,3 +103,5 @@ require (
// replace github.com/chrislusf/raft => /Users/chris/go/src/github.com/chrislusf/raft

replace go.etcd.io/etcd => go.etcd.io/etcd v0.5.0-alpha.5.0.20200425165423-262c93980547

replace github.com/gocql/gocql => github.com/scylladb/gocql v1.4.4-0.20210127085005-3af286340003
2 changes: 2 additions & 0 deletions go.sum
View file
Original file line number Diff line number Diff line change
Expand Up @@ -672,6 +672,8 @@ github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFR
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=
github.com/samuel/go-zookeeper v0.0.0-20190923202752-2cc03de413da/go.mod h1:gi+0XIa01GRL2eRQVjQkKGqKF3SF9vZR/HnPullcV2E=
github.com/scylladb/gocql v1.4.4-0.20210127085005-3af286340003 h1:fGjCtwdJVyLneozxZem+NzbFdPVPenlQaEGVEkWQBhc=
github.com/scylladb/gocql v1.4.4-0.20210127085005-3af286340003/go.mod h1:S154F0u6zQlF3JjuHAidQIExQf9H45yT8z68h0FQYdU=
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=
github.com/seaweedfs/fuse v1.0.8 h1:HBPJTC77OlxwSd2JiTwvLPn8bWTElqQp3xs9vf3C15s=
github.com/seaweedfs/fuse v1.0.8/go.mod h1:W7ubwr1l7KQsMeUpxFFOFOSxUL/ucTRMAlVYs4xdfQ8=
Expand Down
9 changes: 8 additions & 1 deletion weed/filer/cassandra/cassandra_store.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ func (store *CassandraStore) Initialize(configuration util.Configuration, prefix
configuration.GetString(prefix+"username"),
configuration.GetString(prefix+"password"),
configuration.GetStringSlice(prefix+"superLargeDirectories"),
configuration.GetString(prefix+"localDC"),
)
}

Expand All @@ -40,13 +41,19 @@ func (store *CassandraStore) isSuperLargeDirectory(dir string) (dirHash string,
return
}

func (store *CassandraStore) initialize(keyspace string, hosts []string, username string, password string, superLargeDirectories []string) (err error) {
func (store *CassandraStore) initialize(keyspace string, hosts []string, username string, password string, superLargeDirectories []string, localDC string) (err error) {
store.cluster = gocql.NewCluster(hosts...)
if username != "" && password != "" {
store.cluster.Authenticator = gocql.PasswordAuthenticator{Username: username, Password: password}
}
store.cluster.Keyspace = keyspace
fallback := gocql.RoundRobinHostPolicy()
if localDC != "" {
fallback = gocql.DCAwareRoundRobinPolicy(localDC)
}
store.cluster.PoolConfig.HostSelectionPolicy = gocql.TokenAwareHostPolicy(fallback)
store.cluster.Consistency = gocql.LocalQuorum

store.session, err = store.cluster.CreateSession()
if err != nil {
glog.V(0).Infof("Failed to open cassandra store, hosts %v, keyspace %s", hosts, keyspace)
Expand Down

5 comments on commit 0a78a0d

@chrislusf
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to create a scylladb store?

@nivekuil
Copy link
Owner Author

@nivekuil nivekuil commented on 0a78a0d Jun 28, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chrislusf been using this for a while actually, and it works fine. Not sure how Go works, can it be separated out from the current cassandra driver? It uses the same namespaces and is supposedly compatible with Cassandra https://github.com/scylladb/gocql (and faster)

@chrislusf
Copy link

@chrislusf chrislusf commented on 0a78a0d Jun 28, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need to change to "github.com/scylladb/gocql" for scylladb to work?
Does "localDC" also apply to Cassandra, right?

@nivekuil
Copy link
Owner Author

@nivekuil nivekuil commented on 0a78a0d Jun 28, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the intention of scylla's drivers is to be drop-in replacements for cassandra's, but the cassandra driver works with scylla too, it's just higher latency (think it was like 30%?). Not sure how Go's dependency resolution works but it's designed to conflict so I'm not sure both can be used in the same project. I haven't actually tested multi-dc but DCAwareRoundRobinPolicy exists in the main gocql driver too, yes.

It looks like they've been trying to merge the main changes in the fork for a while now: apache/cassandra-gocql-driver#1210

@chrislusf
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. Could you send a PR for the "localDC"? Thanks!

Please sign in to comment.