Skip to content

Commit

Permalink
build: merge pr #442 from tarumes/main
Browse files Browse the repository at this point in the history
feat(sqlite): add basic sqlite support
  • Loading branch information
lukewhrit authored Aug 31, 2024
2 parents 44b5068 + 47856f9 commit 81a849e
Show file tree
Hide file tree
Showing 6 changed files with 180 additions and 27 deletions.
45 changes: 28 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Pastebins are a type of online content storage service where users can store pla
- [x] Modern, JavaScript-free user interface
- [x] Syntax highlighting for all the most popular languages and Raw text mode
- [ ] Password-protected encrypted pastes
- [ ] SQLite Support
- [x] SQLite Support
- [ ] Paste collections
- [ ] Reader view mode (Markdown is formatted and word wrapping is enabled)
- [ ] QR Codes
Expand All @@ -42,6 +42,7 @@ Pastebins are a type of online content storage service where users can store pla
- [Using Docker](#using-docker)
- [Manually](#manually)
- [Environment Variables](#environment-variables)
- [Database Connection URI](#database-connection-uri)
- [Usage](#usage)
- [On the Web](#on-the-web)
- [CLI](#cli)
Expand All @@ -64,7 +65,8 @@ $ sudo docker run -d -p 80:9000 spacebinorg/spirit

#### Manually

> [!IMPORTANT] > **Requires: [Git](https://git-scm.com/downloads), [Go 1.22.4](https://go.dev/doc/install), [GNU Makefile](https://www.gnu.org/software/make/#download), and a [PostgreSQL](https://www.postgresql.org/download/) [server](https://m.do.co/c/beaf675c3e00).**
> [!IMPORTANT]
> **Requires: [Git](https://git-scm.com/downloads), [Go 1.22.4](https://go.dev/doc/install), [GNU Makefile](https://www.gnu.org/software/make/#download), and a SQLite database or [PostgreSQL](https://www.postgresql.org/download/) [server](https://m.do.co/c/beaf675c3e00).**
```sh
# Clone the Github repository
Expand All @@ -75,37 +77,46 @@ $ cd spacebin
$ make spirit

# Start Spacebin
$ SPIRIT_CONNECTION_URI="<your PostgreSQL instance URI>" ./bin/spirit
$ SPIRIT_CONNECTION_URI="sqlite://database.sqlite" ./bin/spirit # SQLite
$ SPIRIT_CONNECTION_URI="postgres://<your PostgreSQL instance URI>" ./bin/spirit # PostgreSQL

# Success! Spacebin is now available at port 9000 on your machine.
```

#### Environment Variables

| Variable Name | Type | Default | Description |
| ----------------------- | --------------------- | ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `SPIRIT_HOST` | String | `0.0.0.0` | Host address to listen on |
| `SPIRIT_PORT` | Int | `9000` | HTTP port to listen on |
| `SPIRIT_RATELIMITER` | String | `200x5` | Requests allowed per second before the user is ratelimited |
| `SPIRIT_CONNECTION_URI` | String | **Required** | [PostgreSQL Database URI String](https://stackoverflow.com/questions/3582552/what-is-the-format-for-the-postgresql-connection-string-url#20722229) |
| `SPIRIT_HEADLESS` | Bool | `False` | Enables/disables the web interface |
| `SPIRIT_ANALYTICS` | String | `""` | `<script>` tag for analytics (leave blank to disable) |
| `SPIRIT_ID_LENGTH` | Int | `8` | Length for document IDs |
| `SPIRIT_ID_TYPE` | `"key"` or `"phrase"` | `key` | Format of IDs: `key` is a random string of letters and [`phrase` is a combination of words](https://github.com/lukewhrit/phrase) |
| `SPIRIT_MAX_SIZE` | Int | `400000` | Max allowed size of a document in bytes |
| `SPIRIT_EXPIRATION_AGE` | Int64 | `720` | Amount of time to expire documents after |
| `SPIRIT_DOCUMENTS` | []String | `[]` | List of any custom documents to serve |
| Variable Name | Type | Default | Description |
| ----------------------- | --------------------- | ------------ | -------------------------------------------------------------------------------------------------------------------------------- |
| `SPIRIT_HOST` | String | `0.0.0.0` | Host address to listen on |
| `SPIRIT_PORT` | Int | `9000` | HTTP port to listen on |
| `SPIRIT_RATELIMITER` | String | `200x5` | Requests allowed per second before the user is ratelimited |
| `SPIRIT_CONNECTION_URI` | String | **Required** | Database connection URI |
| `SPIRIT_HEADLESS` | Bool | `False` | Enables/disables the web interface |
| `SPIRIT_ANALYTICS` | String | `""` | `<script>` tag for analytics (leave blank to disable) |
| `SPIRIT_ID_LENGTH` | Int | `8` | Length for document IDs |
| `SPIRIT_ID_TYPE` | `"key"` or `"phrase"` | `key` | Format of IDs: `key` is a random string of letters and [`phrase` is a combination of words](https://github.com/lukewhrit/phrase) |
| `SPIRIT_MAX_SIZE` | Int | `400000` | Max allowed size of a document in bytes |
| `SPIRIT_EXPIRATION_AGE` | Int64 | `720` | Amount of time to expire documents after |
| `SPIRIT_DOCUMENTS` | []String | `[]` | List of any custom documents to serve |

> [!WARNING]
> Environment variables for Spacebin are prefixed with `SPIRIT_`. They will be updated to `SPACEBIN_` in the next major version.
##### Database Connection URI

Spacebin supports two database formats: **SQLite** and **Postgres**

- For SQLite, use either the scheme `file://` or `sqlite://` and a file name.
- Example: `file://database.db`
- For PostgreSQL, use [the standard PostgreSQL URI format](https://stackoverflow.com/questions/3582552/what-is-the-format-for-the-postgresql-connection-string-url#20722229).

### Usage

#### On the Web

To use Spacebin on the web, our team provides a web app. You can access the web app at **[spaceb.in](https://spaceb.in)**. You must use `https://spaceb.in/api` to access the API routes.

A version of spacebin that is built directly from the `develop` branch is also available at \*\*[staging.spaceb.in](https://staging.spaceb.in)
A version of spacebin that is built directly from the `develop` branch is also available at [staging.spaceb.in](https://staging.spaceb.in).

#### CLI

Expand Down
33 changes: 28 additions & 5 deletions cmd/spacebin/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import (
"errors"
"fmt"
"net/http"
"net/url"
"os"
"os/signal"
"syscall"
Expand All @@ -47,21 +48,43 @@ func init() {
}

func main() {
pg, err := database.NewPostgres()
var db database.Database

// Parse the connection URI
uri, err := url.Parse(config.Config.ConnectionURI)
if err != nil {
log.Fatal().
Err(err).
Msg("Could not connect to database")
Msg("Not a valid Connection URI")
}

if err := pg.Migrate(context.Background()); err != nil {
// Connect either to SQLite or PostgreSQL
switch uri.Scheme {
case "file", "sqlite":
sq, err := database.NewSQLite(uri.Host)
if err != nil {
log.Fatal().
Err(err).
Msg("Could not connect to database")
}
db = sq
case "postgresql":
pg, err := database.NewPostgres(uri.String())
if err != nil {
log.Fatal().
Err(err).
Msg("Could not connect to database")
}
db = pg
}

if err := db.Migrate(context.Background()); err != nil {
log.Fatal().
Err(err).
Msg("Failed migrations; Could not create DOCUMENTS tables.")
}

m := server.NewServer(&config.Config, pg)
m := server.NewServer(&config.Config, db)

m.MountMiddleware()
m.RegisterHeaders()
Expand Down Expand Up @@ -107,7 +130,7 @@ func main() {
}

// Database
err := pg.Close()
err := db.Close()

if err != nil {
log.Fatal().
Expand Down
16 changes: 14 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,23 @@ require (
github.com/lukewhrit/phrase v1.0.0
github.com/rs/zerolog v1.33.0
github.com/stretchr/testify v1.9.0
golang.org/x/exp v0.0.0-20230807204917-050eac23e9de
golang.org/x/exp v0.0.0-20240823005443-9b4947da3948
)

require (
github.com/dlclark/regexp2 v1.11.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
golang.org/x/net v0.28.0 // indirect
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect
modernc.org/libc v1.55.3 // indirect
modernc.org/mathutil v1.6.0 // indirect
modernc.org/memory v1.8.0 // indirect
modernc.org/strutil v1.2.0 // indirect
modernc.org/token v1.1.0 // indirect
)

require (
Expand All @@ -27,9 +38,10 @@ require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/kr/pretty v0.3.1 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
golang.org/x/sys v0.23.0 // indirect
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
modernc.org/sqlite v1.32.0
)
29 changes: 29 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/go-chi/chi/v5 v5.1.0 h1:acVI1TYaD+hhedDJ3r54HyA6sExp3HfXq7QWEEY/xMw=
github.com/go-chi/chi/v5 v5.1.0/go.mod h1:DslCQbL2OYiznFReuXYUmQ2hGd1aDpCnlMNITLSKoi8=
github.com/go-chi/cors v1.2.1 h1:xEC8UT3Rlp2QuWNEr4Fs/c2EAGVKBwy/1vHx3bppil4=
Expand All @@ -27,6 +29,10 @@ github.com/go-chi/httprate v0.12.1/go.mod h1:TUepLXaz/pCjmCtf/obgOQJ2Sz6rC8fSf5c
github.com/go-ozzo/ozzo-validation/v4 v4.3.0 h1:byhDUpfEwjsVQb1vBunvIjh2BHQ9ead57VkAEY4V+Es=
github.com/go-ozzo/ozzo-validation/v4 v4.3.0/go.mod h1:2NKgrcHl3z6cJs+3Oo940FPRiTzuqKbvfrL2RxCj6Ew=
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
Expand All @@ -45,10 +51,16 @@ github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovk
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.19 h1:JITubQf0MOLdlGRuRq+jtsDlekdYPia9ZFsB8h/APPA=
github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/rs/xid v1.5.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
Expand All @@ -60,6 +72,9 @@ github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsT
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
golang.org/x/exp v0.0.0-20230807204917-050eac23e9de h1:l5Za6utMv/HsBWWqzt4S8X17j+kt1uVETUX5UFhn2rE=
golang.org/x/exp v0.0.0-20230807204917-050eac23e9de/go.mod h1:FXUEEKJgO7OQYeo8N01OfiKP8RXMtf6e8aTskBGqWdc=
golang.org/x/exp v0.0.0-20231108232855-2478ac86f678/go.mod h1:zk2irFbV9DP96SEBUUAy67IdHUaZuSnrz1n472HUCLE=
golang.org/x/exp v0.0.0-20240823005443-9b4947da3948 h1:kx6Ds3MlpiUHKj7syVnbp57++8WpuKPcR5yjLBjvLEA=
golang.org/x/exp v0.0.0-20240823005443-9b4947da3948/go.mod h1:akd2r19cwCdwSwWeIdzYQGa/EZZyqcOdwWiwj5L5eKQ=
golang.org/x/net v0.28.0 h1:a9JDOJc5GMUJ0+UDqmLT86WiEy7iWyIhz8gz8E4e5hE=
golang.org/x/net v0.28.0/go.mod h1:yqtgsTWOOnlGLG9GFRrK3++bGOUEkNBoHZc8MEDWPNg=
golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
Expand All @@ -76,3 +91,17 @@ gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EV
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 h1:5D53IMaUuA5InSeMu9eJtlQXS2NxAhyWQvkKEgXZhHI=
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6/go.mod h1:Qz0X07sNOR1jWYCrJMEnbW/X55x206Q7Vt4mz6/wHp4=
modernc.org/libc v1.55.3 h1:AzcW1mhlPNrRtjS5sS+eW2ISCgSOLLNyFzRh/V3Qj/U=
modernc.org/libc v1.55.3/go.mod h1:qFXepLhz+JjFThQ4kzwzOjA/y/artDeg+pcYnY+Q83w=
modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4=
modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo=
modernc.org/memory v1.8.0 h1:IqGTL6eFMaDZZhEWwcREgeMXYwmW83LYW8cROZYkg+E=
modernc.org/memory v1.8.0/go.mod h1:XPZ936zp5OMKGWPqbD3JShgd/ZoQ7899TUuQqxY+peU=
modernc.org/sqlite v1.32.0 h1:6BM4uGza7bWypsw4fdLRsLxut6bHe4c58VeqjRgST8s=
modernc.org/sqlite v1.32.0/go.mod h1:UqoylwmTb9F+IqXERT8bW9zzOWN8qwAIcLdzeBZs4hA=
modernc.org/strutil v1.2.0 h1:agBi9dp1I+eOnxXeiZawM8F4LawKv4NzGWSaLfyeNZA=
modernc.org/strutil v1.2.0/go.mod h1:/mdcBmfOibveCTBxUl5B5l6W+TTH1FXPLHZE6bTosX0=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
5 changes: 2 additions & 3 deletions internal/database/database_pg.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,14 @@ import (
"database/sql"

_ "github.com/lib/pq"
"github.com/lukewhrit/spacebin/internal/config"
)

type Postgres struct {
*sql.DB
}

func NewPostgres() (Database, error) {
db, err := sql.Open("postgres", config.Config.ConnectionURI)
func NewPostgres(uri string) (Database, error) {
db, err := sql.Open("postgres", uri)

return &Postgres{db}, err
}
Expand Down
79 changes: 79 additions & 0 deletions internal/database/database_sqlite.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
/*
* Copyright 2020-2024 Luke Whritenour
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* http://www.apache.org/licenses/LICENSE-2.0
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package database

import (
"context"
"database/sql"
"sync"

_ "modernc.org/sqlite"
)

type SQLite struct {
*sql.DB
sync.RWMutex
}

func NewSQLite(filesath string) (Database, error) {
db, err := sql.Open("sqlite", filesath)

return &SQLite{db, sync.RWMutex{}}, err
}

func (s *SQLite) Migrate(ctx context.Context) error {
_, err := s.Exec(`
CREATE TABLE IF NOT EXISTS documents (
id TEXT PRIMARY KEY,
content TEXT NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
usdated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);`)

return err
}

func (s *SQLite) GetDocument(ctx context.Context, id string) (Document, error) {
s.RLock()
defer s.RUnlock()

doc := new(Document)
row := s.QueryRow("SELECT * FROM documents WHERE id=$1", id)
err := row.Scan(&doc.ID, &doc.Content, &doc.CreatedAt, &doc.UpdatedAt)

return *doc, err
}

func (s *SQLite) CreateDocument(ctx context.Context, id, content string) error {
s.Lock()
defer s.Unlock()

tx, err := s.Begin()

if err != nil {
return err
}

_, err = tx.Exec("INSERT INTO documents (id, content) VALUES ($1, $2)",
id, content) // created_at and updated_at are auto-generated

if err != nil {
return err
}

return tx.Commit()
}

0 comments on commit 81a849e

Please sign in to comment.