Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add user settings key/value DB table #16834

Merged
merged 72 commits into from
Nov 22, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
515311b
add user settings k/v DB table
techknowlogick Aug 27, 2021
40c45f4
resolve lint issues
techknowlogick Aug 29, 2021
7a35bc3
Merge branch 'main' into user-settings
techknowlogick Aug 29, 2021
a0a1dac
add migration & tests
techknowlogick Aug 29, 2021
d6c169c
fix lint
techknowlogick Aug 29, 2021
9614a3f
fix sql query
techknowlogick Aug 29, 2021
147a52c
update per Lunny's feedback
techknowlogick Aug 30, 2021
03f8bca
Merge branch 'main' into user-settings
techknowlogick Aug 30, 2021
0f2bd2b
Merge branch 'main' into user-settings
techknowlogick Aug 30, 2021
2132a96
Merge branch 'main' into user-settings
techknowlogick Aug 31, 2021
ebae68e
Update models/user_setting.go
techknowlogick Sep 30, 2021
9f72e70
Rename v193.go to v196.go
techknowlogick Oct 3, 2021
a3a8271
Rename v196.go to v198.go
techknowlogick Oct 3, 2021
10c4a7c
Merge remote-tracking branch 'upstream/main' into user-settings
techknowlogick Oct 3, 2021
caaf873
refactor PR from recent models/db refactor
techknowlogick Oct 3, 2021
f04598c
refactor PR from recent models/db refactor
techknowlogick Oct 3, 2021
d82b4de
use correct way to start new session
techknowlogick Oct 3, 2021
d48c711
fix tests
techknowlogick Oct 3, 2021
7ea3b8a
Merge branch 'main' into user-settings
techknowlogick Oct 3, 2021
e67e72f
Merge branch 'main' into user-settings
techknowlogick Oct 7, 2021
0314c56
Apply suggestions from code review
techknowlogick Oct 7, 2021
2890b7b
Update models/migrations/v197.go
techknowlogick Oct 7, 2021
e7bce0c
Update models/error.go
techknowlogick Oct 7, 2021
851eb3d
Update models/user_setting.go
techknowlogick Oct 7, 2021
4c56f95
update per delvh feedback
techknowlogick Oct 7, 2021
ee8104c
rename to 198
techknowlogick Oct 10, 2021
844fb46
Merge branch 'main' into user-settings
techknowlogick Oct 10, 2021
f5fe983
upsert & add const to test
techknowlogick Oct 10, 2021
70df32f
rm dead code
techknowlogick Oct 10, 2021
d18b2f9
mk fmt
techknowlogick Oct 10, 2021
d2e9d1c
Rename v198.go to v199.go
techknowlogick Oct 15, 2021
ea98352
Merge branch 'main' into user-settings
techknowlogick Oct 15, 2021
595abd0
temp move migration to a higher number to make merge conflict easier
techknowlogick Nov 8, 2021
553b583
use correct migration number
techknowlogick Nov 8, 2021
261fcbb
update per feedback
techknowlogick Nov 9, 2021
1c1d85a
swap to map
techknowlogick Nov 9, 2021
5d98572
return error when key not lower case
techknowlogick Nov 10, 2021
7401b41
Merge branch 'main' into user-settings
wxiaoguang Nov 10, 2021
cbb145c
match migration struct to user setting struct
techknowlogick Nov 10, 2021
d8aeec7
mv files
techknowlogick Nov 10, 2021
d8e1c18
fix lint
techknowlogick Nov 10, 2021
4489d39
change struct name
techknowlogick Nov 10, 2021
cb50d48
fix caps
techknowlogick Nov 10, 2021
5f719fc
add comment to placate lint
techknowlogick Nov 10, 2021
e8b2f2c
Merge branch 'main' into user-settings
techknowlogick Nov 11, 2021
ba39900
pass tests
techknowlogick Nov 11, 2021
d6c4bb1
Merge branch 'main' into user-settings
techknowlogick Nov 12, 2021
46ba3b7
Merge branch 'main' into user-settings
techknowlogick Nov 13, 2021
4af49fd
Merge branch 'main' into user-settings
techknowlogick Nov 18, 2021
ee6ad18
Merge branch 'main' into user-settings
techknowlogick Nov 19, 2021
09841e6
Update setting_test.go
techknowlogick Nov 19, 2021
23b43b5
Merge branch 'main' into user-settings
techknowlogick Nov 19, 2021
c363283
Merge branch 'main' into user-settings
techknowlogick Nov 19, 2021
a28c975
Merge branch 'main' into user-settings
techknowlogick Nov 20, 2021
da22337
Merge branch 'main' into user-settings
techknowlogick Nov 20, 2021
d39f3c6
Merge branch 'main' into user-settings
techknowlogick Nov 20, 2021
c18f5c0
Merge branch 'main' into user-settings
techknowlogick Nov 21, 2021
43b32d0
swap upsert logic
techknowlogick Nov 21, 2021
00950c6
woops, only update key for specific user
techknowlogick Nov 21, 2021
cef5e77
update per lunny feedback
techknowlogick Nov 21, 2021
374edb2
Merge branch 'main' into user-settings
techknowlogick Nov 21, 2021
63f71fe
Update setting.go
wxiaoguang Nov 21, 2021
83fa71e
Update setting_test.go
wxiaoguang Nov 21, 2021
c81f0b8
Update setting.go comments
wxiaoguang Nov 21, 2021
84aac13
Update setting.go
wxiaoguang Nov 21, 2021
ee57a24
proper casing
techknowlogick Nov 21, 2021
25fbcae
more casing changes
techknowlogick Nov 21, 2021
73a869e
fix transaction session and unit test
wxiaoguang Nov 22, 2021
5d28949
remove unnecessary SQL sorting
wxiaoguang Nov 22, 2021
162bcbe
use WithTx instead of TxContext
wxiaoguang Nov 22, 2021
8f004d1
Merge branch 'main' into user-settings
wxiaoguang Nov 22, 2021
d37c7c8
Merge branch 'main' into user-settings
lunny Nov 22, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cmd/web_https.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import (
"code.gitea.io/gitea/modules/graceful"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/setting"

"github.com/klauspost/cpuid/v2"
)

Expand Down
2 changes: 2 additions & 0 deletions models/migrations/migrations.go
Original file line number Diff line number Diff line change
Expand Up @@ -357,6 +357,8 @@ var migrations = []Migration{
NewMigration("Add table app_state", addTableAppState),
// v201 -> v202
NewMigration("Drop table remote_version (if exists)", dropTableRemoteVersion),
// v202 -> v203
NewMigration("Create key/value table for user settings", createUserSettingsTable),
}

// GetCurrentDBVersion returns the current db version
Expand Down
25 changes: 25 additions & 0 deletions models/migrations/v202.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// Copyright 2021 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

package migrations

import (
"fmt"

"xorm.io/xorm"
)

func createUserSettingsTable(x *xorm.Engine) error {
type UserSetting struct {
ID int64 `xorm:"pk autoincr"`
UserID int64 `xorm:"index unique(key_userid)"` // to load all of someone's settings
SettingKey string `xorm:"varchar(255) index unique(key_userid)"` // ensure key is always lowercase
SettingValue string `xorm:"text"`
}
if err := x.Sync2(new(UserSetting)); err != nil {
return fmt.Errorf("sync2: %v", err)
}
return nil

}
1 change: 1 addition & 0 deletions models/user.go
Original file line number Diff line number Diff line change
Expand Up @@ -1192,6 +1192,7 @@ func DeleteUser(ctx context.Context, u *User) (err error) {
&TeamUser{UID: u.ID},
&Collaboration{UserID: u.ID},
&Stopwatch{UserID: u.ID},
&user_model.Setting{UserID: u.ID},
); err != nil {
return fmt.Errorf("deleteBeans: %v", err)
}
Expand Down
2 changes: 1 addition & 1 deletion models/user/main_test.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Copyright 2020 The Gitea Authors. All rights reserved.
// Copyright 2021 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

Expand Down
116 changes: 116 additions & 0 deletions models/user/setting.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
// Copyright 2021 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

package user

import (
"context"
"fmt"
"strings"

"code.gitea.io/gitea/models/db"

"xorm.io/builder"
)

// Setting is a key value store of user settings
type Setting struct {
ID int64 `xorm:"pk autoincr"`
UserID int64 `xorm:"index unique(key_userid)"` // to load all of someone's settings
SettingKey string `xorm:"varchar(255) index unique(key_userid)"` // ensure key is always lowercase
SettingValue string `xorm:"text"`
}

// TableName sets the table name for the settings struct
func (s *Setting) TableName() string {
return "user_setting"
}

func init() {
db.RegisterModel(new(Setting))
}

// GetSettings returns specific settings from user
func GetSettings(uid int64, keys []string) (map[string]*Setting, error) {
settings := make([]*Setting, 0, len(keys))
if err := db.GetEngine(db.DefaultContext).
Where("user_id=?", uid).
And(builder.In("setting_key", keys)).
Find(&settings); err != nil {
return nil, err
}
settingsMap := make(map[string]*Setting)
for _, s := range settings {
settingsMap[s.SettingKey] = s
}
return settingsMap, nil
}

// GetUserAllSettings returns all settings from user
func GetUserAllSettings(uid int64) (map[string]*Setting, error) {
settings := make([]*Setting, 0, 5)
if err := db.GetEngine(db.DefaultContext).
Where("user_id=?", uid).
Find(&settings); err != nil {
return nil, err
}
settingsMap := make(map[string]*Setting)
for _, s := range settings {
settingsMap[s.SettingKey] = s
}
return settingsMap, nil
}

// DeleteSetting deletes a specific setting for a user
func DeleteSetting(setting *Setting) error {
_, err := db.GetEngine(db.DefaultContext).Delete(setting)
return err
}

// SetSetting updates a users' setting for a specific key
func SetSetting(setting *Setting) error {
if strings.ToLower(setting.SettingKey) != setting.SettingKey {
return fmt.Errorf("setting key should be lowercase")
}
return upsertSettingValue(setting.UserID, setting.SettingKey, setting.SettingValue)
}

func upsertSettingValue(userID int64, key string, value string) error {
return db.WithTx(func(ctx context.Context) error {
e := db.GetEngine(ctx)

// here we use a general method to do a safe upsert for different databases (and most transaction levels)
// 1. try to UPDATE the record and acquire the transaction write lock
// if UPDATE returns non-zero rows are changed, OK, the setting is saved correctly
// if UPDATE returns "0 rows changed", two possibilities: (a) record doesn't exist (b) value is not changed
// 2. do a SELECT to check if the row exists or not (we already have the transaction lock)
// 3. if the row doesn't exist, do an INSERT (we are still protected by the transaction lock, so it's safe)
//
// to optimize the SELECT in step 2, we can use an extra column like `revision=revision+1`
// to make sure the UPDATE always returns a non-zero value for existing (unchanged) records.

res, err := e.Exec("UPDATE user_setting SET setting_value=? WHERE setting_key=? AND user_id=?", value, key, userID)
if err != nil {
wxiaoguang marked this conversation as resolved.
Show resolved Hide resolved
return err
}
rows, _ := res.RowsAffected()
if rows > 0 {
// the existing row is updated, so we can return
return nil
}

// in case the value isn't changed, update would return 0 rows changed, so we need this check
has, err := e.Exist(&Setting{UserID: userID, SettingKey: key})
if err != nil {
return err
}
if has {
return nil
}

// if no existing row, insert a new row
_, err = e.Insert(&Setting{UserID: userID, SettingKey: key, SettingValue: value})
return err
})
}
51 changes: 51 additions & 0 deletions models/user/setting_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
// Copyright 2021 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

package user

import (
"testing"

"code.gitea.io/gitea/models/unittest"

"github.com/stretchr/testify/assert"
)

func TestSettings(t *testing.T) {
keyName := "test_user_setting"
assert.NoError(t, unittest.PrepareTestDatabase())

newSetting := &Setting{UserID: 99, SettingKey: keyName, SettingValue: "Gitea User Setting Test"}

// create setting
err := SetSetting(newSetting)
assert.NoError(t, err)
// test about saving unchanged values
err = SetSetting(newSetting)
assert.NoError(t, err)

// get specific setting
settings, err := GetSettings(99, []string{keyName})
assert.NoError(t, err)
assert.Len(t, settings, 1)
assert.EqualValues(t, newSetting.SettingValue, settings[keyName].SettingValue)

// updated setting
updatedSetting := &Setting{UserID: 99, SettingKey: keyName, SettingValue: "Updated"}
err = SetSetting(updatedSetting)
assert.NoError(t, err)

// get all settings
settings, err = GetUserAllSettings(99)
assert.NoError(t, err)
assert.Len(t, settings, 1)
assert.EqualValues(t, updatedSetting.SettingValue, settings[updatedSetting.SettingKey].SettingValue)

// delete setting
err = DeleteSetting(&Setting{UserID: 99, SettingKey: keyName})
assert.NoError(t, err)
settings, err = GetUserAllSettings(99)
assert.NoError(t, err)
assert.Len(t, settings, 0)
}
2 changes: 1 addition & 1 deletion modules/avatar/identicon/identicon_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
// license that can be found in the LICENSE file.

//go:build test_avatar_identicon
// +build test_avatar_identicon
// +build test_avatar_identicon

package identicon

Expand Down