Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Code indexer crashes gitea on i386/x86 with "panic: unaligned 64-bit atomic operation" #21957

Closed
sebastianertz opened this issue Nov 27, 2022 · 9 comments · Fixed by #22031
Closed
Labels

Comments

@sebastianertz
Copy link

Description

On an i386 system gitea 1.17.3 runs perfectly.
Version 1.18.0-rc0 and 1.18.0-rc1 crashes when indexing code.

Gitea Version

1.18.0-rc0 and 1.18.0-rc1

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

panic: unaligned 64-bit atomic operation

goroutine 2044 [running]:
runtime/internal/atomic.panicUnaligned()
        /usr/local/go/src/runtime/internal/atomic/unaligned.go:8 +0x2d
runtime/internal/atomic.Xadd64(0xf9ede74, 0x3b)
        /usr/local/go/src/runtime/internal/atomic/atomic_386.s:125 +0x11
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).incrementBytesWritten(...)
        /go/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.5/contentcoder.go:114
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).flushContents(0xf9ede00)
        /go/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.5/contentcoder.go:143 +0x24a
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).Close(...)
        /go/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.5/contentcoder.go:110
github.com/blevesearch/zapx/v15.(*interim).writeDicts(0xe334a50)
        /go/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.5/new.go:803 +0x1443
github.com/blevesearch/zapx/v15.(*interim).convert(0xe334a50)
        /go/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.5/new.go:277 +0x2b5
github.com/blevesearch/zapx/v15.(*ZapPlugin).newWithChunkMode(0xd1bbf88, {0xe434ac0, 0x8, 0x8}, 0x402)
        /go/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.5/new.go:72 +0x18b
github.com/blevesearch/zapx/v15.(*ZapPlugin).New(0xd1bbf88, {0xe434ac0, 0x8, 0x8})
        /go/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.5/new.go:47 +0x43
github.com/blevesearch/bleve/v2/index/scorch.(*Scorch).Batch(0x10dfa000, 0xf5459c0)
        /go/pkg/mod/github.com/blevesearch/bleve/v2@v2.3.4/index/scorch/scorch.go:411 +0x5ff
github.com/blevesearch/bleve/v2.(*indexImpl).Batch(0x10de1720, 0x10e15060)
        /go/pkg/mod/github.com/blevesearch/bleve/v2@v2.3.4/index_impl.go:309 +0xa0
code.gitea.io/gitea/modules/indexer/bleve.(*FlushingBatch).Flush(0xf861da0)
        /source/modules/indexer/bleve/batch.go:53 +0x32
code.gitea.io/gitea/modules/indexer/code.(*BleveIndexer).Index(0xe64df10, {0xba44760, 0xe666740}, 0xe31cd80, {0x12299400, 0x28}, 0xfb440a8)
        /source/modules/indexer/code/bleve.go:311 +0x407
code.gitea.io/gitea/modules/indexer/code.(*wrappedIndexer).Index(0xe82a570, {0xba44760, 0xe666740}, 0xe31cd80, {0x12299400, 0x28}, 0xfb440a8)
        /source/modules/indexer/code/wrapped.go:85 +0x7d
code.gitea.io/gitea/modules/indexer/code.index({0xba44760, 0xe666740}, {0xba476ec, 0xe82a570}, 0x81)
        /source/modules/indexer/code/indexer.go:107 +0x11f
code.gitea.io/gitea/modules/indexer/code.Init.func2({0xf37c6e0, 0x1, 0x14})
        /source/modules/indexer/code/indexer.go:157 +0x1e0
code.gitea.io/gitea/modules/queue.NewByteFIFOUniqueQueue.func1({0xf37c6e0, 0x1, 0x14})
        /source/modules/queue/queue_bytefifo.go:399 +0x3e
code.gitea.io/gitea/modules/queue.(*WorkerPool).doWork(0x10e61080, {0xba44820, 0xe3fcbc0})
        /source/modules/queue/workerpool.go:564 +0x69f
code.gitea.io/gitea/modules/queue.(*WorkerPool).addWorkers.func1()
        /source/modules/queue/workerpool.go:312 +0x5d
created by code.gitea.io/gitea/modules/queue.(*WorkerPool).addWorkers
        /source/modules/queue/workerpool.go:310 +0x25

Git Version

2.35.1

Operating System

eisfair.org

How are you running Gitea?

Self hosted, binary from dl.gitea.org

Database

SQLite

@zeripath
Copy link
Contributor

Duplicate of #21869

@zeripath zeripath marked this as a duplicate of #21869 Nov 27, 2022
@zeripath
Copy link
Contributor

As stated in #21869 are you really using a 32bit OS and not a 64bit OS?

@sebastianertz
Copy link
Author

It's a 32bit OS

uname -a
Linux eis 5.15.75-eisfair-1-VIRT #1 SMP Wed Oct 26 23:56:22 CEST 2022 i686 i686 i386 GNU/Linux
uname -m
i686

@zeripath
Copy link
Contributor

zeripath commented Dec 4, 2022

OK now that zapx have merged my PR and updated the tag we can pull the new tag and rebuild.

@sebastianertz would it be possible for you to try #22031?

zeripath added a commit to zeripath/gitea that referenced this issue Dec 4, 2022
There is an unaligned atomic field in zapx 15.3.5 which should have been fixed in blevesearch/zapx#147

This bug causes issues on 32bit builds.

Update bleve and zapx to account for this.

Fix go-gitea#21957

Signed-off-by: Andrew Thornton <art27@cantab.net>
@sebastianertz
Copy link
Author

@zeripath Same issue:

panic: unaligned 64-bit atomic operation

goroutine 3397 [running]:
runtime/internal/atomic.panicUnaligned()
	/home/se/go/src/runtime/internal/atomic/unaligned.go:8 +0x2d
runtime/internal/atomic.Xadd64(0x10125574, 0x3b)
	/home/se/go/src/runtime/internal/atomic/atomic_386.s:125 +0x11
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).incrementBytesWritten(...)
	/home/se/gopath/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.6/contentcoder.go:114
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).flushContents(0x10125500)
	/home/se/gopath/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.6/contentcoder.go:143 +0x24a
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).Close(...)
	/home/se/gopath/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.6/contentcoder.go:110
github.com/blevesearch/zapx/v15.(*interim).writeDicts(0xf2caff0)
	/home/se/gopath/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.6/new.go:803 +0x1443
github.com/blevesearch/zapx/v15.(*interim).convert(0xf2caff0)
	/home/se/gopath/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.6/new.go:277 +0x2b5
github.com/blevesearch/zapx/v15.(*ZapPlugin).newWithChunkMode(0xd11a688, {0x10338080, 0x8, 0x8}, 0x402)
	/home/se/gopath/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.6/new.go:72 +0x18b
github.com/blevesearch/zapx/v15.(*ZapPlugin).New(0xd11a688, {0x10338080, 0x8, 0x8})
	/home/se/gopath/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.6/new.go:47 +0x43
github.com/blevesearch/bleve/v2/index/scorch.(*Scorch).Batch(0xe36bc00, 0xfb28dd0)
	/home/se/gopath/pkg/mod/github.com/blevesearch/bleve/v2@v2.3.5/index/scorch/scorch.go:412 +0x5ff
github.com/blevesearch/bleve/v2.(*indexImpl).Batch(0x10234190, 0xfafcb80)
	/home/se/gopath/pkg/mod/github.com/blevesearch/bleve/v2@v2.3.5/index_impl.go:310 +0xa0
code.gitea.io/gitea/modules/indexer/bleve.(*FlushingBatch).Flush(0x10375da0)
	/home/se/gitea/modules/indexer/bleve/batch.go:53 +0x32
code.gitea.io/gitea/modules/indexer/code.(*BleveIndexer).Index(0xd4a59f0, {0xb9c70e0, 0xe365790}, 0xf725320, {0xfaa4800, 0x28}, 0x117fbe60)
	/home/se/gitea/modules/indexer/code/bleve.go:311 +0x407
code.gitea.io/gitea/modules/indexer/code.(*wrappedIndexer).Index(0xe706750, {0xb9c70e0, 0xe365790}, 0xf725320, {0xfaa4800, 0x28}, 0x117fbe60)
	/home/se/gitea/modules/indexer/code/wrapped.go:85 +0x7d
code.gitea.io/gitea/modules/indexer/code.index({0xb9c70e0, 0xe365790}, {0xb9ca06c, 0xe706750}, 0x81)
	/home/se/gitea/modules/indexer/code/indexer.go:107 +0x11f
code.gitea.io/gitea/modules/indexer/code.Init.func2({0x11c79540, 0x1, 0x14})
	/home/se/gitea/modules/indexer/code/indexer.go:157 +0x1e0
code.gitea.io/gitea/modules/queue.NewByteFIFOUniqueQueue.func1({0x11c79540, 0x1, 0x14})
	/home/se/gitea/modules/queue/queue_bytefifo.go:399 +0x3e
code.gitea.io/gitea/modules/queue.(*WorkerPool).doWork(0x104a2ae0, {0xb9c71a0, 0x10304e80})
	/home/se/gitea/modules/queue/workerpool.go:564 +0x69f
code.gitea.io/gitea/modules/queue.(*WorkerPool).addWorkers.func1()
	/home/se/gitea/modules/queue/workerpool.go:312 +0x5d
created by code.gitea.io/gitea/modules/queue.(*WorkerPool).addWorkers
	/home/se/gitea/modules/queue/workerpool.go:310 +0x25

@sebastianertz
Copy link
Author

@zeripath Your zapx PR is in version v15.3.8

@zeripath
Copy link
Contributor

zeripath commented Dec 5, 2022

Damn. That brings incompatible changes with it.

@zeripath
Copy link
Contributor

zeripath commented Dec 5, 2022

Please try again. I've managed to make a dedicated backport with the alignment fix

@sebastianertz
Copy link
Author

Same:

panic: unaligned 64-bit atomic operation

goroutine 2534 [running]:
runtime/internal/atomic.panicUnaligned()
	/home/se/go/src/runtime/internal/atomic/unaligned.go:8 +0x2d
runtime/internal/atomic.Xadd64(0x1048873c, 0x3b)
	/home/se/go/src/runtime/internal/atomic/atomic_386.s:125 +0x11
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).incrementBytesWritten(...)
	/home/se/gopath/pkg/mod/github.com/zeripath/zapx/v15@v15.3.6-alignment-fix/contentcoder.go:114
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).flushContents(0x10488700)
	/home/se/gopath/pkg/mod/github.com/zeripath/zapx/v15@v15.3.6-alignment-fix/contentcoder.go:143 +0x24a
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).Close(...)
	/home/se/gopath/pkg/mod/github.com/zeripath/zapx/v15@v15.3.6-alignment-fix/contentcoder.go:110
github.com/blevesearch/zapx/v15.(*interim).writeDicts(0xf525ef0)
	/home/se/gopath/pkg/mod/github.com/zeripath/zapx/v15@v15.3.6-alignment-fix/new.go:803 +0x1443
github.com/blevesearch/zapx/v15.(*interim).convert(0xf525ef0)
	/home/se/gopath/pkg/mod/github.com/zeripath/zapx/v15@v15.3.6-alignment-fix/new.go:277 +0x2b5
github.com/blevesearch/zapx/v15.(*ZapPlugin).newWithChunkMode(0xd1b337c, {0xe65fb80, 0x8, 0x8}, 0x402)
	/home/se/gopath/pkg/mod/github.com/zeripath/zapx/v15@v15.3.6-alignment-fix/new.go:72 +0x18b
github.com/blevesearch/zapx/v15.(*ZapPlugin).New(0xd1b337c, {0xe65fb80, 0x8, 0x8})
	/home/se/gopath/pkg/mod/github.com/zeripath/zapx/v15@v15.3.6-alignment-fix/new.go:47 +0x43
github.com/blevesearch/bleve/v2/index/scorch.(*Scorch).Batch(0x10306c00, 0x10c07f90)
	/home/se/gopath/pkg/mod/github.com/blevesearch/bleve/v2@v2.3.5/index/scorch/scorch.go:412 +0x5ff
github.com/blevesearch/bleve/v2.(*indexImpl).Batch(0x1036f680, 0xf499980)
	/home/se/gopath/pkg/mod/github.com/blevesearch/bleve/v2@v2.3.5/index_impl.go:310 +0xa0
code.gitea.io/gitea/modules/indexer/bleve.(*FlushingBatch).Flush(0x100f7da0)
	/home/se/gitea-zeripath/modules/indexer/bleve/batch.go:52 +0x32
code.gitea.io/gitea/modules/indexer/code.(*BleveIndexer).Index(0xf0d9540, {0xba537c0, 0xf0c7fa0}, 0xf88e7e0, {0xf7abc00, 0x28}, 0x11755e30)
	/home/se/gitea-zeripath/modules/indexer/code/bleve.go:310 +0x407
code.gitea.io/gitea/modules/indexer/code.(*wrappedIndexer).Index(0xf26d920, {0xba537c0, 0xf0c7fa0}, 0xf88e7e0, {0xf7abc00, 0x28}, 0x11755e30)
	/home/se/gitea-zeripath/modules/indexer/code/wrapped.go:84 +0x7d
code.gitea.io/gitea/modules/indexer/code.index({0xba537c0, 0xf0c7fa0}, {0xba5674c, 0xf26d920}, 0x81)
	/home/se/gitea-zeripath/modules/indexer/code/indexer.go:106 +0x11b
code.gitea.io/gitea/modules/indexer/code.Init.func2({0x10185ea0, 0x1, 0x14})
	/home/se/gitea-zeripath/modules/indexer/code/indexer.go:156 +0x1e0
code.gitea.io/gitea/modules/queue.NewByteFIFOUniqueQueue.func1({0x10185ea0, 0x1, 0x14})
	/home/se/gitea-zeripath/modules/queue/queue_bytefifo.go:398 +0x3e
code.gitea.io/gitea/modules/queue.(*WorkerPool).doWork(0x10280840, {0xba53880, 0xe621500})
	/home/se/gitea-zeripath/modules/queue/workerpool.go:563 +0x69f
code.gitea.io/gitea/modules/queue.(*WorkerPool).addWorkers.func1()
	/home/se/gitea-zeripath/modules/queue/workerpool.go:311 +0x5d
created by code.gitea.io/gitea/modules/queue.(*WorkerPool).addWorkers
	/home/se/gitea-zeripath/modules/queue/workerpool.go:309 +0x25

jolheiser pushed a commit that referenced this issue Dec 22, 2022
There is an unaligned atomic field in zapx 15.3.5 which should have been
fixed in a subsequent patch

This bug causes issues on 32bit builds.

Update bleve and zapx to account for this.

Fix #21957

Signed-off-by: Andrew Thornton <art27@cantab.net>
zeripath added a commit to zeripath/gitea that referenced this issue Dec 22, 2022
Backport go-gitea#22031

There is an unaligned atomic field in zapx 15.3.5 which should have been
fixed in a subsequent patch

This bug causes issues on 32bit builds.

Update bleve and zapx to account for this.

Fix go-gitea#21957

Signed-off-by: Andrew Thornton <art27@cantab.net>
jolheiser pushed a commit that referenced this issue Dec 22, 2022
Backport #22031

There is an unaligned atomic field in zapx 15.3.5 which should have been
fixed in a subsequent patch

This bug causes issues on 32bit builds.

Update bleve and zapx to account for this.

Fix #21957

Signed-off-by: Andrew Thornton <art27@cantab.net>
@go-gitea go-gitea locked and limited conversation to collaborators May 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants