Backport lock dimension slice #2261
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add implementation for debug waitpoints. Debug waitpoints can be added
to code and can be enabled using the `ts_debug_waitpoint_enable`
function. Once execution reaches this point, the session will block
waiting for a call to `ts_debug_waitpoint_release`.
Waitpoints are added to code by using the macro `DEBUG_WAITPOINT` with
a string, for example:
DEBUG_WAITPOINT("references_fetched");
The string is hashed to compute a 32-bit number that is then used as an
shared advisory lock. The waitpoint can be enabled with the function
`ts_debug_waitpoint_enable`. This function takes a string and computes
a hash and will use the hash to take an exclusive advisory lock. This
will cause all sessions reaching the waitpoint to block until the lock
is released with `ts_debug_waitpoint_release`.
If a hypertable accidentally broke because a dimension slice is missing, a segmentation fault will result when an attempt is made to remove a chunk that references the dimension slice. This happens because no check if the dimension slice was found is made and the assumption is that it should be there (by design). Instead of crashing the server, this commit adds code that prints a warning that the dimension slice did not exist and proceed with removing the chunk. This is safe since the chunk should be removed anyway and the missing dimension slice does not change this.
In the function `ts_hypercube_from_constraints` a hypercube is build from constraints which reference dimension slices in `dimension_slice`. As part of a run of `drop_chunks` or when a chunk is explicitly dropped as part of other operations, dimension slices can be removed from this table causing the dimension slices to be removed, which makes the hypercube reference non-existent dimension slices which subsequently causes a crash. This commit fixes this by adding a tuple lock on the dimension slices that are used to build the hypercube. If two `drop_chunks` are running concurrently, there can be a race if dimension slices are removed as a result removing a chunk. We treat this case in the same way as if the dimension slice was updated: report an error that another session locked the tuple. Fixes timescale#1986
erimatnor
approved these changes
Aug 26, 2020
svenklemm
approved these changes
Aug 26, 2020
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Aug 26, 2020
This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address bugs in compression, drop_chunks and the background worker scheduler. **Bugfixes** * timescale#2059 Improve infering start and stop arguments from gapfill query * timescale#2067 Support moving compressed chunks * timescale#2068 Apply SET TABLESPACE for compressed chunks * timescale#2090 Fix index creation with IF NOT EXISTS for existing indexes * timescale#2092 Fix delete on tables involving hypertables with compression * timescale#2164 Fix telemetry installed_time format * timescale#2184 Fix background worker scheduler memory consumption * timescale#2222 Fix `negative bitmapset member not allowed` in decompression * timescale#2255 Propagate privileges from hypertables to chunks * timescale#2256 Fix segfault in chunk_append with space partitioning * timescale#2259 Fix recursion in cache processing * timescale#2261 Lock dimension slice tuple when scanning **Thanks** * @akamensky for reporting an issue with drop_chunks and ChunkAppend with space partitioning * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @fvannee for reporting an issue with cache invalidation * @nexces for reporting an issue with ChunkAppend on space-partitioned hypertables * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @sezaru for reporting an issue with background worker scheduler memory consumption
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Aug 27, 2020
This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address bugs in compression, drop_chunks and the background worker scheduler. **Bugfixes** * timescale#2059 Improve infering start and stop arguments from gapfill query * timescale#2067 Support moving compressed chunks * timescale#2068 Apply SET TABLESPACE for compressed chunks * timescale#2090 Fix index creation with IF NOT EXISTS for existing indexes * timescale#2092 Fix delete on tables involving hypertables with compression * timescale#2164 Fix telemetry installed_time format * timescale#2184 Fix background worker scheduler memory consumption * timescale#2222 Fix `negative bitmapset member not allowed` in decompression * timescale#2255 Propagate privileges from hypertables to chunks * timescale#2256 Fix segfault in chunk_append with space partitioning * timescale#2259 Fix recursion in cache processing * timescale#2261 Lock dimension slice tuple when scanning **Thanks** * @akamensky for reporting an issue with drop_chunks and ChunkAppend with space partitioning * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @fvannee for reporting an issue with cache invalidation * @nexces for reporting an issue with ChunkAppend on space-partitioned hypertables * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @sezaru for reporting an issue with background worker scheduler memory consumption
svenklemm
added a commit
to svenklemm/timescaledb
that referenced
this pull request
Aug 27, 2020
This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in compression, drop_chunks and the background worker scheduler. **Bugfixes** * timescale#2059 Improve infering start and stop arguments from gapfill query * timescale#2067 Support moving compressed chunks * timescale#2068 Apply SET TABLESPACE for compressed chunks * timescale#2090 Fix index creation with IF NOT EXISTS for existing indexes * timescale#2092 Fix delete on tables involving hypertables with compression * timescale#2164 Fix telemetry installed_time format * timescale#2184 Fix background worker scheduler memory consumption * timescale#2222 Fix `negative bitmapset member not allowed` in decompression * timescale#2255 Propagate privileges from hypertables to chunks * timescale#2256 Fix segfault in chunk_append with space partitioning * timescale#2259 Fix recursion in cache processing * timescale#2261 Lock dimension slice tuple when scanning **Thanks** * @akamensky for reporting an issue with drop_chunks and ChunkAppend with space partitioning * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @fvannee for reporting an issue with cache invalidation * @nexces for reporting an issue with ChunkAppend on space-partitioned hypertables * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @prathamesh-sonpatki for contributing a typo fix * @sezaru for reporting an issue with background worker scheduler memory consumption
Merged
svenklemm
added a commit
that referenced
this pull request
Aug 27, 2020
This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in compression, drop_chunks and the background worker scheduler. **Bugfixes** * #2059 Improve infering start and stop arguments from gapfill query * #2067 Support moving compressed chunks * #2068 Apply SET TABLESPACE for compressed chunks * #2090 Fix index creation with IF NOT EXISTS for existing indexes * #2092 Fix delete on tables involving hypertables with compression * #2164 Fix telemetry installed_time format * #2184 Fix background worker scheduler memory consumption * #2222 Fix `negative bitmapset member not allowed` in decompression * #2255 Propagate privileges from hypertables to chunks * #2256 Fix segfault in chunk_append with space partitioning * #2259 Fix recursion in cache processing * #2261 Lock dimension slice tuple when scanning **Thanks** * @akamensky for reporting an issue with drop_chunks and ChunkAppend with space partitioning * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @fvannee for reporting an issue with cache invalidation * @nexces for reporting an issue with ChunkAppend on space-partitioned hypertables * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @prathamesh-sonpatki for contributing a typo fix * @sezaru for reporting an issue with background worker scheduler memory consumption
svenklemm
added a commit
that referenced
this pull request
Aug 27, 2020
This maintenance release contains bugfixes since the 1.7.2 release. We deem it high priority for upgrading. In particular the fixes contained in this maintenance release address issues in compression, drop_chunks and the background worker scheduler. **Bugfixes** * #2059 Improve infering start and stop arguments from gapfill query * #2067 Support moving compressed chunks * #2068 Apply SET TABLESPACE for compressed chunks * #2090 Fix index creation with IF NOT EXISTS for existing indexes * #2092 Fix delete on tables involving hypertables with compression * #2164 Fix telemetry installed_time format * #2184 Fix background worker scheduler memory consumption * #2222 Fix `negative bitmapset member not allowed` in decompression * #2255 Propagate privileges from hypertables to chunks * #2256 Fix segfault in chunk_append with space partitioning * #2259 Fix recursion in cache processing * #2261 Lock dimension slice tuple when scanning **Thanks** * @akamensky for reporting an issue with drop_chunks and ChunkAppend with space partitioning * @dewetburger430 for reporting an issue with setting tablespace for compressed chunks * @fvannee for reporting an issue with cache invalidation * @nexces for reporting an issue with ChunkAppend on space-partitioned hypertables * @PichetGoulu for reporting an issue with index creation and IF NOT EXISTS * @prathamesh-sonpatki for contributing a typo fix * @sezaru for reporting an issue with background worker scheduler memory consumption
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a backport of #2150. Changes are:
follow_updatesinstead oflockflagsin theScanTupLock.drop_chunks.