-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sql: row-level TTL behaves incorrectly with composite type primary key #116845
Comments
@DrewKimball pointed out that the cockroach/pkg/sql/row/fetcher.go Lines 821 to 824 in 0af1fed
|
Thought of another problem - the TTL code scans the first KV within the spans, but SQL rows (and even primary keys) can span multiple KVs because of column families. Here's an example where the composite value for the primary key is stored in the value of column family 1 (instead of 0):
|
That sounds similar to the reason we made this change: 794aedd |
BTW, I think decimals are probably safe - you'd drop trailing zeroes and convert negative zeroes to positive zeroes, but that's fine for this application. Looks like floats and JSON also have composite encodings, but I think they're also safe to just use the decoded key. |
Row-level TTL uses raw KV scans on the target table's primary key span to generate the query bounds used for its SELECT query. However, it does not handle composite types such as collated strings correctly and generates incorrect query bounds.
Namely,
(Key, Value)
pairs for a row in a table that has a composite type primary key will have part of the composite type's value encoded in theKey
(e.g. collation key for collated strings) and part of the value encoded in theValue
(e.g. contents for collated strings). (See https://github.com/cockroachdb/cockroach/blob/509efb2e24ec768e5a6921afddb65d794dd9f5e1/docs/tech-notes/encoding.md#composite-encoding for more details.) The TTL code only considers theKey
when converting the raw KVs into datums, leading to incorrect query bounds being generated, which can potentially result in rows being missed.Code pointer
cockroach/pkg/sql/ttl/ttljob/ttljob_processor.go
Lines 424 to 433 in cbb3441
Steps to reproduce
The TTL job generates a SELECT query that looks like
where
$2
and$3
are bothe'\x15\xef\x00\x00\x00 ' COLLATE en_US_u_ks_level2
instead of'a' COLLATE en_US_u_ks_level2
. Of note,e'\x15\xef\x00\x00\x00 '
is the collation key for'a' COLLATE en_US_u_ks_level2
.Jira issue: CRDB-34807
Epic CRDB-35306
The text was updated successfully, but these errors were encountered: