Improve Realtime Continuous Aggregate performance

[fabriziomello]

When calling the cagg_watermark function to get the watermark of a
Continuous Aggregate we execute a SELECT MAX(time_dimension) query
in the underlying materialization hypertable.

The problem is that a SELECT MAX(time_dimention) query can be
expensive because it will scan all hypertable chunks increasing the
planning time for a Realtime Continuous Aggregates.

Improved it by creating a new catalog table to serve as a cache table
to store the current Continous Aggregate watermark in the following
situations:

• Create CAgg: store the minimum value of hypertable time dimension
  data type;
• Refresh CAgg: store the last value of the time dimension materialized
  in the underlying materialization hypertable (or the minimum value of
  materialization hypertable time dimension data type if there's no
  data materialized);
• Drop CAgg Chunks: the same as refresh cagg.

Closes #4699, #5307

[fabriziomello]

Some preliminary results:

1. Last TimescaleDB release: 2.10.0

tsbench on  main [?] via tsbench took 12s 
➜ python -m src.tsbench --with-connection pgsql://fabrizio@localhost:5432/2.10.0 --benchmarks cagg_watermark              
*** Processing benchmarks on connection: pgsql://fabrizio@localhost:5432/2.10.0
*** Executing benchmark 'cagg_watermark'
*** All benchmarks are executed - done
============================================
Report for benchmark suite 'cagg_watermark'
+--------------------------------------------------------------------------------------+------------------------------------------+
| Query                                                                                | 8b549b08e28d121946eccbc7452bc476c7754dc8 |
+--------------------------------------------------------------------------------------+------------------------------------------+
| SELECT bucket, a, value FROM agg_1m  WHERE a = 1 AND bucket > '2023-01-01 00:00:00'; |                                     2.03 |
| SELECT bucket, a, value FROM agg_5m  WHERE a = 1 AND bucket > '2023-01-01 00:00:00'; |                                     8.89 |
| SELECT bucket, a, value FROM agg_15m WHERE a = 1 AND bucket > '2023-01-01 00:00:00'; |                                   158.18 |
+--------------------------------------------------------------------------------------+------------------------------------------+

Flamegraph:

2. This PR

tsbench on  main [?] via tsbench took 11s 
➜ python -m src.tsbench --with-connection pgsql://fabrizio@localhost:5432/fabrizio --benchmarks cagg_watermark --no-teardown
*** Processing benchmarks on connection: pgsql://fabrizio@localhost:5432/fabrizio
*** Executing benchmark 'cagg_watermark'
*** All benchmarks are executed - done
============================================
Report for benchmark suite 'cagg_watermark'
+--------------------------------------------------------------------------------------+------------------------------------------+
| Query                                                                                | b7ad720d8243fe17a14a257434beff7430bc97c7 |
+--------------------------------------------------------------------------------------+------------------------------------------+
| SELECT bucket, a, value FROM agg_1m  WHERE a = 1 AND bucket > '2023-01-01 00:00:00'; |                                     1.76 |
| SELECT bucket, a, value FROM agg_5m  WHERE a = 1 AND bucket > '2023-01-01 00:00:00'; |                                     3.17 |
| SELECT bucket, a, value FROM agg_15m WHERE a = 1 AND bucket > '2023-01-01 00:00:00'; |                                    28.63 |
+--------------------------------------------------------------------------------------+------------------------------------------+

Flamegraph:

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (main@699fcf4). Click here to learn what that means.
The diff coverage is 94.63%.

❗ Current head de52e20 differs from pull request most recent head db35aa2. Consider uploading reports for the commit db35aa2 to get more accurate results

@@           Coverage Diff           @@
##             main    #5261   +/-   ##
=======================================
  Coverage        ?   90.69%           
=======================================
  Files           ?      229           
  Lines           ?    53235           
  Branches        ?        0           
=======================================
  Hits            ?    48283           
  Misses          ?     4952           
  Partials        ?        0           

Impacted Files	Coverage Δ
src/ts_catalog/catalog.c	89.37% <ø> (ø)
src/ts_catalog/catalog.h	100.00% <ø> (ø)
src/chunk.c	93.71% <85.71%> (ø)
tsl/src/continuous_aggs/materialize.c	73.85% <88.23%> (ø)
src/ts_catalog/continuous_aggs_watermark.c	95.83% <95.83%> (ø)
src/hypertable.c	87.59% <100.00%> (ø)
src/ts_catalog/continuous_agg.c	96.47% <100.00%> (ø)
tsl/src/continuous_aggs/create.c	89.16% <100.00%> (ø)
tsl/src/continuous_aggs/invalidation_threshold.c	62.24% <100.00%> (ø)
tsl/src/continuous_aggs/refresh.c	97.68% <100.00%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[mkindahl]

In general it looks good. A few minor things that I think need to be fixed. There is a bigger question on why some of the error would be raised since these would rather be indications of us doing something wrong with the locking. Raising errors here are safe, since they would normally not fire, but not sure if we want to have some additional tests for these cases.

[mkindahl]

We've had problems with VACUUM previously, so I think we should be careful about releasing locks too early, but otherwise it looks good.

[timescale-automation]

Automated backport to 2.10.x not done: cherry-pick failed.

Git status

HEAD detached at origin/2.10.x
You are currently cherry-picking commit 38fcd1b7.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   CHANGELOG.md
	modified:   sql/pre_install/tables.sql
	modified:   sql/updates/post-update.sql
	modified:   sql/util_time.sql
	modified:   src/chunk.c
	modified:   src/hypertable.c
	modified:   src/hypertable.h
	modified:   src/ts_catalog/CMakeLists.txt
	modified:   src/ts_catalog/catalog.c
	modified:   src/ts_catalog/catalog.h
	modified:   src/ts_catalog/continuous_agg.c
	new file:   src/ts_catalog/continuous_aggs_watermark.c
	new file:   src/ts_catalog/continuous_aggs_watermark.h
	modified:   test/expected/drop_rename_hypertable.out
	modified:   tsl/src/continuous_aggs/create.c
	modified:   tsl/src/continuous_aggs/invalidation_threshold.c
	modified:   tsl/src/continuous_aggs/materialize.c
	modified:   tsl/src/continuous_aggs/materialize.h
	modified:   tsl/src/continuous_aggs/refresh.c
	modified:   tsl/test/expected/cagg_refresh.out
	modified:   tsl/test/expected/cagg_union_view-12.out
	modified:   tsl/test/expected/cagg_union_view-13.out
	modified:   tsl/test/expected/cagg_union_view-14.out
	modified:   tsl/test/expected/cagg_union_view-15.out
	modified:   tsl/test/shared/expected/extension.out
	modified:   tsl/test/sql/cagg_union_view.sql.in

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   sql/updates/latest-dev.sql
	both modified:   sql/updates/reverse-dev.sql

Job log