Skip to content
This repository has been archived by the owner on Dec 8, 2021. It is now read-only.

*: parse the data source directly into data and skip the KV encoder #145

Merged
merged 17 commits into from
Apr 3, 2019

Commits on Mar 23, 2019

  1. *: parse the data source directly into data and skip the KV encoder

    This skips the more complex pingcap/parser, and speeds up parsing speed
    by 50%.
    
    We have also refactored the KV delivery mechanism to use channels
    directly, and revamped metrics:
    
    - Make the metrics about engines into its own `engines` counter. The
      `tables` counter is exclusively about tables now.
    - Removed `block_read_seconds`, `block_read_bytes`, `block_encode_seconds`
      since the concept of "block" no longer applies. Replaced by the
      equivalents named `row_***`.
    - Removed `chunk_parser_read_row_seconds` for being overlapping with
      `row_read_seconds`.
    - Changed `block_deliver_bytes` into a histogram vec, with kind=index
      or kind=data. Introduced `block_deliver_kv_pairs`.
    kennytm committed Mar 23, 2019
    Configuration menu
    Copy the full SHA
    b00ba33 View commit details
    Browse the repository at this point in the history
  2. tests,restore: prevent spurious error in checkpoint_chunks test

    Only kill Lightning if the whole chunk is imported exactly. The chunk
    checkpoint may be recorded before a chunk is fully written, and this will
    hit the failpoint more than 5 times.
    kennytm committed Mar 23, 2019
    Configuration menu
    Copy the full SHA
    27f5be8 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    b645921 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b15a293 View commit details
    Browse the repository at this point in the history
  5. common: disable IsContextCanceledError() when log level = debug

    This helps debugging some mysterious cancellation where the log is
    inhibited.
    
    Added IsReallyContextCanceledError() for code logic affected by error
    type.
    kennytm committed Mar 23, 2019
    Configuration menu
    Copy the full SHA
    e17c94d View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    91669db View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    a41fa4d View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    6aa40ab View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    7e86c78 View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2019

  1. Configuration menu
    Copy the full SHA
    98e1c18 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    27872b3 View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2019

  1. Configuration menu
    Copy the full SHA
    fa32cf7 View commit details
    Browse the repository at this point in the history
  2. *: addressed comments

    kennytm committed Apr 2, 2019
    Configuration menu
    Copy the full SHA
    c94bffa View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    8edafc6 View commit details
    Browse the repository at this point in the history
  4. *: addressed comments

    kennytm committed Apr 2, 2019
    Configuration menu
    Copy the full SHA
    b4bbacc View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2019

  1. Configuration menu
    Copy the full SHA
    9e4f601 View commit details
    Browse the repository at this point in the history
  2. tests: improve a comment.

    kennytm committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    0ada276 View commit details
    Browse the repository at this point in the history