Skip to content

Latest commit

 

History

History

flink-table

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Table API & SQL

Apache Flink features two relational APIs - the Table API and SQL - for unified stream and batch processing. The Table API is a language-integrated query API for Java, Scala, and Python that allows the composition of queries from relational operators such as selection, filter, and join in a very intuitive way.

This documentation is intended for contributors of the table modules, and not for users. If you want to use Table API & SQL, check out the documentation.

Modules

Common

  • flink-table-common:
    • Type system definition
    • UDF stack and built-in function definitions
    • Internal data definitions
    • Extension points for catalogs, formats, connectors
    • Core APIs for extension points such as Schema
    • Utilities to deal with type system, internal data types and printing

API

  • flink-table-api-java:
    • Java APIs for Table API and SQL
    • Package org.apache.flink.table.delegation, which serves as entrypoint for all planner capabilities
  • flink-table-api-scala: Scala APIs for Table API and SQL
  • flink-table-api-bridge-base: Base classes for APIs to bridge between Table API and DataStream API
  • flink-table-api-java-bridge:
    • Java APIs to bridge between Table API and DataStream API
    • Connectors that are developed using DataStream API, usually need to depend only on this module.
  • flink-table-api-scala-bridge: Scala APIs to bridge between Table API and DataStream API
  • flink-table-api-java-uber:
    • Uber JAR bundling flink-table-common and all the Java API modules, including the bridging to DataStream API and 3rd party dependencies.
    • This module is intended to be used by the flink-dist, rather than from the users directly.

Runtime

  • flink-table-code-splitter: Tool to split generated Java code so that each method does not exceed the limit of 64KB.
  • flink-table-runtime:
    • Operator implementations
    • Built-in functions implementations
    • Type system implementation, including readers/writers, converters and utilities
    • Raw format
    • The produced jar includes all the classes from this module and flink-table-code-splitter, including 3rd party dependencies

Parser and planner

  • flink-sql-parser: Default ANSI SQL parser implementation
  • flink-sql-parser-hive: Hive SQL dialect parser implementation
  • flink-table-planner:
    • AST and Semantic tree
    • SQL validator
    • Query planner, optimizer and rules implementation
    • Code generator
    • Two jars are produced: one doesn't have any classifier and bundles all the classes from this module together with the two parsers, including 3rd party dependencies, while the other jar, classified as loader-bundle, extends the first jar including scala dependencies.
  • flink-table-planner-loader: Loader for flink-table-planner that loads the planner in a separate classpath, isolating the Scala version used to compile the planner.

SQL client

  • flink-sql-client: CLI tool to submit queries to a Flink cluster

Testing

  • flink-table-test-utils: Brings in transitively all the dependencies you need to execute Table pipelines and provides some test utilities such as assertions, mocks and test harnesses.

Notes

No module except flink-table-planner should depend on flink-table-runtime in production classpath, and similarly no module should depend on flink-table-planner or flink-table-planner-loader in production classpath. For testing, you should depend on flink-table-planner-loader and flink-table-runtime. These are already shipped by the Flink distribution.