* A factory for constructing encoders
that convert objects and primitives to and from the internal row format using catalyst expressions and code generation
Flink 的 TypeInformation 类
org.apache.flink.api.common.typeinfo.Types
org.apache.flink.api.common.typeinfo.TypeInformation;
org.apache.flink.api.common.typeinfo.TypeHint
org.apache.flink.api.common.typeutils.TypeSerializer
创建 TypeInformation 或者 TypeSerializer
TypeInformation<String> info = TypeInformation.of(String.class);
This class gives access to the type information of the most common types for which Flink
* has built-in serializers and comparators.
对于泛型类型,你需要通过 TypeHint 来“捕获”泛型类型信息:
TypeInformation<Tuple2<String, Double>> info = TypeInformation.of(new TypeHint<Tuple2<String, Double>>(){});
DataSet<SomeType> result = dataSet
.map(new MyGenericNonInferrableFunction<Long, SomeType>())
.returns(SomeType.class); // returns(new TypeHint<Tuple2<Integer, SomeType>>(){})
Flink的Types
org.apache.flink.table.api.Types
Flink 中DataSet和DataStream 都能与Table 互转
This class enumerates all supported types of the Table API & SQL
Flink中的DataType
1.Data Types in the Table API
org.apache.flink.table.types.DataType
重构类型系统:在Flink 1.9版本中实现了一套全新的数据类型系统,这套全新的类型系统与SQL标准进行了完全对齐,能够支持更加丰富的类型。这套
从 Table API 中移除对 Flink TypeInformation 的依赖,并提高其对 SQL 标准的遵从性
The planning for code generation and serialization of runtime operators
Flink Improvement Proposal (FLIP) process
org.apache.flink.table.types.logical.LogicalType
org.apache.flink.table.types.logical
public abstract class LogicalType implements Serializable {
https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
DataType t = DataTypes.TIMESTAMP(3).bridgedTo(java.sql.Timestamp.class);
the difference between data type and type information
org.apache.flink.table.api.types.TypeConverters提供了DataType和Typeinformation之间的相互转化。
createExternalTypeInfoFromDataType和createInternalTypeInfoFromDataType
2.SQL standard’s data type and Java Expression
3.Row - org.apache.spark.sql.Row
RDD[Row]
JavaRDD<Row>
Dataset<Row>
List<Row> createDataFrame(rows: java.util.List[Row], schema: StructType)
List<Row> data = Arrays.asList(
RowFactory.create("L5", "a1群"),
RowFactory.create("L9", "a2群")
)
type DataFrame = Dataset[Row]
org.apache.flink.types.Row
Creates a new Row with projected fields from another row.
java.sql.ResultSet
参考
Spark SQL中的Encoder https://www.jianshu.com/p/d3c35e18af44
Data Types https://ci.apache.org/projects/flink/flink-docs-release-1.9/zh/dev/table/types.html
https://ci.apache.org/projects/flink/flink-docs-release-1.9/zh/dev/table/types.html#data-types-in-the-table-api