4-31Data Sinks
发布日期:2021-07-01 02:23:20 浏览次数:2 分类:技术文章

本文共 1529 字,大约阅读时间需要 5 分钟。

Data sinks consume DataSets and are used to store or return them. Data sink operations are described using an . Flink comes with a variety of built-in output formats that are encapsulated behind operations on the DataSet:

  • writeAsText() / TextOutputFormat - Writes elements line-wise as Strings. The Strings are obtained by calling the toString() method of each element.
  • writeAsCsv(...) / CsvOutputFormat - Writes tuples as comma-separated value files. Row and field delimiters are configurable. The value for each field comes from the toString() method of the objects.
  • print() / printToErr() - Prints the toString() value of each element on the standard out / standard error stream.
  • write() / FileOutputFormat - Method and base class for custom file outputs. Supports custom object-to-bytes conversion.
  • output()OutputFormat - Most generic output method, for data sinks that are not file based (such as storing the result in a database).

A DataSet can be input to multiple operations. Programs can write or print a data set and at the same time run additional transformations on them. 

object DataSetSinkApp {  def main(args: Array[String]): Unit = {    val env = ExecutionEnvironment.getExecutionEnvironment    import org.apache.flink.api.scala._    val data = 1.to(10)    val text = env.fromCollection(data)    val filePath="file:///F://data/"    //此处设置并行度,如果并行度为1,那么输出的就是一个文件而不是一个文件夹。此处并行度为2,那么就会生成一个data文件夹,文件夹内有两个文件,1和2    text.writeAsText(filePath,WriteMode.OVERWRITE).setParallelism(2)    env.execute("DataSetSinkApp")  }}

 

转载地址:https://mapengsen.blog.csdn.net/article/details/108876023 如侵犯您的版权,请留言回复原文章的地址,我们会给您删除此文章,给您带来不便请您谅解!

上一篇:IDEA自动勾选显示类型(specify type)
下一篇:201403-1_相反数的个数

发表评论

最新留言

表示我来过!
[***.240.166.169]2024年04月29日 07时39分09秒

关于作者

    喝酒易醉,品茶养心,人生如梦,品茶悟道,何以解忧?唯有杜康!
-- 愿君每日到此一游!

推荐文章