flink本地模式简单入门 - Go语言中文社区

flink本地模式简单入门


一 本地模式

      flink 可以运行在 linux,mac os  和 windows上,flink需要工作在 java8以上得版本。window的用户,请参照 flink on windows的文档操作

 

    下面是 我 虚拟机 的环境 及 JAVA 环境变量

[root@localhost bin]# cat /etc/redhat-release 
CentOS Linux release 7.5.1804 (Core) 
[root@localhost bin]# java -version
java version "1.8.0_201"
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)

 

1  首先 我们 要重 flink 官网下载 tar包

https://flink.apache.org/downloads.html

我下载 的是flink-1.7.2-bin-hadoop26-scala_2.11 

 

2 然后 上传 到 /usr/local/src目录

cd /usr/local/src

tar -zxvf flink-1.7.2-bin-hadoop26-scala_2.11.tgz

mv flink-1.7.2  /usr/local/

cd /usr/local/flink-1.7.2/bin

./start-cluster.sh

 

启动 flink 后,flink web应用监听的端口是 8081

 

我们 用浏览器访问一下 

 

 

如果 有 错误 ,在 log文件夹上,我们 可以 查看 日志,排查 错误

[root@localhost log]# tail -f -n  flink-root-standalonesession-0-localhost.localdomain.log 
tail: flink-root-standalonesession-0-localhost.localdomain.log: invalid number of lines
[root@localhost log]# tail -f  flink-root-standalonesession-0-localhost.localdomain.log 
2019-04-08 17:54:46,244 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Rest endpoint listening at localhost:8081
2019-04-08 17:54:46,244 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - http://localhost:8081 was granted leadership with leaderSessionID=00000000-0000-0000-0000-000000000000
2019-04-08 17:54:46,244 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Web frontend listening at http://localhost:8081.
2019-04-08 17:54:46,427 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.resourcemanager.StandaloneResourceManager at akka://flink/user/resourcemanager .
2019-04-08 17:54:46,485 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher at akka://flink/user/dispatcher .
2019-04-08 17:54:46,577 INFO  org.apache.flink.runtime.resourcemanager.StandaloneResourceManager  - ResourceManager akka.tcp://flink@localhost:6123/user/resourcemanager was granted leadership with fencing token 00000000000000000000000000000000
2019-04-08 17:54:46,578 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager  - Starting the SlotManager.
2019-04-08 17:54:46,597 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Dispatcher akka.tcp://flink@localhost:6123/user/dispatcher was granted leadership with fencing token 00000000-0000-0000-0000-000000000000
2019-04-08 17:54:46,716 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Recovering all persisted jobs.
2019-04-08 17:54:47,959 INFO  org.apache.flink.runtime.resourcemanager.StandaloneResourceManager  - Registering TaskManager with ResourceID 8be94165507755fd86fe88e2b9a84920 (akka.tcp://flink@localhost.localdomain:44278/user/taskmanager_0) at ResourceManager

 

二  执行 flink 程序

下载 flink 模板 

 mvn archetype:generate  -DarchetypeGroupId=org.apache.flink  -DarchetypeArtifactId=flink-quickstart-java  -  DarchetypeVersion=1.7.2

 

第一种方式  flink 支持 在 ide上直接运行 调试

用 idea打开代码后 

我们直接 把 flink example 的WordCount 复制到 项目中

https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/wordcount/WordCount.java

直接运行 就可以

 

 

第二种方式 通过 打包 成 jar,在服务端上运行

我们 首先 要修改 pom文件上的 main方法 ,修改成 刚刚的workcount

		<transformers>
			<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
					<mainClass>cn.shendu.WordCount</mainClass>
			</transformer>
		</transformers>

然后打包上传到 服务器 上,然后 执行 

[root@localhost bin]# ./flink run /usr/local/src/flinkdemo-1.0-SNAPSHOT.jar --output  /usr/local/out.txt
Starting execution of program
Executing WordCount example with default input data set.
Use --input to specify file input.
Program execution finished
Job with JobID c49710513738493a29390ed7f591a25b has finished.
Job Runtime: 837 ms
[root@localhost bin]# cat /usr/local/out.txt 
(to,1)
(be,1)
(or,1)
(not,1)
(to,2)
(be,2)
(that,1)
(is,1)
(the,1)
(question,1)
(whether,1)

 

参考 资料 

https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/projectsetup/java_api_quickstart.html

https://ci.apache.org/projects/flink/flink-docs-release-1.7/tutorials/local_setup.html

版权声明:本文来源CSDN,感谢博主原创文章,遵循 CC 4.0 by-sa 版权协议,转载请附上原文出处链接和本声明。
原文链接:https://blog.csdn.net/weixin_39639119/article/details/89109430
站方申明:本站部分内容来自社区用户分享,若涉及侵权,请联系站方删除。
  • 发表于 2019-08-27 13:25:51
  • 阅读 ( 4069 )
  • 分类:

0 条评论

请先 登录 后评论

官方社群

GO教程

猜你喜欢