Python 使用thrift连接hbase || 远程连接hbase - Go语言中文社区

Python 使用thrift连接hbase || 远程连接hbase


1.首先下载Python & thrift &hbase

有两种安装thrift的方式:1.下载的thrift-0.9.3.tar.gz >> 解压 tar xzvf thrift-0.9.3.tar.gz  >> ./configure >> make >> make install >>thrift -version 验证
    2.解压之后进入lib/py/   >> 执行Python setup.py install 安装 。这种方式不需要软连接到python的site-packages/ >> python >> import thrift 验证。


2.解压hbase:

tar xzvf hbase-1.1.2.tar.gz >> cd hbase-thrift  >> thrift -gen py /root/zhutong/hbase-1.1.2/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift 会生成一个 gen-py的文件 >> cd gen-py >> cp -R hbase/usr/local/python2.7/lib/python2.7/site-packages/ 
 



3.启动thrift:

单机模式启动:首先启动hbase >>cd bin  >>  ./start-hbase.sh >> 启动thrift  ./hbase-daemon.sh start thrift  # 默认的端口是9090   
配置hbase的rootdir 
<property>
<name>hbase.rootdir</name>
<value>/root/zhutong/hbase_data</value>
</property>

环境配置好了 现在准备测试:
编写test.py 文件

#!/usr/bin/python
#-*- coding:utf-8 -*-
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from thrift.protocol import TCompactProtocol
from hbase import Hbase
from hbase.ttypes import *
#192.168.110.78:8020
transport1 = TSocket.TSocket('localhost',9090 )
transport = TTransport.TBufferedTransport(transport1)
protocol = TCompactProtocol.TCompactProtocol(transport);
client = Hbase.Client(protocol)
transport.open()
contents=ColumnDescriptor(name='cf:',maxVersions=1)
tablename='ta'
client.createTable(tablename,[contents])
print client.getTableNames()

异常:如果报错找不到thrift模块 ,查看Python安装目录site-packages下没有thrift*.egg 文件 ,使用easy-install 或者第一步中第二种方式安装。
            如果提示主机名错误异常;请前去/etc/hosts文件中添加映射关系。

成功之后再对hbase进行操作;
#!/usr/bin/python
#insert data
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol

from hbase import Hbase

from hbase.ttypes import *

transport = TSocket.TSocket('localhost', 9090)

transport = TTransport.TBufferedTransport(transport)

protocol = TBinaryProtocol.TBinaryProtocol(transport)

client = Hbase.Client(protocol)

transport.open()
for i in range(10):
        row = 'row-key1'+str(i)

        mutations = [Mutation(column="cf:a", value=str(i))]
        client.mutateRow('ta', row, mutations, None)

#!/usr/bin/python

from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol

from hbase import Hbase
from hbase.ttypes import *

transport = TSocket.TSocket('localhost', 9090)
transport = TTransport.TBufferedTransport(transport)

protocol = TBinaryProtocol.TBinaryProtocol(transport)

client = Hbase.Client(protocol)

transport.open()

tableName = 'ta'
rowKey = 'row-key1'

result = client.getRow(tableName, rowKey, None)
print result
print type(result)
for r in result:
    print 'the row is ' , r.row
    print 'the values is ' , r.columns.get('cf:a').value

#!/usr/bin/python
# get some data 
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol

from hbase import Hbase
from hbase.ttypes import *

transport = TSocket.TSocket('localhost', 9090)
transport = TTransport.TBufferedTransport(transport)

protocol = TBinaryProtocol.TBinaryProtocol(transport)

client = Hbase.Client(protocol)
transport.open()

scan = TScan()
tableName = 'ta'
id = client.scannerOpenWithScan(tableName, scan, None)

result2 = client.scannerGetList(id, 10)

print result2
print type(result2)

for i in result2:
        print i.row


4.远程连接hbase

 修改Python文件的IP地址和端口进行连接会出现如下的错误:



或者



这是因为远程访问的hbase也是需要thrift来进行访问的,所以需要在访问的hbase服务器上启动thrift:
nohup hbase thrift -p 9999 start  &  启动 >> ps aux | grep thrift 验证
然后再执行 : 端口号为你设置的端口号;

 

版权声明:本文来源CSDN,感谢博主原创文章,遵循 CC 4.0 by-sa 版权协议,转载请附上原文出处链接和本声明。
原文链接:https://blog.csdn.net/zt_fucker/article/details/49099771
站方申明:本站部分内容来自社区用户分享,若涉及侵权,请联系站方删除。
  • 发表于 2020-03-01 23:14:05
  • 阅读 ( 1084 )
  • 分类:

0 条评论

请先 登录 后评论

官方社群

GO教程

猜你喜欢