cdh的sentry

2024-01-22 11:33:52

CDH平台中的安全，认证（Kerberos/LDAP）是第一步，授权（Sentry）是第二步。如果要启用授权，必须先启用认证。但在CDH平台中给出了一种测试模式，即不启用认证而只启用Sentry授权。但强烈不建议在生产系统中这样使用，因为如果没有用户认证，授权没有任何意义形同虚设，用户可以随意使用任何超级用户登录HiveServer2或者Impala，并不会做密码校验。注：本文档仅适用于测试环境。

本文档主要描述如何在CDH未启用认证的情况下安装，配置及使用Sentry。

内容概述

1.如何安装Sentry服务

2.Hive/Impala/Hue/HDFS服务如何与Sentry集成

3.Sentry测试

测试环境

1.操作系统为CentOS6.5

2.CM和CDH版本为5.11.1

3.采用root用户操作

前置条件

1.CDH集群运行正常

2.集群未启用认证服务(如Kerberos或LDAP)

2.Sentry安装

1.在MySQL中创建sentry数据库

建表语句：

create database sentry default character set utf8;
CREATE USER 'sentry'@'%' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON sentry. * TO 'sentry'@'%';
FLUSH PRIVILEGES;

命令行操作：

[root@ip-172-31-6-148 527-hive-HIVEMETASTORE]# mysql -uroot -p
Enter password: 
...
mysql> create database sentry default character set utf8;
Query OK, 1 row affected (0.00 sec)
mysql> CREATE USER 'sentry'@'%' IDENTIFIED BY 'password';
Query OK, 0 rows affected (0.00 sec)
mysql> GRANT ALL PRIVILEGES ON sentry.* TO 'sentry'@'%';
Query OK, 0 rows affected (0.00 sec)
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)
mysql>

cdh的sentry

 cdh的sentry

2.进入Cloudera Manager控制台点击“添加服务”

cdh的sentry

3.进入服务添加界面

cdh的sentry

4.选择Sentry服务，点击“继续”

cdh的sentry

5.选择Sentry Server及Gateway的安装节点，点击“继续“，注意需提前在MySQL中建立好相关用户和数据库

cdh的sentry

6.输入Sentry服务的数据库信息，点击测试，测试通过，点击“继续”

cdh的sentry

7.等待服务安装成功，点击“继续”

cdh的sentry

 

 

8.点击“完成”，Sentry服务到此安装完成。

 

3.Sentry配置

3.1Hive配置

1.配置Hive使用Sentry服务

cdh的sentry

2.关闭Hive的用户模拟功能

cdh的sentry

3.集群未启用安全认证环境下，需要配置以下参数

<property>
    <name>sentry.hive.testing.mode</name>
    <value>true</value>
</property>

 

cdh的sentry

 

 

3.2Impala配置

配置Impala与Sentry集成

cdh的sentry

3.3Hue配置

配置Hue与Sentry集成

cdh的sentry

3.4HDFS配置

配置HDFS开启ACLs与Sentry权限同步

 

cdh的sentry

 

 

 

完成以上配置后，回到Cloudera Manager主页，部署客户端配置并重启相关服务。

4.Sentry测试

4.1创建hive超级用户

1.使用beeline连接HiveServer2，并登录hive用户

[root@ip-172-31-6-148 ~]# beeline 
Beeline version 1.1.0-cdh5.12.0 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: hive
Enter password for jdbc:hive2://localhost:10000: 
Connected to: Apache Hive (version 1.1.0-cdh5.12.0)
Driver: Hive JDBC (version 1.1.0-cdh5.12.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000>

cdh的sentry

注意：标红部分为输入的hive用户，输入的hive用户并未真正的校验

 

2.创建一个admin角色

0: jdbc:hive2://localhost:10000> create role admin;
...
INFO  : OK
No rows affected (2.52 seconds)
0: jdbc:hive2://localhost:10000>

cdh的sentry

 

3.为admin角色赋予超级权限

0: jdbc:hive2://localhost:10000> grant all on server server1 to role admin;
...
INFO : OK
No rows affected (0.221 seconds)
0: jdbc:hive2://localhost:10000>

 cdh的sentry

 

 

 

4.2创建test表

使用beeline登录hive用户，创建一个test表，并插入测试数据

0: jdbc:hive2://localhost:10000> create table test (s1 string, s2 string) 
row format delimitedfields terminated by ',';
...
INFO : OK
No rows affected (0.592 seconds)
0: jdbc:hive2://localhost:10000> insert into test values('a','b'),('1','2');
...
INFO : OK
No rows affected (20.123 seconds)
0: jdbc:hive2://localhost:10000>

cdh的sentry

 

 

 

4.3创建测试角色并授权给用户组

创建两个角色：

read：只能读default库test表，并授权给fayson用户组

write：只能写default库test表，并授权给user_w用户组

注意：集群所有节点必须存在fayson和user_w用户，用户默认用户组与用户名一致，赋权是针对用户组而不是针对用户。[root@ip-172-31-6-148 cdh-shell-master]# id fayson

[root@ip-172-31-6-148 cdh-shell-master]# useradd fayson
[root@ip-172-31-6-148 cdh-shell-master]# id fayson
uid=501(fayson) gid=501(fayson) groups=501(fayson)
[root@ip-172-31-6-148 cdh-shell-master]# useradd user_w
[root@ip-172-31-6-148 cdh-shell-master]# id user_w
uid=502(user_w) gid=502(user_w) groups=502(user_w)
[root@ip-172-31-6-148 cdh-shell-master]#

cdh的sentry

 

 

 

1.使用hive用户创建创建read和write角色，并授权read角色对test表select权限，write角色对test表insert权限

0: jdbc:hive2://localhost:10000> create role read;
...
INFO : OK
No rows affected (0.094 seconds)
0: jdbc:hive2://localhost:10000> grant select on table test to role read;
INFO : OK
No rows affected (0.1 seconds)
0: jdbc:hive2://localhost:10000> create role write;
...
INFO : OK
No rows affected (0.105 seconds)
0: jdbc:hive2://localhost:10000> grant insert on table test to role write;
...
INFO : OK
No rows affected (0.112 seconds)
0: jdbc:hive2://localhost:10000>

cdh的sentry

 

 

 

2.为fayson用户组授权read角色，为user_w用户组授权write角色

0: jdbc:hive2://localhost:10000> grant role read to group fayson;
…
INFO  : OK
No rows affected (0.187 seconds)
0: jdbc:hive2://localhost:10000> grant role write to group user_w;
…
INFO  : OK
No rows affected (0.101 seconds)
0: jdbc:hive2://localhost:10000>

cdh的sentry

 

 

 

4.4beeline验证

1.使用fayson用户登录beeline进行验证

 

[root@ip-172-31-6-148 ~]# beeline 
Beeline version 1.1.0-cdh5.12.0 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000
scan complete in 2ms
Connecting to jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: fayson
Enter password for jdbc:hive2://localhost:10000: 
Connected to: Apache Hive (version 1.1.0-cdh5.12.0)
Driver: Hive JDBC (version 1.1.0-cdh5.12.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> show tables;
...
INFO  : OK
+-----------+--+
| tab_name  |
+-----------+--+
| test      |
+-----------+--+
1 row selected (0.351 seconds)
0: jdbc:hive2://localhost:10000> select * from test;
...
INFO  : OK
+----------+----------+--+
| test.s1  | test.s2  |
+----------+----------+--+
| a        | b        |
| 1        | 2        |
+----------+----------+--+
2 rows selected (0.24 seconds)
0: jdbc:hive2://localhost:10000>
0: jdbc:hive2://localhost:10000> insert into test values("2", "222");
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 User fayson does not have privileges for QUERY
 The required privileges: Server=server1->Db=default->Table=test->action=insert; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000>

cdh的sentry

 

 

 

2.使用user_w用户登录beeline验证

 

[root@ip-172-31-6-148 ~]# beeline 
Beeline version 1.1.0-cdh5.12.0 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000
scan complete in 2ms
Connecting to jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: user_w
Enter password for jdbc:hive2://localhost:10000: 
Connected to: Apache Hive (version 1.1.0-cdh5.12.0)
Driver: Hive JDBC (version 1.1.0-cdh5.12.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> show tables;
INFO  : OK
+-----------+--+
| tab_name  |
+-----------+--+
| test      |
+-----------+--+
1 row selected (0.365 seconds)
0: jdbc:hive2://localhost:10000> select  * from test;
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 User user_w does not have privileges for QUERY
 The required privileges: Server=server1->Db=default->Table=test->Column=s1->action=select; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000> insert into test values("2", "333");
...
INFO  : Completed executing command(queryId=hive_20170902183535_56bcd189-544a-453f-9752-e40a9fed60c5); Time taken: 17.762 seconds
INFO  : OK
No rows affected (18.035 seconds)
0: jdbc:hive2://localhost:10000>

cdh的sentry

 cdh的sentry

验证总结:

fayson用户所属组为fayson拥有test表读权限，所以只能对test表进行selecth和count操作不能进行insert操作；

user_w用户所属组为user_w拥有test表写权限，所以只能对test表进行insert操作不能进行select和count操作；

 

4.5HDFS验证

1.切换至fayson用户下，浏览和查看/user/hive/warehouse/test数据目录及文件

[root@ip-172-31-6-148 ~]# su fayson
[fayson@ip-172-31-6-148 root]$ cd /home/fayson/
[fayson@ip-172-31-6-148 ~]$ ll
total 4
-rw-rw-r-- 1 fayson fayson 19 Sep  5 12:55 test.txt
[fayson@ip-172-31-6-148 ~]$ hadoop fs -ls /user/hive/warehouse
ls: Permission denied: user=fayson, access=READ_EXECUTE, inode="/user/hive/warehouse":hive:hive:drwxrwx--x
[fayson@ip-172-31-6-148 ~]$ hadoop fs -ls /user/hive/warehouse/test
Found 1 items
-rwxrwx--x+  3 hive hive          8 2017-09-05 12:52 /user/hive/warehouse/test/000000_0
[fayson@ip-172-31-6-148 ~]$ hadoop fs -cat /user/hive/warehouse/test/000000_0
a,b
1,2
[fayson@ip-172-31-6-148 ~]$ hadoop fs -put test.txt /user/hive/warehouse/test
put: Permission denied: user=fayson, access=WRITE, inode="/user/hive/warehouse/test":hive:hive:drwxrwx--x
[fayson@ip-172-31-6-148 ~]$

cdh的sentry

 

 

 

2.切换user_w用户下，浏览和查看/user/hive/warehouse/test数据目录及文件

 

[root@ip-172-31-6-148 ~]# su user_w
[user_w@ip-172-31-6-148 root]$ cd /home/user_w/
[user_w@ip-172-31-6-148 ~]$ cat test.txt 
333,5555
eeee,dddd
[user_w@ip-172-31-6-148 ~]$ hadoop fs -ls /user/hive/warehouse
ls: Permission denied: user=user_w, access=READ_EXECUTE, inode="/user/hive/warehouse":hive:hive:drwxrwx--x
[user_w@ip-172-31-6-148 ~]$ hadoop fs -ls /user/hive/warehouse/test
ls: Permission denied: user=user_w, access=READ_EXECUTE, inode="/user/hive/warehouse/test":hive:hive:drwxrwx--x
[user_w@ip-172-31-6-148 ~]$ hadoop fs -put test.txt /user/hive/warehouse/test
[user_w@ip-172-31-6-148 ~]$

cdh的sentry

测试总结：

fayson用户所属用户组为fayson，该组只拥有对test表的读权限，因此fayson用户不能对/user/hive/warehouse目录下除test以外的其它目录进行查看（包含父目录），并且不能向test目录put文件，只能浏览和查看test目录下的文件。

user_w用户所属用户组为user_w，该组只拥有对test表的写权限，因此user_w用户不能对/user/hive/warehouse目录下的任何目录进行查看（包含父目录），并且只拥有向test目录put文件的权限。说明Sentry实现了HDFS的ACL同步。

 

4.6Hue验证

1.使用admin用户登录Hue，分别创建fayson和user_w用户

 cdh的sentry

2.使用fayson用户登录Hue

 cdh的sentry

可以查看test表

cdh的sentry

可以对test表进行count操作

 cdh的sentry

只有SELECT权限不能向test表插入数据

 cdh的sentry

 

FileBrower验证

cdh的sentry

 

不可以浏览/user/hive/warehouse目录

cdh的sentry

 

可以浏览有SELECT权限的/user/hive/warehouse/test数据目录

 cdh的sentry

可以查看/user/hive/warehouse/test目录下的所有数据文件，但不能修改

 cdh的sentry

3.使用user_w用户登录Hue

cdh的sentry

因为无SELECT权限，所以不能查询表信息

cdh的sentry 

不能对test表进行count操作

 cdh的sentry

 

可以向test表中插入数据

 cdh的sentry

 

FileBrowser

 cdh的sentry

 

不可以浏览test表数据目录的父目录/user/hive/warehouse

 cdh的sentry

无SELECT权限也不能浏test表的数据目录/user/hive/warehouse/test

 cdh的sentry

测试总结：

fayson和user_w用户均能通过hue界面看到test表，拥有read角色的fayson用户组能对test表进行select和count操作，并且能通过File Browser浏览和查看test表的数据目录/user/hive/warehouse/test。拥有write角色的user_w用户组只能对test表进行insert操作，但不能通过File Browser浏览和查看test表的数据目录/user/hive/warehouse/test。说明Sentry在命令行的操作和授权在Hue中依旧有效。

4.7Impala验证

1.使用fayson用户测试

登录集群任意节点命令行下切换到fayson用户下

[root@ip-172-31-6-148 ~]# su fayson
[fayson@ip-172-31-6-148 root]$

cdh的sentry

在命令行执行impala-shell命令

[Not connected] > connect ip-172-31-10-118.fayson.com:21000;
...
[ip-172-31-10-118.fayson.com:21000] > show tables;
Query: show tables
+------+
| name |
+------+
| test |
+------+
Fetched 1 row(s) in 0.05s
[ip-172-31-10-118.fayson.com:21000] > select * from test;
...
+----+----------+
| s1 | s2       |
+----+----------+
| 1  | tttttttt |
+----+----------+
Fetched 1 row(s) in 5.32s
[ip-172-31-10-118.fayson.com:21000] > select count(*) from test;
...
+----------+
| count(*) |
+----------+
| 1        |
+----------+
Fetched 1 row(s) in 0.14s
[ip-172-31-10-118.fayson.com:21000] > insert into test values('2', 'test2');
Query: insert into test values('2', 'test2')
Query submitted at: 2017-09-11 01:37:56 (Coordinator: http://ip-172-31-10-118.fayson.com:25000)
ERROR: AuthorizationException: User 'fayson' does not have privileges to execute 'INSERT' on: default.test
[ip-172-31-10-118.fayson.com:21000] >

cdh的sentry

2.使用user_w用户测试

登录集群任意节点命令行下切换到user_w用户下

[root@ip-172-31-6-148 ~]# su user_w
[user_w@ip-172-31-6-148 root]$ impala-shell

在命令行执行命令impala-shell，进行如下操作

[user_w@ip-172-31-6-148 root]$ impala-shell
...
[Not connected] > connect ip-172-31-10-118.fayson.com:21000;
...
Query: show tables
+------+
| name |
+------+
| test |
+------+
Fetched 1 row(s) in 0.06s
[ip-172-31-10-118.fayson.com:21000] > select * from test;
Query: select * from test
Query submitted at: 2017-09-11 01:41:17 (Coordinator: http://ip-172-31-10-118.fayson.com:25000)
ERROR: AuthorizationException: User 'user_w' does not have privileges to execute 'SELECT' on: default.test

[ip-172-31-10-118.fayson.com:21000] > select count(*) from test;
Query: select count(*) from test
Query submitted at: 2017-09-11 01:41:23 (Coordinator: http://ip-172-31-10-118.fayson.com:25000)
ERROR: AuthorizationException: User 'user_w' does not have privileges to execute 'SELECT' on: default.test

[ip-172-31-10-118.fayson.com:21000] > insert into test values('2', 'impala insert');
Query: insert into test values('2', 'impala insert')
Query submitted at: 2017-09-11 01:41:48 (Coordinator: http://ip-172-31-10-118.fayson.com:25000)
Query progress can be monitored at: http://ip-172-31-10-118.fayson.com:25000/query_plan?query_id=bd4a433465037682:77a7c3c400000000
Modified 1 row(s) in 0.71s

验证总结：

Impala与Sentry集成后可以使用Sentry来进行权限管理，拥有read角色的fayson用户组只能对test表进行select和count操作不能插入数据，拥有write角色的user_w

用户组只能对test表插入数据不能进行select和count操作。说明Sentry实现了Hive权限与Impala的同步。

5.Sentry列权限管理验证

1.在集群所有节点新增fayson_r用户

cdh的sentry

2.使用beeline登录hive用户

使用hive用户创建columnread角色，并为角色授权test表s1列的读权限，将columnread角色授权给fayson_r用户组。

[root@ip-172-31-6-148 cdh-shell-bak]# beeline 
Beeline version 1.1.0-cdh5.12.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: hive
Enter password for jdbc:hive2://localhost:10000: 
Connected to: Apache Hive (version 1.1.0-cdh5.12.1)
Driver: Hive JDBC (version 1.1.0-cdh5.12.1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> create role columnread;
...
INFO  : OK
No rows affected (0.225 seconds)
0: jdbc:hive2://localhost:10000> grant select(s1) on table test to role columnread;
...
INFO  : OK
No rows affected (0.095 seconds)
0: jdbc:hive2://localhost:10000> grant role columnread to group fayson_r;
...
INFO  : OK
No rows affected (0.091 seconds)
0: jdbc:hive2://localhost:10000>

cdh的sentry

3.使用beeline登录fayson_r用户测试

[root@ip-172-31-6-148 cdh-shell-bak]# beeline 
Beeline version 1.1.0-cdh5.12.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000
scan complete in 2ms
Connecting to jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: fayson_r
Enter password for jdbc:hive2://localhost:10000: 
Connected to: Apache Hive (version 1.1.0-cdh5.12.1)
Driver: Hive JDBC (version 1.1.0-cdh5.12.1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> show tables;
...
INFO  : OK
+-----------+--+
| tab_name  |
+-----------+--+
| test      |
+-----------+--+
1 row selected (0.304 seconds)
0: jdbc:hive2://localhost:10000> select * from test;
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 User fayson_r does not have privileges for QUERY
 The required privileges: Server=server1->Db=default->Table=test->Column=s2->action=select; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000> select s1 from test;
...
INFO  : OK
+-------+--+
|  s1   |
+-------+--+
| a     |
| 1     |
| 111   |
| 333   |
| eeee  |
+-------+--+
5 rows selected (0.197 seconds)
0: jdbc:hive2://localhost:10000> select s2 from test;
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 User fayson_r does not have privileges for QUERY
 The required privileges: Server=server1->Db=default->Table=test->Column=s2->action=select; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000> select count(*) from test;
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 User fayson_r does not have privileges for QUERY
 The required privileges: Server=server1->Db=default->Table=test->action=select; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000> select count(s1) from test;
...
INFO  : OK
+------+--+
| _c0  |
+------+--+
| 5    |
+------+--+
1 row selected (23.855 seconds)
0: jdbc:hive2://localhost:10000>

cdh的sentry

4.浏览HDFS目录

[fayson_r@ip-172-31-6-148 ~]$ hadoop fs -ls /user/hive/warehouse
ls: Permission denied: user=fayson_r, access=READ_EXECUTE, inode="/user/hive/warehouse":hive:hive:drwxrwx--x
[fayson_r@ip-172-31-6-148 ~]$ hadoop fs -ls /user/hive/warehouse/test
ls: Permission denied: user=fayson_r, access=READ_EXECUTE, inode="/user/hive/warehouse/test":hive:hive:drwxrwx--x
[fayson_r@ip-172-31-6-148 ~]$

cdh的sentry

5.使用admin用户登录Hue，创建fayson_r用户，然后使用fayson_r用户登录

 cdh的sentry

cdh的sentry

 

 cdh的sentry

 

 cdh的sentry

 

 cdh的sentry

测试总结：

fayson_r用户所属用户组为fayson_r，该组只拥有对test表s1列的读权限，因此在select和count的时候只能对s1列进行select和count，fayson_r用户无权限浏览/user/hive/warehouse下的所有目录；使用hue只能对test表s1列进行select和count操作，无权限浏览/user/hive/warehouse目录及目录下所有子目录。

注意：Sentry只支持SELECT的列授权，不能用于INSERT和ALL的列授权。

6.备注

在使用beeline进行授权验证时，只是输入了username未做用户信息校验。在未启用认证服务的集群下，该文档的Sentry配置方式只适用于测试环境，不能用于生产环境。

在集群启用Sentry服务后，由于Sentry不支持Hive CLI权限管理，所以建议禁用Hive CLI。但在非安全环境下，不能通过hadoop.proxyuser.hive.groups来限制访问用户组。

 

7.Hive授权参考

7.1 角色创建和删除

create role test;
drop role test;

7.2 角色授权和取消授权

grant select on table test_table to role role_name;
revoke select on table test_table to role role_name;

表授权给角色

grant select on table test_table to role role_name;
revoke select on table test_table to role role_name;

列授权给角色

grant select(column1,column2) on table test_table to role role_name;
revoke select(column1,column2) on table test_table to role role_name;

7.3 组授权和取消授权

grant role role_name to group user_group;
revoke role role_name to group user_group;

注意：Sentry适用于用户组授权，不适用与用户授权；

8.常见问题

不能创建角色，异常如下

0: jdbc:hive2://localhost:10000> create role admin;
Error: Error whilecompiling statement: FAILED:InvalidConfigurationException hive.server2.authentication can't be none innon-testing mode (state=42000,code=40000)
0: jdbc:hive2://localhost:10000>

原因：由于集群未启用Kerberos，需要配置sentry.hive.testing.mode为true

解决方法：参考3.1的第3步配置。

 

上一篇：VSCode必备插件


下一篇：sentry使用docker-compose部署