开发者学堂课程【大数据 Hive 教程精讲:Apache Hive--DML--select 查询】学习笔记,与课程紧密联系,让用户快速学习知识。
课程地址:https://developer.aliyun.com/learning/course/90/detail/1378
Apache Hive--DML--select查询
Select:
基本的 Select 操作
语法结构
SELECT[ALL | DISTINCT] select_expr, select_expr,...FROM table_reference
JOIN table_other ON expr[WHERE where_condition]
[GROUP BY col_list [HAVING condition]][CLUSTER BY col_list
[DISTRIBUTE BY col_list] [SORT BY| ORDER BY col_list]
]
[LIMIT number]
说明:
1、order by 会对输入做全局排序,因此只有一个 reducer,会导致当输入规模较大时,需要较长的计算时间。
2、sort by不是全局排序,其在数据进入 reduce r前完成排序。因此,如果用 sort by进行排序,并且设置 mapred.reduce.tasks>1,则 sort by 只保证每个 reducer 的输出有序,不保证全局有序。
3、distribute by(字段)根据指定字段将数据分到不同的 reducer,分发算法是hash散列。4、Cluster by(字段)除了具有 Distribute by的功能外,还会对该字段进行排序。
如果distribute和 sort的字段是同一个时,此时,cluster by = distribute by + sort by
分桶、排序等查询:
cluster by . sort by
、distribute by
select * from student cluster by (sno);
insert overwrite table student_buck
select * from student cluster by(Sno) sort by(Sage);
报错,cluster 和 sort
不能共存
对某列进行分桶的同时,根据另一列进行排序
insert overwrite table stu_buck
select * from student distribute by (Sno) sort by (Sage asc) ;
总结:
cluster(分且排序,必须一样) ==distribute(分)+ sort(排序)(可以不一样)
LOAD DAPA local INPATH '/root/hivedata/aaa.txt ' INTO TABLEsource_table;insert overwrite table stu_buck
select * from student cluster by (sno) ;
create table student(Sno int ,Sname string,Sex string ,sage int,sdept string)row format delimited
fields terminated by ', ';
LOAD DATA local INPATH '/root/hivedata/students.txt ' INTO TABLE student;
create external table student_ext (8no int ,Sname string ,sex string,Sage int,sdept string) row format delimited field:
insert overwrite local directory '/root/aaa666'
select *from source_table;
insert overwrite local directory '/root/aaa777'select *from student cluster by (Sno);