hive 的mapjoin起作用了我们如何确定?
- 只是inner join 的时候
看日志吧,最为直观:
2021-12-10 12:05:41 Starting to launch local task to process map join; maximum memory = 954728448
2021-12-10 12:05:44 Processing rows: 200000 Hashtable size: 199999 Memory usage: 135058920 percentage: 0.141
2021-12-10 12:05:44 Dump the side-table into file: file:/tmp/hive_2021-12-10_11-47-34_913_2061727660300134431-1/-local-10007/HashTable-Stage-13/MapJoin-mapfile10--.hashtable
2021-12-10 12:05:44 Uploaded 1 File to: file:/tmp/hive_2021-12-10_11-47-34_913_2061727660300134431-1/-local-10007/HashTable-Stage-13/MapJoin-mapfile10--.hashtable (3517 bytes)
2021-12-10 12:05:44 Dump the side-table into file: file:/tmp/hive_2021-12-10_11-47-34_913_2061727660300134431-1/-local-10007/HashTable-Stage-13/MapJoin-mapfile12--.hashtable
2021-12-10 12:05:44 Uploaded 1 File to: file:/tmp/hive_2021-12-10_11-47-34_913_2061727660300134431-1/-local-10007/HashTable-Stage-13/MapJoin-mapfile12--.hashtable (8683158 bytes)
2021-12-10 12:05:44 End of local task; Time Taken: 3.034 sec.
Execution completed successfully
关键点:
- Starting to launch local task to process map join; 这个说的够直白了吧
- Uploaded 1 File to: file:/tmp/hive_2021-12-10_11-47-34_913_2061727660300134431-1/-local-10007/HashTable-Stage-13/MapJoin-mapfile10--.hashtable hashtable
- end of local task
- 起一个local task 映射成一个hashtable