在MySQL处理ORDER BY语句时，如果查询无法利用索引的有序性，则需要额外操作对数据进行排序。
在MySQL中有三种排序算法：

1、快速排序(Quick Sort)，对冒泡排序的一种改进，基本思想是选取一个记录作为枢轴，经过一趟排序，将整段序列分为两个部分，其中一部分的值都小于枢轴，另一部分都大于枢轴。然后继续对这两部分继续进行排序，从而使整个序列达到有序。
2、快速排序(Quick Sort)，利用归并的思想实现的排序方法，该算法采用经典的分治（divide-and-conquer）策略（分治法将问题分(divide)成一些小的问题然后递归求解，而治(conquer)的阶段则将分的阶段得到的各答案"修补"在一起，即分而治之)。
3、堆排序是利用堆的数据结构实现的排序算法，通过调整二叉树上父节点和左右子节点的位置，最终得到一个有序序列.

 

在MySQL中有三种数据排序方式：

1、常规排序(双路排序)
2、优化排序(单路排序)
3、堆排序

堆排序在MySQL 5.6版本中引入，对于LIMIT M OFFSET N操作，仅需要排出M+N条记录，因此可以利用"大顶堆"或"小顶堆"两种堆排序算法来优化，避免对全部记录排序的资源消耗。

常规排序和优化排序都是采用分治(divide-and-conquer)策略将记录拆分为多个记录块，对每个记录块采用"快速排序"算法进行排序，再将排序后的记录块采用"归并排序"算法进行合并。
由于常规排序和优化排序都需要使用"临时文件"来存放中间排序结果，因此被称为文件排序(FileSort)。
当查询计划的extra部分显示"Using filesort"时表明该查询使用"常规排序"或"优化排序"，但不表示一定会造成IO操作，只有当"中间结果集"较大时才会将"中间结果集"写入到磁盘。

 

相关参数：

1、参数max_length_for_sort_data用于决定FileSort采用常规排序还是优化排序，该参数在MySQL 8.0.20版本中被弃用。
2、参数sort_buffer_size用于控制每次快速排序的buffer大小，默认值为256KB，增大该值有利于减少"归并排序"操作次数和"临时文件"写入磁盘次数。
    A) 在MySQL 5.7及之前版本中，会直接分配参数sort_buffer_size指定的内存，因此参数sort_buffer_size值不能设置过大。
    B) 在MySQL 8.0.12版本中对参数sort_buffer_size进行优化，会根据实际需要增量分配内存，因此参数sort_buffer_size值能设置相对较大些。
3、参数read_rnd_buffer_size用于控制常规排序中需要随机读缓存大小，默认值为256KB，增大该值有利于增大"合并随机读概率"和降低"随机读"IO次数。
4、参数tmp_dir用于控制临时文件的存储位置，将tmp_dir指向多块磁盘或将tmp_dir指向高速磁盘能有效降低IO响应时间，提升排序效率。

 

MySQL官方文档How MySQL Does Sorting (filesort)

MySQL has two filesort algorithms for sorting and retrieving results. The original method uses only the ORDER BY columns. The modified method uses not just the ORDER BY columns, but all the columns used in the query.

The optimizer selects which filesort algorithm to use. Prior to MySQL 4.1, it uses the original algorithm. As of MySQL 4.1, it normally uses the modified algorithm except when BLOB or TEXT columns are involved, in which case it uses the original algorithm.

The original filesort algorithm works as follows:

    Read all rows according to key or by table scanning. Rows that do not match the WHERE clause are skipped.
    For each row, store a pair of values in a buffer (the sort key and the row pointer). The size of the buffer is the value of the sort_buffer_size system variable.
    When the buffer gets full, run a qsort (quicksort) on it and store the result in a temporary file. Save a pointer to the sorted block. (If all pairs fit into the sort buffer, no temporary file is created.)
    Repeat the preceding steps until all rows have been read.
    Do a multi-merge of up to MERGEBUFF (7) regions to one block in another temporary file. Repeat until all blocks from the first file are in the second file.
    Repeat the following until there are fewer than MERGEBUFF2 (15) blocks left.
    On the last multi-merge, only the pointer to the row (the last part of the sort key) is written to a result file.
    Read the rows in sorted order by using the row pointers in the result file. To optimize this, we read in a big block of row pointers, sort them, and use them to read the rows in sorted order into a row buffer. The size of the buffer is the value of the read_rnd_buffer_size system variable. The code for this step is in the sql/records.cc source file.

One problem with this approach is that it reads rows twice: One time when evaluating the WHERE clause, and again after sorting the pair values. And even if the rows were accessed successively the first time (for example, if a table scan is done), the second time they are accessed randomly. (The sort keys are ordered, but the row positions are not.)

The modified filesort algorithm incorporates an optimization such that it records not only the sort key value and row position, but also the columns required for the query. This avoids reading the rows twice. The modified filesort algorithm works like this:

    Read the rows that match the WHERE clause.
    For each row, record a tuple of values consisting of the sort key value and row position, and also the columns required for the query.
    Sort the tuples by sort key value
    Retrieve the rows in sorted order, but read the required columns directly from the sorted tuples rather than by accessing the table a second time.

Using the modified filesort algorithm, the tuples are longer than the pairs used in the original method, and fewer of them fit in the sort buffer (the size of which is given by sort_buffer_size). As a result, it is possible for the extra I/O to make the modified approach slower, not faster. To avoid a slowdown, the optimization is used only if the total size of the extra columns in the sort tuple does not exceed the value of the max_length_for_sort_data system variable. (A symptom of setting the value of this variable too high is that you should see high disk activity and low CPU activity.)

 

参考资料：
https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_sort_buffer_size
https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_read_rnd_buffer_size
https://dev.mysql.com/doc/refman/8.0/en/order-by-optimization.html