Introduction to Parallel Programming

并行程序设计导论

Parallel programming should be used any time you have a fair amount of computation work that can be split up into independent chunks. Parallel programming increases the CPU usage temporarily to improve throughput; this is desirable on client systems where CPUs are often idle, but it’s usually not appropriate for server systems. Most servers have some parallelism built in; for example, ASP.NET will handle multiple requests in parallel. Writing parallel code on the server may still be useful in some situations (if you know that the number of concurrent users will always be low), but in general, parallel programming on the server would work against its built-in parallelism and therefore wouldn’t provide any real benefit.

当您有相当数量的计算工作，并且可以将其划分为独立的块时，就应该使用并行编程。并行编程暂时增加CPU的使用以提高吞吐量;这在cpu经常空闲的客户机系统上是可取的，但通常不适合服务器系统。大多数服务器都有一些并行性;例如,ASP.NET将并行处理多个请求。在某些情况下，在服务器上编写并行代码可能仍然有用(如果您知道并发用户的数量总是很低的话)，但通常情况下，在服务器上进行并行编程将违背其内置的并行性，因此不会提供任何实际的好处。

There are two forms of parallelism: data parallelism and task parallelism. Data parallelism is when you have a bunch of data items to process, and the processing of each piece of data is mostly independent from the other pieces. Task parallelism is when you have a pool of work to do, and each piece of work is mostly independent from the other pieces. Task parallelism may be dynamic; if one piece of work results in several additional pieces of work, they can be added to the pool of work.

并行有两种形式:数据并行和任务并行。数据并行是指有一堆数据项要处理，并且每个数据块的处理基本上独立于其他数据块。任务并行性是指您有一个工作池要做，并且每一项工作在很大程度上独立于其他部分。任务并行性可以是动态的;如果一项工作导致了几项额外的工作，它们可以添加到工作池中。

There are a few different ways to do data parallelism. Parallel.ForEach is similar to a foreach loop and should be used when possible. Parallel.ForEach is covered in Recipe 4.1. The Parallel class also supports Parallel.For, which is similar to a for loop, and can be used if the data processing depends on the index. Code that uses Parallel.ForEach looks like the following:

有几种不同的方法来实现数据并行。Parallel.ForEach类似于ForEach循环，应该在可能的情况下使用。Parallel.ForEach在本书4.1中有介绍。Parallel类也支持Parallel.For，类似于For循环，可以在数据处理依赖于索引的情况下使用。使用Parallel.ForEach看起来像下面这样:

void RotateMatrices(IEnumerable<Matrix> matrices, float degrees)
{
    Parallel.ForEach(matrices, matrix => matrix.Rotate(degrees));
}

Another option is PLINQ (Parallel LINQ), which provides an AsParallel extension method for LINQ queries. Parallel is more resource friendly than PLINQ; Parallel will play more nicely with other processes in the system,while PLINQ will (by default) attempt to spread itself over all CPUs. The downside to Parallel is that it’s more explicit; PLINQ in many cases has more elegant code. PLINQ is covered in Recipe 4.5 and looks like this:

另一个选项是PLINQ (Parallel LINQ)，它为LINQ查询提供了一个AsParallel扩展方法。Parallel比PLINQ更加资源友好;Parallel与系统中的其他进程一起运行会更好，而PLINQ(默认情况下)会尝试将自己扩展到所有cpu上。并行的缺点是它更明确;PLINQ在很多情况下都有更优雅的代码。PLINQ包含在配方4.5中，看起来像这样:

IEnumerable<bool> PrimalityTest(IEnumerable<int> values)
{
    return values.AsParallel().Select(value => IsPrime(value));
}

Regardless of the method you choose, one guideline stands out when doing parallel processing.

无论您选择哪种方法，在进行并行处理时，有一条准则是突出的。

TIP:The chunks of work should be as independent from one another as possible.

工作块应该尽可能彼此独立。

As long as your chunk of work is independent from all other chunks, you maximize your parallelism. As soon as you start sharing state between multiple threads, you have to synchronize access to that shared state, and your application becomes less parallel. Chapter 12 covers synchronization in more detail.

只要您的工作块独立于所有其他工作块，您就可以最大化并行性。一旦开始在多个线程之间共享状态，就必须同步对该共享状态的访问，您的应用程序的并行性就会降低。第12章更详细地介绍同步。

The output of your parallel processing can be handled in various ways. You can place the results in some kind of a concurrent collection, or you can aggregate the results into a summary. Aggregation is common in parallel processing; this kind of map/reduce functionality is also supported by the Parallel class method overloads. Recipe 4.2 looks at aggregation in more detail.

可以通过多种方式处理并行处理的输出。您可以将结果放在某种类型的并发集合中，也可以将结果聚合为一个摘要。聚合在并行处理中很常见;并行类方法重载也支持这种映射/减少功能。章节4.2更详细地介绍了聚合。

Now let’s turn to task parallelism. Data parallelism is focused on processing data; task parallelism is just about doing work. At a high level, data parallelism and task parallelism are similar; “processing data” is a kind of “work.” Many parallelism problems can be solved either way; it’s convenient to use whichever API is more natural for the problem at hand. Parallel.Invoke is one type of Parallel method that does a kind of fork/join task parallelism. This method is covered in Recipe 4.3; you just pass in the delegates you want to execute in parallel:

现在让我们转向任务并行性。数据并行主要集中在数据处理上;任务并行就是做工作。在高层上，数据并行和任务并行是相似的;“处理数据”是一种“工作”。“许多并行问题都可以用任何一种方法解决;对于手头的问题，使用任何更自然的API都很方便。Parallel.Invoke是一种并行方法，它执行一种fork/join任务并行性。这个方法在章节4.3中有介绍;你只需传入你想并行执行的委托:

void ProcessArray(double[] array)
{
    Parallel.Invoke(
        () => ProcessPartialArray(array, 0, array.Length / 2),
        () => ProcessPartialArray(array, array.Length / 2, array.Length)
    );
}

void ProcessPartialArray(double[] array, int begin, int end)
{
    // CPU-intensive processing...  cpu密集型操作
}

The Task type was originally introduced for task parallelism, though these days it’s also used for asynchronous programming. A Task instance—as used in task parallelism—represents some work. You can use the Wait method to wait for a task to complete, and you can use the Result and Exception properties to retrieve the results of that work.

任务类型最初是为了任务并行性而引入的，不过最近它也被用于异步编程。任务实例(在任务并行中使用)表示某些工作。您可以使用Wait方法来等待任务完成，并且可以使用Result和Exception属性来检索该工作的结果。

Code using Task directly is more complex than code using Parallel, but it can be useful if you don’t know the structure of the parallelism until runtime. With this kind of dynamic parallelism, you don’t know how many pieces of work you need to do at the beginning of the processing; you find out as you go along. Generally, a dynamic piece of work should start whatever child tasks it needs and then wait for them to complete.

直接使用Task的代码比使用Parallel的代码更复杂，但如果您直到运行时才知道并行性的结构，那么它可能会很有用。使用这种动态并行，你不知道在处理开始时需要做多少工作;你一边做一边看。通常，一个动态的工作应该启动它所需要的任何子任务，然后等待它们完成。

The Task type has a special flag, TaskCreationOptions.AttachedToParent,which you could use for this. Dynamic parallelism is covered in Recipe 4.4.

任务类型有一个特殊的标志，TaskCreationOptions.AttachedToParent你可以用它来做这个。章节4.4中介绍了动态并行性。

Task parallelism should strive to be independent, just like data parallelism. The more independent your delegates can be, the more efficient your program can be. Also, if your delegates aren’t independent, then they need to be synchronized, and it’s harder to write correct code if that code needs synchronization. With task parallelism, be especially careful of variables captured in closures. Remember that closures capture references (not values), so you can end up with sharing that isn’t obvious.

任务并行应该努力做到独立，就像数据并行一样。你的代表越独立，你的程序就越有效。另外，如果你的委托不是独立的，那么它们就需要同步，如果代码需要同步，那么就很难编写正确的代码。对于任务并行性，要特别注意闭包中捕获的变量。请记住，闭包捕获的是引用(而不是值)，因此您最终可能会得到不明显的共享。

Error handling is similar for all kinds of parallelism. Because operations are proceeding in parallel, it’s possible for multiple exceptions to occur, so they are wrapped up in an AggregateException that’s thrown to your code. This behavior is consistent across Parallel.ForEach, Parallel.Invoke, Task.Wait, etc. The AggregateException type has some useful Flatten and Handle methods to simplify the error handling code：

错误处理类似于所有类型的并行。由于操作是并行进行的，可能会发生多个异常，因此它们被封装在一个抛出到代码的AggregateException中。这种行为在并行中的Parallel.ForEach，Parallel.Invoke, Task.Wait, 等等是一致的。AggregateException类型有一些有用的Flatten和Handle方法来简化错误处理代码:

try
{
    Parallel.Invoke(() => { throw new Exception(); },
        () => { throw new Exception(); });
    }
catch (AggregateException ex)
{
    ex.Handle(exception =>
    {
        Trace.WriteLine(exception);
        return true; // "handled"
    });
}

Usually, you don’t have to worry about how the work is handled by the thread pool. Data and task parallelism use dynamically adjusting partitioners to divide work among worker threads. The thread pool increases its thread count as necessary. The thread pool has a single work queue, and each threadpool thread also has its own work queue. When a threadpool thread queues additional work, it sends it to its own queue first because the work is usually related to the current work item; this behavior encourages threads to work on their own work, and maximizes cache hits. If another thread doesn’t have work to do, it’ll steal work from another thread’s queue. Microsoft put a lot of work into making the thread pool as efficient as possible, and there are a large number of knobs you can tweak if you need maximum performance. As long as your tasks are not extremely short, they should work well with the default settings.

通常，您不必担心线程池如何处理工作。数据和任务并行性使用动态调整分区程序来在工作线程之间划分工作。线程池会根据需要增加线程数。线程池有一个工作队列，每个线程池线程也有自己的工作队列。当threadpool的线程将额外的工作排队时，它首先将它发送到自己的队列中，因为这些工作通常与当前的工作项相关;这种行为鼓励线程进行自己的工作，并最大化缓存命中。如果另一个线程没有工作要做，它就会从另一个线程的队列中窃取工作。微软为使线程池尽可能高效投入了大量的工作，如果您需要最大的性能，您可以调整大量的旋钮。只要你的任务不是特别短，它们应该可以很好地使用默认设置。

TIP:Tasks should neither be extremely short, nor extremely long.

任务既不应该太短，也不应该太长。

If your tasks are too short, then the overhead of breaking up the data into tasks and scheduling those tasks on the thread pool becomes significant. If your tasks are too long, then the thread pool cannot dynamically adjust its work balancing efficiently. It’s difficult to determine how short is too short and how long is too long; it really depends on the problem being solved and the approximate capabilities of the hardware. As a general rule, I try to make my tasks as short as possible without running into performance issues (you’ll see your performance suddenly degrade when your tasks are too short). Even better, instead of using tasks directly, use the Parallel type or PLINQ. These higher level forms of parallelism have partitioning built in to handle this automatically for you (and adjust as necessary at runtime).

如果任务太短，那么将数据分解为任务并在线程池中调度这些任务的开销就会变得非常大。如果任务太长，那么线程池就不能有效地动态调整其工作平衡。很难确定多长时间太短，多长时间太长;这实际上取决于要解决的问题和硬件的大致能力。一般来说，我尽量让我的任务越短越好，而不会遇到性能问题(当任务太短时，性能会突然下降)。更好的是，不要直接使用任务，而是使用并行类型或PLINQ。这些更高级别的并行形式内置了分区，可以为您自动处理这一点(并在运行时根据需要进行调整)。

If you want to dive deeper into parallel programming, the best book on the subject is Parallel Programming with Microsoft .NET, by Colin Campbell et al.(Microsoft Press).

如果你想深入研究并行编程，关于这个主题最好的书是Colin Campbell等人(微软出版社)所著的《与Microsoft . net并行编程》。