织梦网站模板/网站优化推广价格
Spring Batch(7): 并行与扩展
2016-03-26 23:54 846人阅读 评论(0) 收藏 举报
分类：版权声明：本文为博主原创文章，未经博主允许不得转载。
目录(?)[+]
1. 概述
spring Batch提供了多种方式用于处理并行，提高性能。主要分为2大类：
- 单个进程，多线程
- 多个进程
因此，可以细分为以下几类：
- 多线程Step（Multi-thread Step,single process)
- 并行Step（Parallel Steps, single process )
- Remote Chunking of Step( multi process)
- Partitioning a step(single or multi process)
2. Multi-Thread Step
最直接的方式是给Step配置一个TaskExecutor
<code class="hljs r has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;"><step id=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"loading"</span>><tasklet task-executor=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"taskExecutor"</span>><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span></tasklet> </step></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>
此时，taskExecutor的线程并行来执行Item处理（统一item的read，process，write在同一个线程中执行）。可以限制TaskExecutor的阈值（默认为4）：
<code class="hljs r has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;"><step id=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"loading"</span>> <tasklettask-executor=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"taskExecutor"</span>throttle-limit=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"20"</span>><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span></tasklet> </step></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>
需要注意的是，在多线程Step中，需要确保Reader、Processor和Writer是线程安全的，否则容易出现并发问题。Spring Batch提供的大部分组件都是非线程安全的，他们都保存有部分状态信息，主要是为了支持任务重启。
因此，使用多线程Step的核心任务是实现无状态化，例如不保存当前读取的item的cursor，而是同item的flag字段来区分item是否被处理过，已经被处理过的下次重启的时候，直接被过滤掉。
多线程Step实现的是单个Step的多线程化。
3. Parallel Steps
如果多个Step没有先后关系，可以并行执行，这是通过split和flow来实现的：
<code class="hljs xml has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;"><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">job</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">id</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"job1"</span>></span><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">split</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">id</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"split1"</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">task-executor</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"taskExecutor"</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">next</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"step4"</span>></span><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">flow</span>></span><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">step</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">id</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"step1"</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">parent</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"s1"</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">next</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"step2"</span>/></span><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">step</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">id</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"step2"</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">parent</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"s2"</span>/></span><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">flow</span>></span><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">flow</span>></span><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">step</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">id</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"step3"</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">parent</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"s3"</span>/></span><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">flow</span>></span><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">split</span>></span><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">step</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">id</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"step4"</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">parent</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"s4"</span>/></span> <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">job</span>></span><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">beans:bean</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">id</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"taskExecutor"</span> <span class="hljs-attribute" style="box-sizing: border-box; color: rgb(102, 0, 102);">class</span>=<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 136, 0);">"org.spr...SimpleAsyncTaskExecutor"</span>/></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li></ul>
该种模式提供的是多个Step的并行处理。
4. Remote Chunking
Remote chunking 的示意图如下：
Master为单个进程，因此只有在处理所需要的时间远远大于读取所需要的时间的时候，这个方式才适用，否则Master容易成为瓶颈。
Master是一个常规的Step实现，只不过它的ItemWriter知道如何将Items分块，并发送到中间件（例如JMS），通过实现ChunkProvider接口来实现。
<code class="language-java hljs has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-class" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">interface</span> <span class="hljs-title" style="box-sizing: border-box; color: rgb(102, 0, 102);">ChunkProvider</span><<span class="hljs-title" style="box-sizing: border-box; color: rgb(102, 0, 102);">T</span>>{</span>Chunk<T> provide(StepContribution contribution) <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">throws</span> Exception;<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> postProcess(StepContribution contribution, Chunk<T> chunk);
}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul> Slave则充当中间件的Listener，通过ItemProcessor和ItemWriter来实现item处理，具体的是通过实现ChunkProcessor接口
<code class="language-java hljs has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-class" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">interface</span> <span class="hljs-title" style="box-sizing: border-box; color: rgb(102, 0, 102);">ChunkProcessr</span><<span class="hljs-title" style="box-sizing: border-box; color: rgb(102, 0, 102);">T</span>> {</span><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> process(StepContribution contribution,Chunk<T> chunk) <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">throws</span> Exception;
}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul> 可以看到，remote chunking实现的是（Processor、Writer）的并行化。分区不需要对数据源的结构有很明确的了解。
5. Partitioning
Step分区处理示意图如下:
一个分区配置如下：
<code class="hljs r has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;"><step id=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"step1.master"</span>><partition step=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"step1"</span> partitioner=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"partitioner"</span>><handler grid-size=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"10"</span> task-executor=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"taskExecutor"</span>/></partition> </step><step id=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"step1"</span>><tasklet><chunk reader=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">""</span> writer<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">""</span> processor=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">""</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span>/></tasklet> </step></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul>
主要包括2个步骤：
1. 数据分区
2. 分区处理
具体的分区执行流程如下：
PartitionHandler
其中PartitionHandler知道集群环境，根据下面要介绍的Splitter进行分区，发送执行请求（通过WebService ，RMI等方式) 并收集执行结果，聚合，最终反馈给Job。Spring Batch提供了一个同一台机器上的Handler实现，在同一机器上创建多个Step Execution。
<code class="hljs applescript has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;"><step <span class="hljs-property" style="box-sizing: border-box;">id</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"step1.master"</span>><partition step=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"step1"</span> handler=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"handler"</span>/> </step><bean <span class="hljs-type" style="box-sizing: border-box;">class</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"org.spr...TaskExecutorPartitionHandler"</span>><<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">property</span> <span class="hljs-property" style="box-sizing: border-box;">name</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"taskExecutor"</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">ref</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"taskExecutor"</span>/><<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">property</span> <span class="hljs-property" style="box-sizing: border-box;">name</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"step"</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">ref</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"step1"</span> /><<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">property</span> <span class="hljs-property" style="box-sizing: border-box;">name</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"gridSize"</span> value=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"10"</span> /> </bean></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li></ul>
Partitioner
Partitioner负责生成执行上下文，作为Step Execution的输入参数，其接口定义如下：
<code class="language-java hljs has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-class" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">interface</span> <span class="hljs-title" style="box-sizing: border-box; color: rgb(102, 0, 102);">Partitioner</span> {</span>Map<String, ExecutionContext> partition(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> gridSize);
}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul> 返回结果中Map的key，是一个唯一的名字，常见的实现方式是step_name + counter。或者通过PartitioneNameProvider来提供。 名字关联到对应的执行上下文。ExecutionContext只是一个key/value容器，因此它可能包含主键范围，行数等信息。
StepExecutionSplitter
Partitioner生成的ExecutionContext，经过StepExecutionSplitter处理之后形成StepExecution，然后交给Handler处理。StepExecutionSplitter接口定义如下：
<code class="language-java hljs has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-class" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">interface</span> <span class="hljs-title" style="box-sizing: border-box; color: rgb(102, 0, 102);">StepExecutionSplitter</span> {</span>String getStepName();Set<StepExecution> split(StepExecution stepExecution , <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> gridSize) <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">throws</span> JobExecutionException;
}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul> 通常，Slave中的Step配置都是相同的，他们通过获取Partitioner划分好的ExecutionContext，获取Step的输入参数，动态绑定到Step中。例如划分的情况如下表：
| step execution name(key) | ExecutionContext(value) |
|---|---|
| filecopy:partition0 | file_name=/home/data/0 |
| filecopy:partition1 | file_name=/home/data/1 |
| filecopy:partition2 | file_name=/home/data/2 |
然后该文件名被绑定到Step的组件中：
<code class="hljs applescript has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;"><bean <span class="hljs-property" style="box-sizing: border-box;">id</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"itemReader"</span> scope=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"step"</span><span class="hljs-type" style="box-sizing: border-box;">class</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"org.spr...MultiResourceItemReader"</span>><<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">property</span> <span class="hljs-property" style="box-sizing: border-box;">name</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"resource"</span> value=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"#{stepExecutionContext[file_name]}/*"</span>/>
</bean></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul> 整个具体流程如下：
可以看出，Patitioning提供的是（Reader、Processor、Writer）的并行化。分区模式需要对数据源的结构有一定的了解，比如知道主键范围。