batch applications for the java platform
DESCRIPTION
Presentation on the Batch JSR (JSR-352) in JavaOne India, Hyderabad 2013. Thanks to http://www.slideshare.net/reza_rahman and http://www.slideshare.net/arungupta1 for the source slides.TRANSCRIPT
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.1
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.2
Batch Applications for the Java PlatformSivakumar ThyagarajanOracle [email protected]
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.3
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.4
Batch Applications for the Java Platform
Standardizes batch processing for Java– Non-interactive, bulk-oriented, long-running
– Data pr computationally intensive
– Sequentially or in parallel
– Ad-hoc, scheduled or on-demand execution
Led by IBM Spring Batch, WebSphere Compute Grid (WCG), z/OS Batch Part of Java EE 7, can be used in Java SE
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.5
JBatch Domain Language
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.6
JBatch Domain Language
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.7
JBatch Domain Language
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.8
Reader, Processor, Writerpublic interface ItemReader<T> { public void open(Externalizable checkpoint); public T readItem(); public Externalizable checkpointInfo(); public void close();}
public interface ItemProcessor<T, R> { public R processItem(T item);}
public interface ItemWriter<T> { public void open(Externalizable checkpoint); public void writeItems(List<T> items); public Externalizable checkpointInfo(); public void close();}
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.9
//For a job<step id=”sendStatements”> <chunk reader=”accountReader” processor=”accountProcessor” writer=”emailWriter” item-count=”10” /></step>
Batch Applications for the Java Platform Step Example using Job Specification Language (JSL)
@Named(“accountReader")...implements ItemReader... {public Account readItem() { // read account using JPA
@Named(“accountProcessor")...implements ItemProcessor... {Public Statement processItems(Account account) { // read Account, return Statement
@Named(“emailWriter")...implements ItemWriter... {public void writeItems(List<Statements> statements) { // use JavaMail to send email
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.10
Batch Chunks
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.11
Checkpointing
For data intensive tasks, long periods of time– Checkpoint/restart is a common design requirement
Basically saves Reader, Writer positions– Naturally fits into Chunk oriented steps
– reader.checkpointInfo() and writer.checkpointInfo() are called
– The resulting Externalizable data is persisted
– When the Chunk restarts, the reader and writer are initialized with the respective Externalizable data
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.12
Reader, Processor, Writerpublic interface ItemReader<T> { public void open(Externalizable checkpoint); public T readItem(); public Externalizable checkpointInfo(); public void close();}
public interface ItemProcessor<T, R> { public R processItem(T item);}
public interface ItemWriter<T> { public void open(Externalizable checkpoint); public void writeItems(List<T> items); public Externalizable checkpointInfo(); public void close();}
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.13
Handling Exceptions
<job id=...>... <chunk skip-limit=”5” retry-limit=”5”> <skippable-exception-classes> <include class="java.lang.Exception"/> <exclude class="java.io.FileNotFoundException"/> </skippable-exception-classes> <retryable-exception-classes> </retryable-exception-classes> <no-rollback-exception-classes> </no-rollback-exception-classes> </chunk>...</job>
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.14
Partitioned Step
A batch step may run as a partitioned step– A partitioned step runs as multiple instances of the same step definition
across multiple threads, one partition per thread
<step id="step1" > <chunk ...> <partition> <plan partitions=“10" threads="2"/> <reducer .../> </partition> </chunk></step>
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.15
Partitioning – Advanced Scenarios
Partition Mapper– Dynamically decide number of partitions (partition plan)
Partition Reducer
– Demarcate logical unit of work around partition processing Partition Collector
– Sends interim results from individual partition to step's partition analyzer
Partition Analyzer
– Collection point of interim results, single point of control and analysis
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.16
Flow and Split
Flow defines a set of steps to be executed as a unit<flow id=”flow-1" next=“{flow, step, decision}-id” > <step id=“flow_1_step_1”> </step> <step id=“flow_1_step_2”> </step></flow>
Split defines a set of flows to be executed in parallel
<split …> <flow …./> <!– each flow runs on a separate thread --> <flow …./></split>
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.17
Batchlet
Task oriented unit of work Doesn't support resume Not item-oriented Simple Batchlet interface – process(), stop() In job xml:<batchlet ref=”{name}”/>
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.18
Programming Model – Advanced Scenarios
CheckpointAlgorithm Decider Listeners – Job, Step, Chunk Listeners … @BatchProperty
String fname = “/tmp/default.txt” @BatchContext
JobContext jctxt;
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.19
Batch Runtime Specification
JobOperator interface – for programmatic lifecycle control of jobs Step level Metrics through the StepExecution object Batch Artifact loading through META-INF/batch.xml Job XML loading through META-INF/batch-jobs/my-job.xml Packaging – jar, war, ejb-jar
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.20
Example: Programmatic Invocation of Jobs
In a Servlet/EJB:
import javax.batch.runtime.BatchRuntime;...
JobOperator jo = BatchRuntime.getJobOperator();jo.start("myJob", new Properties());
...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.21
Example: META-INF/batch-jobs/my-job.xml
<job id="myJob" xmlns="http://batch.jsr352/jsl"> <step id="myStep"> <chunk item-count="3"> <reader ref="myItemReader"/> <processor ref="myItemProcessor"/> <writer ref="myItemWriter"/> </chunk> </step></job>
If in a WAR, this would be WEB-INF/classes/META-INF/batch-jobs/my-job.xml
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.22
Example: META-INF/batch.xml
<batch-artifacts xmlns="http://jcp.org.batch/jsl"> <ref id="myItemReader" class="org.acme.MyItemReader"/> <ref id="myItemProcessor" class="org.acme.MyItemProcessor"/> <ref id="myItemWriter" class="org.acme.MyItemWriter"/></batch-artifacts>
If in a WAR, this would be WEB-INF/classes/META-INF/batch.xml
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.23
Where to Find Out More
Read the final spec and API docs at http://java.net/projects/jbatch/ Join [email protected] and send questions and comments RI integrated in GlassFish 4.0
– http://glassfish.java.net/
– http://dlc.sun.com.edgesuite.net/glassfish/4.0/promoted/
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.24
Graphic Section Divider
Copyright © 2012, Oracle and/or its affiliates. All rights reserved.25