jsr-352, spring batch and you

9/13/13 4:33 PMJSR-352, Spring Batch, and You

of 94http://mminella.github.io/jsr352-springbatch-and-you/?print-pdf#/

JSR-352, SPRING BATCH,JSR-352, SPRING BATCH,AND YOUAND YOU

Created by / Michael T Minella @michaelminella

http://springframework.io/team/mminella

http://twitter.com/michaelminella



WHO IS THIS GUY?WHO IS THIS GUY?

Author of Pro Spring BatchMember of the expert group for JSR-352Project lead for Spring Batch



I DO A LOT OF BATCH ;)I DO A LOT OF BATCH ;)



NO NEED TO TAKE NOTES!NO NEED TO TAKE NOTES!These talks are being recorded...These slides and code are on Github

https://github.com/mminella/jsr352-springbatch-and-you.git



FIND IT ALL HEREFIND IT ALL HERE



??



33



JSR-352JSR-352



SPRING BATCH'S IMPLE‐SPRING BATCH'S IMPLE‐MENTATION OF JSR-352MENTATION OF JSR-352



SPRING BATCH BEYONDSPRING BATCH BEYONDJSR-352JSR-352



EXPECTATIONSEXPECTATIONSBasic knowledge of SpringBasic knowledge of Spring Batch



? WHEN YOU HAVE THEM? WHEN YOU HAVE THEM



JSR-352JSR-352



JSR-352 introduces an exciting new Javaspecification for building, deploying, and

running batch applications.- Chris Vignola

http://jaxenter.com/java-ee-7-introduction-to-batch-jsr-352-47400.html



SPRING'S INVOLVEMENTSPRING'S INVOLVEMENT



SPRING BATCH IS ASPRING BATCH IS ALEADER IN THE SPACELEADER IN THE SPACE



RESULTS OF THE SPEC?RESULTS OF THE SPEC?



STRUCTURALLY THESTRUCTURALLY THESAMESAME



33API for building batch componentsA DSL for configuring batch componentsA contract of how batch components interact



HIGH LEVEL DIFFERENCESHIGH LEVEL DIFFERENCES



SPRING BATCH JOBSPRING BATCH JOB<job id="personJob"> <step id="step1"> <tasklet> <chunk commit-interval="5" reader="itemReader" processor="itemProcessor" writer="itemWriter" /> </tasklet> </step></job>



JSR-352 JOBJSR-352 JOB<job id="personJob"> <step id="step1"> <chunk item-count="5"> <reader ref="itemReader"/> <processor ref="itemProcessor"/> <writer ref="itemWriter"/> </chunk> </step></job>

<job id="personJob"> <step id="step1"> <tasklet> <chunk commit-interval="5" reader="itemReader" processor="itemProcessor" writer="itemWriter"/> </tasklet> </step></job>

<job id="personJob"> <step id="step1"> <chunk item-count="5"> <reader ref="itemReader"/> <processor ref="itemProcessor"/> <writer ref="itemWriter"/> </chunk> </step></job>



??



DEPENDENCY INJECTIONDEPENDENCY INJECTIONFoundational technology of Spring BatchNot required with JSR-352



REFERENCES IN JSR-352REFERENCES IN JSR-352<job id="personJob"> <step id="step1"> <chunk item-count="5"> <reader ref="itemReader"/> <processor ref="itemProcessor"/> <writer ref="itemWriter"/> </chunk> </step></job>



REFERENCES POINT TO 1REFERENCES POINT TO 1OF 3 PLACESOF 3 PLACES

Spring beanbatch.xml definitionThe class itself



SPRING BEANSPRING BEAN<bean id="itemWriter" class="org.springframework.batch.item.xml.StaxEventItemWriter" <property name="resource" ref="outputFile" /> <property name="marshaller" ref="customerMarshaller" /> <property name="rootTagName" value="customers" /></bean>



BATCH.XMLBATCH.XML<batch-artifacts xmlns="http://xmlns.jcp.org/xml/ns/javaee">

<ref id="itemReader" class="org.sf.jsr.demo.ItemReader" /> <ref id="itemProcessor" class="org.sf.jsr.demo.ItemProcessor" /> <ref id="itemWriter" class="org.sf.jsr.demo.ItemWriter" />

</batch-artifacts>



THE CLASS ITSELFTHE CLASS ITSELF<job id="personJob"> <step id="step1"> <chunk item-count="5"> <reader ref="org.sf.jsr.demo.ItemReader"/> <processor ref="org.sf.jsr.demo.ItemProcessor"/> <writer ref="org.sf.jsr.demo.ItemWriter"/> </chunk> </step></job>



PROPERTIESPROPERTIESSpring injects them on bean definitionsJSR-352 injects properties from the job definitionJSR-352 properties are Strings only



SPRING BATCH JOBSPRING BATCH JOB<job id="formatJob"> <step id="step1"> <tasklet> <chunk reader="fileReader" writer="xmlWriter" commit-interval="#{jobParameters['commit.interval']}"/> </tasklet> </step></job>



JSR-352 JOBJSR-352 JOB<job id="personJob"> <step id="step1"> <chunk item-count="#{jobParameters['commit.interval']}"> <reader ref="fileReader"> <properties> <property name="fileName" value="#{jobParameters['input.fileName']}" </properties> </reader> <writer ref="xmlWriter"> <properties> <property name="fileName" value="#{jobParameters['output.fileName']}" </properties> </writer> </chunk> </step></job>



INJECTION OF PROPER‐INJECTION OF PROPER‐TIESTIES

@Inject@BatchPropertypublic String fileName;

@Inject@BatchProperty(name="otherProperty")public String otherProperty;



??



LET'S BUILD AN APP!LET'S BUILD AN APP!



00:00 -00:34



LOGS OF A VIRAL VIDEOLOGS OF A VIRAL VIDEOReleased in 2003.Downloaded 1.1 million times the first 2 weeksSince been downloaded 40 million times since it wasmoved to YouTube2008, Andy Baio posted the Apache logs for the first 6monthsFile is 1.6 GB unzipped and available via BitTorentFile was originally called "ghyslain_razaa.wmv"File was later renamed to "Star_Wars_Kid.wmv"



WHO WAS WATCHINGWHO WAS WATCHINGTHIS VIDEO?THIS VIDEO?



OUR JOBOUR JOB



STEP PROCESSING MOD‐STEP PROCESSING MOD‐ELSELS



BATCHLETBATCHLET



BATCHLET = STOPPABLE‐BATCHLET = STOPPABLE‐TASKLETTASKLET



public interface Tasklet { RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception;}

public interface StoppableTasklet extends Tasklet { void stop();}

public interface Batchlet { public String process() throws Exception; public void stop() throws Exception; }



STEP 1...SPLIT THE FILESTEP 1...SPLIT THE FILE



LAUNCHINGLAUNCHING



JOBOPERATORJOBOPERATORJobOperator jobOperator = BatchRuntime.getJobOperator();

// startlong jobExecutionId = jobOperator.start("myJob", new Properties());

// restartlong newJobExecutionId = jobOperator.restart(jobExecutionId, new Properties());

// stopjobOperator.stop(jobExecutionid);



CHUNK PROCESSINGCHUNK PROCESSING



SPRING BATCHSPRING BATCH



SPRING BATCH REALLYSPRING BATCH REALLY



JSR-352JSR-352



CHUNK RELATED INTER‐CHUNK RELATED INTER‐FACESFACES

ItemReaderItemProcessorItemWriter



CHECKPOINTSCHECKPOINTSItemCustom



STEP 2...IMPORT THE FILESTEP 2...IMPORT THE FILE



ERROR HANDLINGERROR HANDLING

SKIPSKIP<job id="skip" xmlns="http://xmlns.jcp.org/xml/ns/javaee" version="1.0"> <step id="step1"> <chunk skip-limit="3"> <reader ref="retryableListReader"/> <writer ref="fileWriter"/>  <skippable-exception-classes> <include class="java.lang.RuntimeException"/> </skippable-exception-classes> </chunk> </step></job>

RETRYRETRY<job id="retry" xmlns="http://xmlns.jcp.org/xml/ns/javaee" version="1.0"> <step id="step1"> <chunk retry-limit="3"> <reader ref="retryableListReader"/> <writer ref="fileWriter"/>

 <retryable-exception-classes> <include class="java.lang.RuntimeException"/> </retryable-exception-classes> </chunk> </step></job>



READ RETRIESREAD RETRIESSpring Batch hasn't supported read retriesJSR-352 allows read retriesExisting Spring Batch readers won't support read retry with3.0



SCALINGSCALING



SPLITSSPLITSSame as Spring Batch



PARTITIONINGPARTITIONINGExecutes chunks or multiple instances of Batchlets inparallelTwo configuration options

Hardcoded in XMLPartitionMapper

Adds two additional concepts...



PARTITION REDUCERPARTITION REDUCERProvides a hook for compensating logic upon failureReally just a specializedStepListener/StepExecutionListenr



PARTITION COLLECTOR/PARTITION COLLECTOR/ANALYZERANALYZER

Collector collects data from batch artifactsAnalyzer allows the ability to set ExitStatus after apartitioned step



SPRING BATCH'SSPRING BATCH'SIMPLEMENTATIONIMPLEMENTATION



SPRING BATCH STATUSSPRING BATCH STATUSReleased 3.0.0.M2 last nightPasses > 50% of the TCK tests3.0 will be JSR-352 compliant



"RAW" JSR-352 VERSION"RAW" JSR-352 VERSIONClasses: 8Lines of custom code: 826



33API for building batch componentsA DSL for configuring batch componentsA contract of how batch components interact



??



COMPONENTS!!!COMPONENTS!!!



JSR-352 PROVIDES 0JSR-352 PROVIDES 0IMPLEMENTATIONSIMPLEMENTATIONS



ITEM READERSITEM READERSAggregateItemReader

AmqpItemReader

FlatFileItemReader

HibernateCursorItemReader

HibernatePagingItemReader

IbatisPagingItemReader

ItemReaderAdapter

JdbcCursorItemReader

JdbcPagingItemReader

JmsItemReader

JpaPagingItemReader

ListItemReader

MongoItemReader

Neo4jItemReader

RepositoryItemReader

StoredProcedureItemReader

StaxEventItemReader

ITEM WRITERSITEM WRITERS



SAME APP...SPRING-IFIEDSAME APP...SPRING-IFIED



COMPARISONCOMPARISON



"RAW" JSR-352 VERSION"RAW" JSR-352 VERSIONClasses: 8Lines of custom code: 826



SPRING JSR-352 VERSIONSPRING JSR-352 VERSIONClasses: 5Lines of custom code: 596



BEYOND JSR-352BEYOND JSR-352



SPRING!SPRING!



TYPE SAFE PROPERTIESTYPE SAFE PROPERTIES



NO INHERITENCENO INHERITENCE



SCALABILITYSCALABILITY



JSR-352 IS SINGLE JVMJSR-352 IS SINGLE JVMONLYONLY



SPRING BATCH SCALESSPRING BATCH SCALESBEYOND 1 JVMBEYOND 1 JVM



REMOTE CHUNKINGREMOTE CHUNKING



REMOTE PARTITIONINGREMOTE PARTITIONING



BATCH DOESN'T OCCURBATCH DOESN'T OCCURIN A VACUUMIN A VACUUM



LAUNCHING JOBS VIALAUNCHING JOBS VIAMESSAGESMESSAGES



SPRING HADOOPSPRING HADOOP



SPRING XDSPRING XD



??



FIND THIS DECK ANDFIND THIS DECK ANDDEMO CODE:DEMO CODE:

Github

https://github.com/mminella/jsr352-springbatch-and-you.git



KEEP IN TOUCH!KEEP IN TOUCH!Michael Minella

Twitter: Github:

Spring BatchSite: Session replays on YouTube:

SpringSite:

@MichaelMinellamminella

Spring BatchYouTube

Spring IO

https://twitter.com/michaelminella

https://github.com/mminella

http://spring.io/projects/spring-batch

http://spring.io/video

http://spring.io/

jsr-352, spring batch and you

Technology

configuring

building batch

spring batch

spring batch

throws exception

352 version

custom code

33 pm spring