1. Overview
In this tutorial, we’re going to look at a practical, code-focused intro to Spring Batch. Spring Batch is a processing framework designed for robust execution of jobs.
It’s current version 5.0.0 supports Spring 6 and Java 17.
Here are a few interesting and practical use cases of the framework.
2. Workflow Basics
Spring Batch follows the traditional batch architecture where a job repository does the work of scheduling and interacting with the job.
A job can have more than one step. And every step typically follows the sequence of reading data, processing it and writing it.
And of course the framework will do most of the heavy lifting for us here — especially when it comes to the low-level persistence work of dealing with the jobs — using h2 for the job repository.
2.1. Example Use Case
The simple use case we’re going to tackle here is migrating some financial transaction data from CSV to XML.
The input file has a very simple structure.
It contains a transaction per line, made up of a username, the user id, the date of the transaction and the amount:
username, userid, transaction_date, transaction_amount
devendra, 1234, 31/10/2015, 10000
john, 2134, 3/12/2015, 12321
robin, 2134, 2/02/2015, 23411
3. The Maven POM
Dependencies required for this project are Spring Core, Spring Batch and H2 database:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-oxm</artifactId>
<version>6.0.6</version>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>2.1.214</version>
</dependency>
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-core</artifactId>
<version>5.0.0</version>
</dependency>
4. Spring Batch and Job Config
The basic Spring Batch configuration is displayed below along with our job description for the CSV to XML functionality.
Java-based job configuration:
@Profile("spring")
public class SpringBatchConfig {
@Value("input/record.csv")
private Resource inputCsv;
@Value("file:xml/output.xml")
private Resource outputXml;
@Bean
public ItemReader<Transaction> itemReader()
throws UnexpectedInputException, ParseException {
FlatFileItemReader<Transaction> reader = new FlatFileItemReader<Transaction>();
DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer();
String[] tokens = { "username", "userid", "transactiondate", "amount" };
tokenizer.setNames(tokens);
reader.setResource(inputCsv);
DefaultLineMapper<Transaction> lineMapper =
new DefaultLineMapper<Transaction>();
lineMapper.setLineTokenizer(tokenizer);
lineMapper.setFieldSetMapper(new RecordFieldSetMapper());
reader.setLineMapper(lineMapper);
return reader;
}
@Bean
public ItemProcessor<Transaction, Transaction> itemProcessor() {
return new CustomItemProcessor();
}
@Bean
public ItemWriter<Transaction> itemWriter(Marshaller marshaller)
throws MalformedURLException {
StaxEventItemWriter<Transaction> itemWriter =
new StaxEventItemWriter<Transaction>();
itemWriter.setMarshaller(marshaller);
itemWriter.setRootTagName("transactionRecord");
itemWriter.setResource(outputXml);
return itemWriter;
}
@Bean
public Marshaller marshaller() {
Jaxb2Marshaller marshaller = new Jaxb2Marshaller();
marshaller.setClassesToBeBound(new Class[] { Transaction.class });
return marshaller;
}
@Bean
protected Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager, ItemReader<Transaction> reader,
ItemProcessor<Transaction, Transaction> processor, ItemWriter<Transaction> writer, ) {
return new StepBuilder("step1", jobRepository).<Transaction, Transaction> chunk(10, transactionManager)
.reader(reader).processor(processor).writer(writer).build();
}
@Bean(name = "firstBatchJob")
public Job job(JobRepository jobRepository, @Qualifier("step1") Step step1) {
return new JobBuilder("firstBatchJob", jobRepository).preventRestart().start(step1).build();
}
public DataSource dataSource() {
EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder();
return builder.setType(EmbeddedDatabaseType.H2)
.addScript("classpath:org/springframework/batch/core/schema-drop-h2.sql")
.addScript("classpath:org/springframework/batch/core/schema-h2.sql")
.build();
}
@Bean(name = "transactionManager")
public PlatformTransactionManager getTransactionManager() {
return new ResourcelessTransactionManager();
}
@Bean(name = "jobRepository")
public JobRepository getJobRepository() throws Exception {
JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
factory.setDataSource(dataSource());
factory.setTransactionManager(getTransactionManager());
factory.afterPropertiesSet();
return factory.getObject();
}
@Bean(name = "jobLauncher")
public JobLauncher getJobLauncher() throws Exception {
TaskExecutorJobLauncher jobLauncher = new TaskExecutorJobLauncher();
jobLauncher.setJobRepository(getJobRepository());
jobLauncher.afterPropertiesSet();
return jobLauncher;
}
}
And the XML-based configuration:
<bean id="itemReader" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="resource" value="input/record.csv" />
<property name="lineMapper">
<bean
class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer">
<bean
class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="names"
value="username,userid,transactiondate,amount" />
</bean>
</property>
<property name="fieldSetMapper">
<bean
class="com.baeldung.batch.service.RecordFieldSetMapper" />
</property>
</bean>
</property>
<property name="linesToSkip" value="1" />
</bean>
<bean id="itemProcessor" class="com.baeldung.batch.service.CustomItemProcessor" />
<bean id="itemWriter" class="org.springframework.batch.item.xml.StaxEventItemWriter">
<property name="resource" value="file:xml/output.xml" />
<property name="marshaller" ref="marshaller" />
<property name="rootTagName" value="transactionRecord" />
</bean>
<bean id="marshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
<property name="classesToBeBound">
<list>
<value>com.baeldung.batch.model.Transaction</value>
</list>
</property>
</bean>
<batch:job id="firstBatchJob">
<batch:step id="step1">
<batch:tasklet>
<batch:chunk reader="itemReader" writer="itemWriter"
processor="itemProcessor" commit-interval="10">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
<!-- connect to H2 database -->
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.h2.Driver" />
<property name="url" value="jdbc:h2:file:~/repository" />
<property name="username" value="" />
<property name="password" value="" />
</bean>
<!-- create job-meta tables automatically -->
<jdbc:initialize-database data-source="dataSource">
<jdbc:script
location="org/springframework/batch/core/schema-drop-h2.sql" />
<jdbc:script location="org/springframework/batch/core/schema-h2.sql" />
</jdbc:initialize-database>
<!-- stored job-meta in database -->
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="transactionManager" ref="transactionManager" />
<property name="databaseType" value="h2" />
</bean>
<bean id="transactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
Now that we have the whole config, let’s break it down and start discussing it.
4.1. Read Data and Create Objects With ItemReader
First, we configured the cvsFileItemReader that will read the data from the record.csv and convert it into the Transaction object:
@SuppressWarnings("restriction")
@XmlRootElement(name = "transactionRecord")
public class Transaction {
private String username;
private int userId;
private LocalDateTime transactionDate;
private double amount;
/* getters and setters for the attributes */
@Override
public String toString() {
return "Transaction [username=" + username + ", userId=" + userId
+ ", transactionDate=" + transactionDate + ", amount=" + amount
+ "]";
}
}
To do so, it uses a custom mapper:
public class RecordFieldSetMapper implements FieldSetMapper<Transaction> {
public Transaction mapFieldSet(FieldSet fieldSet) throws BindException {
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("d/M/yyy");
Transaction transaction = new Transaction();
transaction.setUsername(fieldSet.readString("username"));
transaction.setUserId(fieldSet.readInt(1));
transaction.setAmount(fieldSet.readDouble(3));
String dateString = fieldSet.readString(2);
transaction.setTransactionDate(LocalDate.parse(dateString, formatter).atStartOfDay());
return transaction;
}
}
4.2. Processing Data With ItemProcessor
We have created our own item processor, CustomItemProcessor. This doesn’t process anything related to the transaction object.
All it does is pass the original object coming from reader to the writer:
public class CustomItemProcessor implements ItemProcessor<Transaction, Transaction> {
public Transaction process(Transaction item) {
return item;
}
}
4.3. Writing Objects to the FS With ItemWriter
Finally, we are going to store this transaction into an XML file located at xml/output.xml:
<bean id="itemWriter"
class="org.springframework.batch.item.xml.StaxEventItemWriter">
<property name="resource" value="file:xml/output.xml" />
<property name="marshaller" ref="marshaller" />
<property name="rootTagName" value="transactionRecord" />
</bean>
4.4. Configuring the Batch Job
So, all we have to do is connect the dots with a job using the batch:job syntax.
Note the commit-interval. That’s the number of transactions to be kept in memory before committing the batch to the itemWriter.
It will hold the transactions in memory until that point (or until the end of the input data is encountered).
The Java bean and correspondent XML configuration are displayed:
@Bean
protected Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager, @Qualifier("itemProcessor") ItemProcessor<Transaction,
Transaction> processor, ItemWriter<Transaction> writer) {
return new StepBuilder("step1", jobRepository)
.<Transaction, Transaction> chunk(10, transactionManager)
.reader(itemReader(inputCsv))
.processor(processor)
.writer(writer)
.build();
}
<batch:job id="firstBatchJob">
<batch:step id="step1">
<batch:tasklet>
<batch:chunk reader="itemReader" writer="itemWriter"
processor="itemProcessor" commit-interval="10">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
In this section, we’ll create a Spring Boot application and convert the previous Spring Batch Config to run in the Spring Boot environment. In fact, this is roughly the equivalent of the previous Spring Batch example.
5.1. Maven Dependencies
Let’s start by declaring spring-boot-starter-batch dependency in a Spring Boot application in the pom.xml:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
</dependency>
We need a database to store the Spring Batch job information. In this tutorial, we use an in-memory H2 database. Therefore, we need to use h2 with Spring Boot:
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>2.1.214</version>
<scope>runtime</scope>
</dependency>
5.2. Spring Boot Config
We use the @Profile annotation to distinguish between the Spring and Spring Boot configurations. We set the spring-boot profile in our application:
@SpringBootApplication
public class SpringBatchApplication {
public static void main(String[] args) {
SpringApplication springApp = new SpringApplication(SpringBatchApplication.class);
springApp.setAdditionalProfiles("spring-boot");
springApp.run(args);
}
}
5.3. Spring Batch Job Config
We use the batch job configuration the same as the SpringBatchConfig class from earlier:
@Configuration
public class SpringBootBatchConfig {
@Value("input/record.csv")
private Resource inputCsv;
@Value("input/recordWithInvalidData.csv")
private Resource invalidInputCsv;
@Value("file:xml/output.xml")
private Resource outputXml;
// ...
}
Starting with spring-boot 3.0, @EnableBatchProcessing annotation is discouraged. We declare manually JobRepository, JobLauncher and TransactionManager beans. In addition, JobBuilderFactory and StepBuilderFactory are deprecated, and it’s recommended to use JobBuilder and StepBuilder class with the name of the job/step builder.