Experience with Spring batch and docker

Windows 11 | IntelliJ | Maven

Sample spring batch application can be created from the following link

Creating a Batch Service

The following dependency is needed.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch</artifactId>
</dependency>

My sample application doesn’t need any database. If the database is not configured, you will get an error while starting the spring batch app. To circumvent this, I have extended my Configuration class with DefaultBatchConfigurer and added the following snippet.

@Configuration
@EnableBatchProcessing
public class BatchConfiguration extends DefaultBatchConfigurer {

    @Inject
    JobBuilderFactory jobBuilderFactory;

    @Inject
    StepBuilderFactory stepBuilderFactory;

    @Override
    public void setDataSource(DataSource dataSource) {
        //This BatchConfigurer ignores any DataSource
    }

Note: IntelliJ keeps throwing warnings for field injections. Go to IntelliJ settings -> Editor -> Inspections -> search for “Non recommended field injection” and change severity to “No highlighting”.

Working with FlatFileItemReader

1. Added tab-delimiter to read data from my flat file.

.delimiter(DelimitedLineTokenizer.DELIMITER_TAB)

2. FlatFileItemReader throws an exception and stops the batch if one of the rows is not delimited/parsed properly in a big file. To avoid this I have created the following ExceptionHandler class.

public class LocalExceptionHandler implements ExceptionHandler {

    private static final Logger log = LoggerFactory.getLogger(LocalExceptionHandler.class);

    @Override
    public void handleException(RepeatContext repeatContext, Throwable throwable) throws Throwable {
        if (throwable instanceof FlatFileParseException) {
            FlatFileParseException fe = (FlatFileParseException)throwable;
            log.error("!!!! FlatFileParseException, line # is: " + fe.getLineNumber());
            log.error("!!!! FlatFileParseException, input is: " + fe.getInput());
        }
        log.error("!!!! Message : " + throwable.getMessage());
        log.error("!!!! Cause : " + throwable.getCause());
    }
}

And I added the above class in Step bean. So while parsing/reading data from a large file, if one of the rows has bad data, we just log it and proceed.

.exceptionHandler(new LocalExceptionHandler())

To avoid batch to auto-start jobs during mvn install or while running SpringBootApplication, I added the following to application.properties

spring.batch.job.enabled=false

I wanted to have an API to control which job to run. So I added the following controller.

@RestController
public class JobLauncherController {

    @Inject
    JobLauncher jobLauncher;

    @Qualifier("<job_bean_name>")
    @Inject
    Job job;

    @RequestMapping("/first_job.html")
    public String handle() throws Exception{
        JobExecution jobExecution = jobLauncher.run(job, new JobParameters());
        return "Job started";
    }
}

The controller is part of spring-boot-web artifact, so added the following dependency.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>

Dockerize spring batch

Note: To install docker on a windows machine, I enabled following things.

1. Enabled Virtualization from the motherboard.
2. Enabled WSL (Windows Subsystem for Linux). Ran the following command to upgrade to version 2

wsl --set-default-version 2

3. Installed Ubuntu from the Microsoft Store app.

I didn’t create any Dockerfile in spring application, instead, I used Buildpacks which is part of SpringBoot. To create a docker image simply execute the following command.

mvn spring-boot:build-image

And run docker image with the following command.

docker run -d -p 8080:8080 --name "<my_container_name>" -v c:/data:/data <image-name>:<tag-name>

options used:

d -> To run the container in detached mode.

p -> To publish container ports. In the above example, I used the -p flag to publish the 8080 port from the host to the container to port 8080 within the container. This means anyone connecting to this host over port 8080 will be routed to the container via port 8080.

v -> To mount host volumes. In the above example, I am sharing the c:/data directory on the host (Windows 11) as /data to the container.

Get into the docker container’s shell

docker exec --user="root" -it <container_id> /bin/bash

option ‘user==root’ allows one to log in as root, so one can install required packages.

To install packages use the following commands.

apt-get update
apt-get install vim