Nugget Friday - Understanding Jakarta Batch Step Processing: A Developer's Guide

One of the most common questions about Jakarta Batch specification is "How do steps actually work under the hood?" Today's nugget takes a look at batch processing in enterprise Java applications by exploring step execution and the different types of steps available in Jakarta Batch. So grab your favorite beverage and let's dig in!

The Problem

When building batch applications, you may often struggle with choosing between different step types and understanding how they behave during execution. Should you use a chunk-oriented step or a batchlet? How do you configure them properly? And what about execution monitoring? These decisions can significantly impact your batch application's performance and maintainability.

The Solution: Grokking Step Types and Configuration

The Jakarta Batch specification provides two main types of steps: chunk-oriented steps and batchlets. Each type requires proper configuration in both Java code and the Job XML. Let's break them down.

Chunk-Oriented Steps

Chunk steps are perfect for data-intensive operations that process items in a "read-process-write" pattern. Here's a complete example including both code and configuration.

First, let's define our batch artifacts:

@Named

public class CustomerReader implements ItemReader {

    @Inject 
    @BatchProperty(name = "input.file")

    private String inputFile;

    

    @Override

    public Object readItem() throws Exception {

        // Read the next customer record

        return customer;

    }

}




@Named

public class CustomerProcessor implements ItemProcessor {

    @Override

    public Object processItem(Object item) throws Exception {

        Customer customer = (Customer) item;

        // Enrich or transform customer data

        return customer;

    }

}




@Named

public class CustomerWriter implements ItemWriter {

    @Override

    public void writeItems(List<Object> items) throws Exception {

        // Write the chunk of customers to the database

    }

}

Then let's configure the step in our Job XML:

<step id="processCustomers">

    <chunk item-count="10">

        <reader ref="customerReader">

            <properties>

                <property name="input.file" value="customers.csv"/>

            </properties>

        </reader>

        <processor ref="customerProcessor"/>

        <writer ref="customerWriter"/>

    </chunk>

</step>

The key thing to understand about chunk processing is that it:

• Reads items one at a time
• Processes each item as it's read
• Collects processed items into a "chunk"
• Writes the entire chunk in a single transaction

Batchlet Steps

Batchlets are perfect for task-oriented processing that doesn't fit the item-oriented pattern. Think file transfer, system command execution, or any other "do something once" type operation.

@Named

public class DataCleanupBatchlet implements Batchlet {

    @Inject

    @BatchProperty(name = "retention.days")

    private String retentionDays;

    

    @Override

    public String process() throws Exception {

        // Perform cleanup operation

        cleanupStaleData(Integer.parseInt(retentionDays));

        return "COMPLETED";

    }

    

    @Override

    public void stop() throws Exception {

        // Handle stop request gracefully

    }

}

And its corresponding Job XML configuration:

<step id="cleanup">

    <batchlet ref="dataCleanupBatchlet">

        <properties>

            <property name="retention.days" value="30"/>

        </properties>

    </batchlet>

</step>

Step Execution Monitoring with Listeners

Listeners allow you to monitor and intercept step execution at various points. They're perfect for logging, metrics collection and cross-cutting concerns. There are several types:

• StepListener: Monitors overall step lifecycle
• ChunkListener: Monitors chunk processing (for chunk steps)
• ItemReadListener: Monitors individual item reads
• ItemProcessListener: Monitors individual item processing
• ItemWriteListener: Monitors chunk writes

Here's an example of a step listener:

@Named

public class CustomerStepListener implements StepListener {

    

    @Override

    public void beforeStep() throws Exception {

        // Called once before step execution begins

        logger.info("Starting customer processing step");

    }

    

    @Override

    public void afterStep() throws Exception {

        // Called once after step execution completes

        logger.info("Completed customer processing step");

    }

}

And then we can configure it in your Job XML:

<step id="processCustomers">

    <listeners>

        <listener ref="customerStepListener"/>

    </listeners>

    <chunk item-count="10">

        <!-- chunk configuration as before -->

    </chunk>

</step>

Step Partitioning for Parallel Processing

With partitioning, a step can execute multiple instances of itself in parallel, with each instance working on a subset of the data. This is configured through a partition plan or mapper:

@Named

public class CustomerPartitionMapper implements PartitionMapper {

    

    @Override

    public PartitionPlan mapPartitions() throws Exception {

        PartitionPlan plan = new PartitionPlanImpl();

        plan.setPartitions(3); // Run on 3 threads

        

        Properties[] props = new Properties[3];

        // Set up properties for each partition

        for(int i = 0; i < 3; i++) {

            props[i] = new Properties();

            props[i].setProperty("partition.id", String.valueOf(i));

            props[i].setProperty("chunk.start", String.valueOf(i * 1000));

            props[i].setProperty("chunk.end", String.valueOf((i + 1) * 1000));

        }

        plan.setPartitionProperties(props);

        

        return plan;

    }

}

Configure partitioning in your Job XML:

<step id="processCustomers">

    <chunk item-count="10">

        <!-- chunk configuration as before -->

    </chunk>

    <partition>

        <mapper ref="customerPartitionMapper"/>

    </partition>

</step>

Why You Should Care

Understanding these step components helps you:

1. Make Better Design Decisions: Choose the right step type and know when to use listeners and partitioning.
2. Improve Performance: Use chunk steps for efficient data processing and partitioning for parallel execution.
3. Monitor Execution: Implement listeners to track progress and gather metrics.

Caveats

1. Transaction Scope: Chunk steps automatically handle transactions, but you need to manage your own transactions in batchlets, if needed.
2. Memory Management: With chunk steps, choose your chunk size carefully. Too large, and you risk memory issues; too small, and you lose performance.
3. Partition Overhead: While partitioning can improve performance, there's overhead in managing multiple threads. For small datasets, the overhead might outweigh the benefits.

Conclusions

Steps are the fundamental components of Jakarta Batch jobs. Learning about the different types of steps and their supporting components, such as listeners and partitioning, will help you to create more resilient and efficient batch applications.

• Use chunk steps for data-intensive, item-oriented processing
• Use batchlets for task-oriented processing
• Add listeners for monitoring and cross-cutting concerns
• Consider partitioning for parallel processing of large datasets

That's it for this week's nugget! Download your free copy of Payara Platform Community and start building better batch applications today. Happy Friday and happy coding!