Questions tagged [spring-batch]
Spring Batch is a lightweight, comprehensive framework aimed at enabling the development of batch applications vital for the daily operations of enterprise systems. Batch applications in this context refers to automated offline systems targeted towards bulk data processing.
8,528
questions
0
votes
0
answers
6
views
Restarting the appropriate chunk due to abnormal shutdown of the spring batch
There might be some awkward parts because of the translator
I'm in the process of deploying and I'm going to start again when I'm out of trouble
job execution-exit_code = unknown
step execution-...
0
votes
0
answers
14
views
Spring Batch Linemapper for reading a CSV file which can have multiple columns with the same name
The requirement is to read and apply certain analytical functions on the Jira extracts for several projects(in order of 100s) in the organization. The Jira extract function exports the CSV file which ...
- 390
-1
votes
0
answers
15
views
Spring Batch : async with AsyncItemProcessor and AsyncItemWriter
I want to improve my spring batch performance with async processing
I have noticed the class AsyncItemProcessor has a method setTaskExecutor :
public AsyncItemProcessor asyncDataProcessor() throws ...
- 1,259
0
votes
0
answers
9
views
Spring Batch : Adding TaskExecutor bean on my config causes errors
I have a basic Spring Boot batch application (reading datas from SQL Database, processing it and updating in the same SQL database). I don't put the code as it is long but it's very basic.
My ...
- 1,259
-1
votes
0
answers
17
views
If the spring batch ends incorrectly, start with the chunk [closed]
First time writing on a famous stack overflow. Thank you
Spring Batch question.
We are developing a batch program that reads all csv files from a specific path and writes them to db
[problem]
I want ...
1
vote
1
answer
25
views
AOP configuration seems to be invalid - Spring batch (Skip Listener)
I am using spring batch and I'm trying to do a POC to skip the entries for which I get an exception.
To do this, I have written a SkipListener with some logic to populate an external table with the ...
- 11
-1
votes
0
answers
14
views
How can i restart my Spring batch step successfully
I have a spring boot 3 projet with spring batch with one step.
When I execute my batch for the first time, at the batch_job_execution table level everything goes well and when I want to execute it for ...
- 315
1
vote
0
answers
29
views
How to Pass Data Among Steps in a java Spring Batch Application
I have a spring batch project to read two txt files (Cyties and Histories) in two different steps. I use the ItemsReaders, everything is fine, I can even write them together in a single new file in ...
- 11
0
votes
0
answers
13
views
Partitions are getting repeated in spring batch
I am using partition in spring batch and pgbouncer (Connection pooler for postgre) and I am creating the Partition in my code using following code -
public Map<String, ExecutionContext> ...
- 23
0
votes
0
answers
15
views
Parameter 0 of constructor in required a bean ... that could not be found
In a SpringBatch application I'm developing, I want to make my REST calls from an infra-rest module, but I'm encountering an error concerning bean recognition.
Here is the classes :
@Service
@...
- 1
0
votes
0
answers
11
views
How to deal with the removal of support of running multiple batch jobs at once?
Spring Batch recently removed the opportunity to run multiple batch jobs at once.
So now, you can still package multiple batch jobs into the same application but for every execution of the batch ...
- 354
1
vote
0
answers
25
views
Caused by: Error : 8177, Position : 0, Sql = INSERT INTO BATCH_JOB_INSTANCE(JOB_INSTANCE_ID, JOB_NAME, JOB_KEY, VERSION) in spring batch 5
I am using spring batch 5 using oracle. I have more than 30 batches from multiple projects running using the database. But it shows below error:
error code [8177]; ORA-08177: can't ...
- 4,961
0
votes
0
answers
14
views
Multi-Threading KafkaItemReader using SynchronizedItemStreamReader in Spring Batch
I'm comparing multiple options regarding to KafkaConsumer in SpringBatch.
With the example from spring batch project, I wrote simple code which runs KafkaItemReader as it is intended to use - single ...
-1
votes
1
answer
30
views
Initialize secondary datasource based on JobParameter at runtime / lazily
I'm working on a Spring Batch app where I have a batch data source configured during context startup (as standard) and I want to have a secondary data source that must be initialized based on a job ...
- 1,842
0
votes
1
answer
15
views
Can Spring Batch run in Master Slave mode on AWS Batch?
I'm developing a Spring Batch application to process 1 million records within 1 hour duration everyday. I'm planning to use Remote Partioning feature to concurrently process the records in multiple ...
- 31
0
votes
0
answers
24
views
Can 2 scheduled batch jobs running in spring boot, on seperate microservices interfere with each other?
I have a strange error occurring in my java spring boot project where I create an @entity database object, and it says that it violates the primary keys uniqueness constraint when being written to the ...
0
votes
0
answers
20
views
extend execution time limit in Spring Batch
I have a big problem here, Spring Batch + TaskExecutor doesn't execute a thread by more than 10 minutes and of course that I need that eventually the batch job should take 2 hours if necessary.
Y just ...
- 23
0
votes
2
answers
47
views
read and write two files in one step in one file
I am doing a spring batch project in which I have to read and modify several TXT files and write them in a new file in JSON format. I created my steps for each file, they are written one after the ...
- 11
0
votes
1
answer
80
views
+50
Spring batch-5 ORA-08177: can't serialize access for this transaction using Oracle database
I am using spring batch 5 using oracle. I have more than 15 batches running using the database. But it shows below error:
error code [8177]; ORA-08177: can't serialize access for this ...
- 4,961
0
votes
0
answers
48
views
MockStatic and MockConstruction not working in spring batch test cases
I am trying to mock static function and new instantiations in the spring tasklet step testing, but the mocks are not working and are going to the actual implementations.
@SpringBatchTest
@...
0
votes
0
answers
24
views
spring batch tasklet to write a file reexecute the whole job
I implemented a spring batch tasklet that
reads two files
merges both txt files into a single String list,
writes the list content to a file under resource folder
The issue is that the step 3 (the ...
- 19
0
votes
1
answer
35
views
o.s.boot.SpringApplication , Error creating bean with name 'entityManagerFactory' defined in class path resource
My app is a simple spring-batch application which is introduced in https://spring.io/guides/gs/batch-processing by some changes
but I find the following error when I compile it:
Error starting ...
0
votes
0
answers
20
views
Spring batch jpaPagingItemReader Read Map from for step context and apply to queryString's inclause using xml configuration
Below is my step configuration. productIds is a HashMap attached to Step Execution Context
<bean id="readCustomer" class="org.springframework.batch.item.database.JpaPagingItemReader&...
0
votes
1
answer
33
views
spring batch exeption java.lang.IllegalStateException: Transaction manager must be provided
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'runjob' defined in class path resource [com/example/inseespringbatch/BatchConfiguration.class]: Failed to ...
- 11
1
vote
0
answers
36
views
Pages are getting skipped in a step of a job in Spring Batch
Context: I am currently working on reading data from a database using pagination. To achieve this, I am using a RepositoryItemReader with a batch size of 1000. The reader fetches data from the ...
- 33
0
votes
0
answers
10
views
Progress bar for my Spring Batch Application - need to get number of Chunks or total number of items
I am trying to build a progress bar for my spring batch application and for that I am stuck at point.
I am unable to get the total number of items or total number of Chunks during runtime.
Can someone ...
0
votes
0
answers
31
views
Spring Batch 3 to 5 migration: Job not launching
I am migrating Spring Batch 3 to 5.1.1, and have made the following changes in the Configuration class, after which Jobs are not launching.
Spring Batch 3:
package com;
import com.jobs.PropManager;
...
- 11
1
vote
0
answers
10
views
csv file reader, batch java spring
I am looking to read a csv file and convert the data into a Tarification object. At execution I get a parsing error. I do not understand why.
I have already tried removing quotes, changing delimiters, ...
- 37
0
votes
0
answers
25
views
JdbcCursorItemReader vs JdbcPagingItemReader impl and JpaPgaingItemReader and queryprovider
public JdbcPagingItemReader<Point> reader() {
return new JdbcPagingItemReaderBuilder<Point>()
.name("reader")
.pageSize(chunkSize)
....
- 1
-1
votes
2
answers
59
views
How to customize the Spring Batch default configuration without breaking autoconfiguration
How can I customize Spring Batch's DefaultBatchConfiguration without breaking the autoconfiguration provided by Spring Boot in SpringBootBatchConfiguration?
I am using a Spring Batch job in a Spring ...
- 169
0
votes
1
answer
64
views
Is there any catch in Reading and Updating to the same Oracle table in Single Threaded Spring batch job?
I have written a Single-threaded Spring Batch Job to fix data in a table column.
i.e. it's a simple job with no parallel or multi-threaded steps. It simply uses a JdbcCursorItemReader to "stream&...
- 665
-1
votes
2
answers
42
views
Starting first job causes - Dispatcher has no subscribers for channel exception when declared multiple DirectChannels
I have a Spring Batch application which is successfully running spring batch jobs but i have exception when declaring multiple DirectChannels.
The exception happens when i start the "firstJob&...
0
votes
0
answers
10
views
Looping Over a Group of Steps in Spring Batch
I am working on a Spring Batch job with the following workflow:
Start
Step: Fetch the list of IDs to export via an SQL query.
While there are data in the list, perform the following steps:
Step: ...
- 250
-2
votes
0
answers
20
views
Already value [datasource.ConnectionHolder@6b144bd8] for key [datasource.DriverManagerDataSource@56739ee9] bound to thread [main]
org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is java.lang.IllegalStateException: Already value [org....
- 1
0
votes
0
answers
31
views
How to Use ControllerAdvice for Exception Handling in Spring Batch Application?
I am relatively new to Spring Boot and Spring Batch, and I'm currently working on a project where I need to implement exception handling. However, I'm facing difficulties in getting ControllerAdvice ...
- 15
0
votes
0
answers
12
views
Writing in multiple related tables in spring batch
I need to write three tables with a steplet: A, B and C.
B references A with a foreing key, C references B with a foreing key
In both cases, the foreign key is the primary key of the parent table, ...
0
votes
0
answers
19
views
Restarting a FAILED job is not processing the failed chunk data again : Continuation
This is continuation for Restarting a FAILED job is not processing the failed chunk data again.
This is my step definition for spring batch partition approach,
@Bean public Step step1() { return new ...
- 169
0
votes
0
answers
35
views
Spring Batch Not persisting Data Properly in MYSQL Database
I am fairly new to Spring batch so maybe I am missing something. After running this spring batch I cannot seem to persist the data(List) in database(Order_Summary_Details Table) This is my ...
0
votes
1
answer
26
views
Strange Spring Batch exception, something I did wrong?
I can break a working test with this diff:
- protected void ingest(String filename) throws Exception {
+ protected JobExecution ingest(String filename) throws Exception {
JobParameters ...
- 2,900
0
votes
0
answers
20
views
Parameter 0 of method jobBean in com.nextuple.BatchProcessing.config.BatchConfig required a bean named 'dataSource' that could not be found
So actually I am developing a Springboot application to read a static csv file and put the data into MongoDB after performing some small processing. Any help would be greatly appreciated.
This is the ...
- 11
0
votes
0
answers
18
views
is spring batch supports parquet file reading and writing [duplicate]
I want to read parquet file in java using apache-parquet but lack of knowledge and documentation available on internet not able to achive it.
trying to find if spring batch supports the parquet file ...
- 41
2
votes
1
answer
30
views
Spring Batch 5 - Don't want it to automatically create tables in database
I am migrating from Spring Boot 2 to Spring Boot 3, and from Spring Batch 4 to Spring Batch 5. I don't want Spring Batch to create the BATCH_JOB_INSTANCE tables in my database.
@Autowired
private ...
- 23
0
votes
0
answers
21
views
Transaction flow for distributed transaction in Spring Batch
I am relatively new to spring batch. I have a spring batch which fetch's an event from oracle DB and perform business validations and updates business tables for example - generates new entries in ...
0
votes
0
answers
64
views
How to solve JobBuilder deprecated in SpringBatch 5.1.1
I am using STS 3.2.3 and SpringBatch 5.1.1
When I build a batch job, I replace JobBuilderFactory with JobBuilder according to the 5.0 migration guide. But JobBuilder is also deplicated. How can I ...
0
votes
2
answers
45
views
Why is spring boot ignoring my schema.sql file?
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>3.2.2</version>
...
- 2,900
0
votes
1
answer
26
views
How Can I use Streaming or Pagination in Spring Batch
I'm so new on spring-batch. I just try to fetch all data from table via stream and save to csv file.
You know that to read table effectively, we can use jpa stream.
@Repository
public interface ...
- 1,079
0
votes
0
answers
29
views
Spring Cloud DataFlow - SpringBoot3 not showing stats of execution
When running Spring DataFlow in kubernetes and deploying docker images I'm not seeing boot3 stats from the execution. I do see the data in the database (in the tables prefixed with boot3), but not in ...
- 4,339
0
votes
0
answers
18
views
Use AggregateItemReader to read multiple db records to group for processing spring batch
We have to migrate data from an old DB to another one. We use a stored procedure to retrieve the data from the DB. The data is a list of items where each item has its listId. For each list to be ...
- 1
0
votes
0
answers
16
views
Hibernate 6.4 : org.hibernate.NonUniqueObjectException: A different object with the same identifier value was already associated with the session
Technology stack:
openJdk17
Spring Batch 5
Spring Boot 3
Spring data Jpa 3
Hiberante 6.4
My Entity :
@Entity
@Table(name = "CMN_TXN_DATA")
public class TransactionEntity
{
@Column(name = &...
0
votes
0
answers
34
views
Spring Batch: Why do I get exceptions from nowhere in my Custom Item Excel Writer (Apache POI)?
I'm implementing my own custom item writer that gets an excel as input data, and returns as a output another excel which is the same file but with a specific row modified where all its cells are -1.
...
