Spring Batch: org.springframework.batch.item.ReaderNotOpenException: Reader must be open before it can be read

Question

I read SO related questions but the solutions don't work for me.

I get the org.springframework.batch.item.ReaderNotOpenException: Reader must be open before it can be read exception.

Below is my configuration:

@Bean
@StepScope
public ItemReader<Player> reader(@Value("#{jobParameters[inputZipfile]}") String inputZipfile) {
                final String [] header = { .. this part omitted for brevity ... };
                FlatFileItemReader<Player> reader = new FlatFileItemReader<Player>();


                System.out.println("\t\t\t\t\t"+inputZipfile);

                reader.setResource(new ClassPathResource(inputZipfile));
                reader.setLineMapper(new DefaultLineMapper<Player>() {{
                    setLineTokenizer(new DelimitedLineTokenizer() {{
                        setNames( header );
                    }});
                    setFieldSetMapper(new BeanWrapperFieldSetMapper<Player>() {{
                        setTargetType(Player.class);
                    }});
                }});
                reader.setComments( header );
                return reader;
}

@Bean
@StepScope
public ItemProcessor<Player, PlayersStats> processor(@Value("#{jobParameters[statType]}") String statType,
                                                                 @Value("#{jobParameters[season]}") String season){
                PlayersStatsProcessor psp = new PlayersStatsProcessor();
                psp.setStatisticType( StatisticType.valueOf(statType) );
                psp.setSeason( season );
                return psp;
}


@Bean
@StepScope
public ItemWriter<PlayersStats> writer(){
            return new CustomWriter();
}


@Bean
public Job generateStatisticsJob() {

        return this.jobs.get("generateStatisticsJob")
                .incrementer(new RunIdIncrementer())
                .start(processPlayerStats())
                //.end()
                .build();
}

@Bean
public Step processPlayerStats() {
           return this.steps.get("processPlayerStats")        
                        .<Player, PlayersStats> chunk(10)
                        .reader(reader(null))
                        .processor(processor(null,null))
                        .writer(writer())
                        .build();
}

The inputZipFile variable is set properly and the file exists on the drive. I checked in the FlatFileItemReader code and the ReaderNotOpenException occurs when the reader member of the reader class is not set. The reader member is set in doOpen method. It looks that doOpen is not called. The question is why ?

Answer 1

The issue disappeared when I change the return type of my reader bean from Item to FlatFileItemReader. It is still not clear to me why this is a problem since chunk().reader() accepts ItemReader as an input. I assume that there is some AOP magic under the hood which does FlatFileReader init and matches by the return type.

Answer 2

Since you put the reader in StepScope, the bean return type should be the implementing type FlatFileItemReader:

@Bean
@StepScope
public FlatFileItemReader<Player> reader(@Value("#{jobParameters[inputZipfile]}") String inputZipfile) {
            ...
            return reader;
}

If you specify the interface, the Spring proxy only has access to the methods and annotations specified on the interface ItemReader and is missing important annotations. There is also a warning (with a typo) in the logs:

2015-05-07 10:40:22,733 WARN  [main] org.springframework.batch.item.ItemReader is an interface.  The implementing class will not be queried for annotation based listener configurations.  If using @StepScope on a @Bean method, be sure to return the implementing class so listner annotations can be used.
2015-05-07 10:40:22,748 WARN  [main] org.springframework.batch.item.ItemReader is an interface.  The implementing class will not be queried for annotation based listener configurations.  If using @StepScope on a @Bean method, be sure to return the implementing class so listner annotations can be used. 

Currently the Spring Boot Batch example is also returning the ItemReader, so I guess other people will struggle with the same issues.

Answer 3

It is because ItemReader don't have the open method, using hte StepScope will create a proxy class based on the return type. It is also ok to return ItemStreamReader

Answer 4

I have fixed it by:

reader.open(new ExecutionContext());

Answer 5

I think you should increase your chunk size in processPlayerStats() step bean class, i.e from chunk(10) to chunk(100/more may be).

Answer 6

The method I defined is like below:

    @Bean
    @StepScope
    public ItemReader<BP> BPReader(){
       
        JdbcCursorItemReader<BP> itemReader = new JdbcCursorItemReader<BP>();
        ...
        return itemReader;
    }

The type I defined in the method is ItemReader which is an interface, the return type is JdbcCursorItemReader which is a sub-class of it. By changing the return type defination to JdbcCursorItemReader solved my problem

Answer 7

Instead of returning ItemReader, return ItemStreamReader, since it's a combination of ItemReader and ItemStream interfaces, so it recognizes that there is the open method that needs to be called.

Answer 8

Same problem here. Changing return type of my reader to the real implementation and adding to the reader

implements ItemStream

Did the trick for me