48

I am trying to find an equivalent of Celery project for Java environment, I have looked at Spring Batch, but are there any better alternatives for distributed task queues.

Thanks.

Improve this question
1

9 Answers 9

Reset to default
42

What Celery is doing is very much akin to EIP, and SEDA with convenient task scheduling... (all you have left to do is add some DB, and async HTTP networking and you have got a complete enterprise quality stack).

Basically in Java there is the Spring way, the Java EE way, and the Hadoop way:

  • Spring: Spring Integration + Spring Batch + RabbitMQ
  • Java EE: Mule + Quartz or EJB Scheduling + HornetMQ
  • Hadoop: Capacity + ZooKeeper

Those are roughly in order of ease of setting up.

Improve this answer
12

Jesque (https://github.com/gresrun/jesque) is a Java distributed task queue library. It is a Java port of the Resque library (https://github.com/defunkt/resque), which is described like this on its GitHub page:

Resque (pronounced like "rescue") is a Redis-backed library for creating background jobs, placing those jobs on multiple queues, and processing them later."

Improve this answer
5

Quartz has worked for me in the past. It's integrated with Terracotta now, so it should be easy to distribute. http://quartz-scheduler.org/

Improve this answer
7
  • 3
    Is it possible to do on-demand job execution with Quartz scheduler, rather than executing a job on a specific schedule?
    – Zakiullah Khan
    Mar 6, 2012 at 2:28
  • @ZakiullahKhanMohamed I'm very late but yes, this is possible. You can schedule a job to be run immediately.
    – GuiSim
    Jul 29, 2016 at 21:01
  • 2
    Just a heads up: the open source 'free' version does not have any support for distribution. Terracotta claims the enterprise version does.
    – Adam Marcionek
    Sep 27, 2016 at 20:07
  • @AdamMarcionek not sure what you mean by that. all i see is apache license and i thought that means you're basically free to distribute your software even if you're using quartz
    – dtc
    May 22, 2020 at 20:39
  • 1
    @dtc Not distribution of code, but distribution of tasks, which is locked behind the enterprise version. And yes, as you say, its not the same as the OP.
    – Adam Marcionek
    Mar 23, 2021 at 18:58
3

Celery is primarily based on Erlang/RabbitMQ. RabbitMQ has a Java client library that might be useful. Also, there is octobot which has a RabbitMQ backend.

Improve this answer
2
  • If I am not wrong Celery is largely written in Python and not Erlang, I would agree on the note that RabbitMQ is erlang based. Looking at octobot, thanks.
    – Zakiullah Khan
    Mar 6, 2012 at 14:44
  • 1
    I think he means that RabbitMQ is written in Erlang, so you need to install that to use it.
    – fabspro
    Jun 1, 2014 at 9:41
3

Distributed Java tasks scheduling and execution https://redisson.org/

Improve this answer
3

I found JobRunr in Java world is pretty similar to Celery in Python world. It is a library for background processing in Java, distributed in nature and backed by persistent storage.

From their docs,

Incredibly easy way to perform fire-and-forget, delayed, scheduled and recurring jobs inside Java applications using only Java 8 lambda's. CPU and I/O intensive, long-running and short-running jobs are supported. Persistent storage is done via either RDBMS (e.g. Postgres, MariaDB/MySQL, Oracle, SQL Server, DB2 and SQLite) or NoSQL (ElasticSearch, MongoDB and Redis).

Improve this answer
0

The closest thing I've found is Octobot: https://github.com/cscotta/Octobot Not so much documentation though... there used to be a website for it at octobot.taco.cat, but I haven't seen that load lately. I haven't personally used Octobot, but I've often seen it recommended as a Celery for Java.

Improve this answer
0

I haven't been able to find anything as easy to use as Celery for Java. Most of the solutions recommended to use a message queue. But Celery sits one level of abstraction higher than the queue. Instead of messages and consumers, you can think in terms of tasks and workers, results, retries etc.

I also needed to implement some bridge for a company using both Java and Python so I started this project:

celery-java - Celery client and worker in Java, compatible with their Python counterparts.

Beware, it's very immature as of now.

Improve this answer
-7

Apache ActiveMQ http://activemq.apache.org/

Apache Kafka http://kafka.apache.org/

Improve this answer
1
  • 5
    I dont think ActiveMQ and Kafta are Worker/Task Queue like celery. They are Message brokers and can be compared with RabbitMQ if you mean that.
    – Rupesh
    Feb 16, 2016 at 8:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.