Questions tagged [node-worker-threads]
The node-worker-threads tag has no usage guidance.
121
questions
40
votes
1
answer
14k
views
How do 'cluster' and 'worker_threads' work in Node.js?
Did I understand correctly: If I use cluster package, does it mean that
a new node instance is created for each created worker?
What is the difference between cluster and worker_threads packages?
10
votes
1
answer
4k
views
How to pass class instances with function through worker_thread.Worker to use function?
To contextualize, I would like to use class instances functions through a Worker Thread from NodeJS "worker_thread" module.
In one main.js file I declare a class and instanciate a new Worker ...
- 164
9
votes
3
answers
28k
views
Typescript cannot find module 'worker_threads'
I am trying to use node.js 'worker_threads' library with Node v14.15.1 and getting this error
Cannot find module 'worker_threads' or its corresponding type declarations.ts(2307)
tsc
src/api/...
- 404
7
votes
2
answers
437
views
Optimizing a file content parser class written in TypeScript
I got a typescript module (used by a VSCode extension) which accepts a directory and parses the content contained within the files. For directories containing large number of files this parsing takes ...
- 613
7
votes
3
answers
654
views
Request are not distributed across their worker process
I was just experimenting worker process hence try this:
const http = require("http");
const cluster = require("cluster");
const CPUs = require("os").cpus();
const numCPUs ...
- 426
5
votes
1
answer
3k
views
How to share context between worker threads
I would like to create a worker thread in a node.js app and pass the current context to the new thread, so I would be able to access my variables and functions within the new thread, Is there is a ...
- 138
5
votes
1
answer
1k
views
Managing multiple long-running tasks concurrently in JS (Node.js)
Golang developer here, trying to learn JS (Node.js).
I'm used to working with goroutines in Go, which for the sake of simplicity let's assume are just threads (actually they're not exactly threads, ...
- 479
4
votes
1
answer
5k
views
How to listen terminate signal from worker in nodejs?
I have a main process which spawn one new thread using worker_threads. Main process in some particular cases have to close this thread without mattering if it has finished its task or not, so ...
- 181
4
votes
0
answers
809
views
Is there a way to view any worker_threads that are currently in use in node.js?
I am looking at worker_threads in node and have a simple endpoint that will create a new worker thread and run a CPU intensive script that takes a long time to run.
app.get('/test', (req, res) => {
...
- 43
4
votes
1
answer
461
views
Why is all of my output appearing before each brute-force attempt when attempting to crack an ethereum keystore wallet file in node.js
I found a post describing how to recover an Ethereum wallet keystore by guessing a single password, however, it uses node synchronous code, and I'm trying to convert it into asynchronous code so that ...
- 65
4
votes
0
answers
310
views
tsconfig paths in worker_threads
I'm spinning up some node workers and need each worker to pull code from a path expressed in tsconfig's paths compiler options. How do you configure worker to acknowledge paths from tsconfig?
here's ...
- 5,247
4
votes
0
answers
2k
views
How to handle Node.js Worker threads in Webpack?
Using Web Workers for the browser is handled by the official worker-loader Webpack module.
Unfortunately worker-loader does not handle Node.js Worker threads.
I'm using Webpack to bundle an AWS Lambda ...
- 30.5k
3
votes
2
answers
8k
views
Is it possible to use Node worker threads to perform database inserts?
I recently read about Node's "worker_threads" module that allows parallel execution of Javascript code in multiple threads which is useful for CPU-intensive operations. (NOTE: these are not ...
- 152
3
votes
1
answer
4k
views
Worker threads in express application
I have an express backend application, which listens for http requests from a web application. This backend application is running on AWS ECS Fargate.
So my question is, does it makes sense use ...
- 434
3
votes
1
answer
3k
views
NodeJS Worker Threads and AWS Lambda: Inconsistent Output
I am trying to execute the following AWS lambda:
index.js:
const test = require("test");
exports.handler = async (event) => {
await test.main();
};
test.js:
const {Worker} = require(&...
3
votes
2
answers
4k
views
Nodejs Worker threads - How many workers should I spawn in the pool?
I want to use Nodejs Worker threads to handle cpu intensive tasks. I will create a pool of workers available for this.
My question is: How many workers should I spawn in the pool?
Assuming I have a 4 ...
- 192
3
votes
2
answers
5k
views
How can I get the memory usage of a worker created by worker_threads in nodejs
I created a worker using the worker_threads. Like this:
const WorkerThreads = require('worker_threads');
const worker = new WorkerThreads.Worker('./path/to/a.js');
if (countMemory(worker) > 100 * ...
- 1,974
3
votes
0
answers
550
views
how to work with response object in nodejs stream, exceljs and worker thread
I am using worker thread and stream at same time in node JS project. At initial I was not able to pass res object through main process to worker thread. I saw many stackoverflow question and solution ...
2
votes
1
answer
975
views
Is there a way to run native code modules using worker threads?
Is it possible to use native code modules on worker threads?
I am creating each worker thread as shown:
const { Worker } = require("worker_threads");
const auth = require("./auth.json&...
- 177
2
votes
1
answer
443
views
Adding fast random access in reading and parsing file using csv-parse and node
I was using the csv-parser library to handle csv parsing in node. The file can be huge ranging from 50,000 to 500,000 lines, maybe even larger. I had to perform some computations on the csv, after ...
- 39
2
votes
2
answers
4k
views
how to go multithreaded for puppeteer using worker-threads for web-automation purpose
hello so am doing some web automation and I want to open run puppeteer multithreaded what I mean like open the same page 10s of times and what I understood of what I read the worker thread is the best ...
- 501
2
votes
1
answer
296
views
How would you create X amount of worker threads based on user input?
For example, a user inputs 10, and 10 worker threads are created that run a function.
I cannot figure out how to do this, i've looked at the docs and i'm blanking.
- 98
2
votes
1
answer
2k
views
How to use socket.io with node.js worker_threads
Currently I am introducing worker_threads to a game I am developing. Thea idea is to have each game room in a separate thread, so that they can be processed in parallel (yeah I know that will only ...
- 1,015
2
votes
0
answers
1k
views
When to use child process over worker threads?
I'm still trying to understand what is worker threads, and how it's different from child process so please bear with me.
So I'm currently building a desktop app with Node.JS + Electron. The app would ...
- 146
2
votes
0
answers
437
views
Inconsistent console logs for NodeJS worker threads in AWS Lambda
I have an lambda function which spins up a worker thread.
hello-world.js:
const { Worker } = require('node:worker_threads')
export const hello = async (event, context) => {
console.log('Hello ...
- 1,013
2
votes
1
answer
448
views
Best way to trigger worker_thread OOM exception in Node.js
I would like to test code that reports when a worker_thread OOMs. Or at least that the parent is OK despite the thread crashing due to OOM reasons. I would like to specifically test Node.js killing a ...
- 51
2
votes
0
answers
484
views
Unable to import modules in node-worker-thread-pool
I am using node-worker-thread-pool package.I am unable to import certain node modules into my worker.ts, giving an error SyntaxError: Cannot use import statement outside a module. How to resolve this. ...
- 21
2
votes
1
answer
2k
views
How to bundle worker_thread dependencies with webpack
I'm writing an electron application that uses electron forge for setup and configuration.
The project uses Webpack 5 as a bundler and typescript as the development language. I'm having trouble running ...
- 461
2
votes
0
answers
905
views
Worker thread immediately exits
I want to run a blocking task in a working thread to avoid blocking the event loop.
I got two environments, my workstation and a server. Both have the same version of NodeJS (14.16.1).
Here is a ...
- 41
2
votes
0
answers
830
views
Node v12 cant resolve worker_threads
I am using nodejs v12.18.3 and npm v6.14.6 to make a react project. I am trying to import worker_threads module but it can't resolve the module name (even though vs code can find the definition). I ...
- 346
2
votes
0
answers
398
views
how to open multiple instances of puppeteer without affecting the speed of the instances/network
so am doing automation for some website which i want to open multiple(6-10) instances at the same time in that purpose am doing that using Worker threads but the thing is that the more instances i ...
- 501
2
votes
1
answer
2k
views
Worker thread communication protocol
I stumbled upon worker threads in NodeJS and I started investigating on lower level abstraction how intercommunication and data sharing works between them, especially postMessage function that is ...
- 89
1
vote
1
answer
445
views
Worker Thread, other file than self, cannot find module
I want to have my worker thread as an external file from the runtime file.
My current folder structure is
src/
> service.ts // my 'main'
> thread/
>> test.js
inside my service.ts i have ...
1
vote
2
answers
1k
views
Is it possible to use a string for NodeJS worker thread constructor?
In NodeJS documentation, the Worker constructor looks like it requires a path to a file that contains the code to execute on the new worker thread.
It's something like:
const encode_worker = new ...
- 2,889
1
vote
2
answers
1k
views
Node.js performance in file system I/O across multiple disk drives: worker threads or not?
I've read several questions and answers here about the performance benefits of Node.js' ability to performantly handle file I/O operations quickly in a non-blocking way versus using worker threads ...
- 52
1
vote
1
answer
248
views
Should I use worker threads for sending more than 5000 HTTP requests?
If I want to send messages to more than 5000 users in Node.js, which takes a long time, and I don't want it to be render blocking, is "Worker threads" what I need?
The documentation says:
...
- 666
1
vote
1
answer
557
views
Facing problem of making node worker thread promises
I am trying to make a test of offloading task to worker thread.
The workflow is quite simple, I have 100 promises of array, each promise will create a worker thread.
The main thread will pass the ...
- 93
1
vote
1
answer
929
views
create worker file (worker threads nodejs)
I want to create a worker to send back an array as response to my main file, but I can't understand why the worker is not sending postMessage to main. Here is some dummy code to explain
My main file:
...
1
vote
1
answer
72
views
Why worker threads aren't working when creating an class instance from other file?
I have this typescript file:
import { Worker, WorkerOptions, isMainThread, parentPort } from "worker_threads";
import path from "path";
export class SimpleWorker {
private ...
- 1,068
1
vote
0
answers
70
views
How many threads Node creates per core when using worker threads?
I'm running node js worker threads on MacBook Air M1. There are 8 core. And using PISCINA to create threads. I'm getting 12 threads. Are those for one core or on all 8 core. How do I know if main ...
1
vote
0
answers
273
views
How to get worker.threadId inside a worker using worker_threads?
I'm using the nodejs with worker_threads module.
I created several workers.
I need to get the id of each worker within the worker itself.
I know that in the cluster module, just call cluster.worker.id ...
- 63
1
vote
0
answers
68
views
Module not found when using typescript in worker_threads
const worker = new Worker('./counter.js')
my index.ts file
import { Worker, isMainThread } from "worker_threads";
if (isMainThread) {
const worker = new Worker('./counter.js')
...
- 19
1
vote
0
answers
33
views
Problems using sqlite3 while processing images with sharp
I'm downscaling images within a separate worker thread using the sharp library. The code snippet goes like this:
async function create_preview(path) {
return lib.sharp(path)
.resize({ width: ...
- 6,395
1
vote
0
answers
137
views
Cannot use worker_threads in electron main thread
In Electron v21.3.0 and electron-forge, when i try to run a node worker thread in the main process, i cannot require anything from the module 'worler_threads'
// myWorker.js
const { parentPort } = ...
- 405
1
vote
1
answer
369
views
Worker thread postMessage() vs command line command
I recently learned about Worker threads in Node JS. I was trying to create a worker thread to run Stockfish chess engine in node js.
The npm package I am using for this is called stockfish. I tried ...
1
vote
0
answers
828
views
How to wait for worker thread to complete in the worker thread itself to do some computations?
I have an existing app which uses worker concept to do audio download from an API. Now I would like to cut the audio once the download is complete in the same worker but the problem is before the full ...
- 11
1
vote
0
answers
320
views
Cannot load data url in Worker Thread
My current worker thread which works fine with file path that tries to console log in that JS file:
console.log("hit here");
Now I get the base64 of that file and use Data URL to replace JS ...
- 388
1
vote
0
answers
250
views
Is it possible to pass stdout to the nodejs worker?
I write a nodejs console application that does some logic that blocks the main thread.
I also want to use ora terminal spinner:
const sp1 = ora('runnning job 1').start();
job1();
sp1.succeed();
...
- 11.1k
1
vote
1
answer
1k
views
Node.js- share a heavy variable from parent to worker threads
I want to share a variable between parent and worker threads. The variable is heavy (100+ MB - arrays of JSON data) so if I pass it as an argument or re-read from data source, it drastically increases ...
- 197
1
vote
0
answers
280
views
Packaging dependencies imported into a Node worker thread written in TypeScript, with webpack
We have a NodeJS worker thread written in TypeScript. The worker thread imports a third-party package (fluent-ffmpeg, though I don't think that specific package is the problem here) and uses said ...
- 1,480