PowerShell Multithreading with PoshRSJob
Intro
PowerShell has had a native method for spawning multiple “threads” ever since I can remember, in the form of the *-Job
functions. They work OK, but there are a couple downsides:
- Each job is its own PowerShell process, so it takes a non-trivial amount of time and memory to spin each up
- There’s no built-in method for throttling the number of concurrent jobs
This combination will become an ugly mess if something spins out of control and you spawn dozens or hundreds of jobs. PowerShell jobs are better tailored to small-scale asynchronous background processing. You can wrap the functions to limit the number of concurrent jobs but again, there’s a lot of overhead involved in creating and tearing down jobs. Warren Frame created Invoke-Parallel
, which uses runspaces (lighter-weight than jobs) and allows for throttling, but isn’t quite as full-featured as jobs are.