In this case, apparently, it would seem that the processing consists of two distinct steps: first, the discovery of the complete list of paths that need to be processed (which cannot be parallelized), followed by, second, the processing of files within the paths thus-discovered (which, apparently, can).
Therefore, I simply suggest that you should follow BrowserUK’s sage advice exactly as given. The main-thread will first undertake, and fully complete, the file-path discovery process. “That is step-one.” (Since the underlying software does not let you be recursive, you will have to “push” directory-names onto some kind of pushdown-stack as-found, then continue until that stack is empty.) Then, having done all of that as described, it will move forward to “step two.” It will launch some “pool” of independent processes to deal with each path in parallel, because the “step two” process clearly is something that can(??) be parallelized. The parent thread will feed path-names into the queue, then it will feed undefs into the same queue, then it will wait for all of the children to end. Clearly, none of my suggestions would apply to this particular case. Therefore, take BrowserUK’s excellent recommendations, and go.
Uhhhh... but first... Do be certain that omnidb can, in fact, support an arbitrary number of parallel requests that originate from the same process context! (And when I say, “be certain,” I do mean ... “prove it!”) The folks who designed omnidb, all those years ago, might have taken certain shortcuts.
And here’s a quick way to (maybe) find out, in a Un*x-like environment: issue one omnidb-command in the shell, adding the “&” command to cause it to launch as a background-job, then immediately issue a second one in the foreground. Prefix both of these with the time command, and be sure that both commands will sensibly take some time to complete. (Enough time to actually allow you to issue both of the commands quickly-enough that both actually run in parallel.) Verify that both of these actually do run in-parallel with one another, (a) without conflict, and (b) without one of them simply being blocked by the other one. The designers of omnidb may or may not have anticipated parallelism (especially, within the context of a single process), and, if they did, they might have taken a shortcut. Now is the proper time to find out.