dispatcher needs option to queue job but also observe log

this could perhaps be done by having the client side also tail (and parse) the log file that is produced by the server

But that would assume bi-directional communication on the remote machine?

if you use CLI tool or web API to run jobs/adhoc commands from a different
client, then it would require some new api ability to tail log output
remotely...

the new api ability would use the same underlying code. the ajax/json used
by the gui is not documented as an api.

the gui polls for new data. That is the simplest way to do this for the api.

if you want more realtime output, this requires the server to keep
connection open and the thread running

At some point yeah. Given the request/response nature of the webui, you'd have to have a way of changing to some stream mechanism, i.e. websockets. All in due time :)