implement framework for remote processing
sometimes, the request that is received via DASF should not be processed on the server that serves the backend, but rather on another server. as an example, the backend runs on a VM, and the real computation is done on a cluster that, however, has no connection to the internet. the backend on the VM that is receiving the request should just forward it to the cluster, and either execute a command on this cluster, or on the cluster there is a cronjob that then processes the request.
To enable such a framework, we need to add more options to the listen
command. namely
-
--dump-to
: a path where the request received via DASF should be dumped. Here we will first dump it to a temporary file and then move the request with the command specified in--dump-tool
-
--dump-tool
: an alternative to--dump-to
to use a custom way of modifying the dumped request, e.g. to usescp
-
--cmd
: this parameter can be specified to run a specific command when the request has been dumped to the location specified via-dump-to
. for instance something likessh user@machine python -m some.backend process-request
. We will then add the dumped request as parameter
Edited by Philipp S. Sommer