Controller base class and simple implementation
Once both the Client<>Controller network API and the Controller<>Worker network APIs are completed, this task is to put together a dummy controller to tie everything together.
The controller is a standalone binary that listens on a port for connections from clients. The controller is configured to connect to some specific hostnames/ports for the workers.
The base class for the controller implements both the workerapi::Controller
and the clientapi::Client
interfaces; to receive requests from clients and to receive results from workers.
This dummy worker should do the following simple logic: when an inference request shows up, break it down into a loadWeights
, an infer
, and an `evictWeights action. Only submit each action after the previous has completed; then respond to the client.
Part of implementing this controller will be to separate out the start-up phase where the controller connects to all workers.