Model profiler implementation
We need to profile models to record at least weights_load_time
and execution_time
offline, the read them into the model object in the controller, when loading the model in order to have the latency numbers available when scheduling the requests.
It could be implemented as a separate program that takes the model address, runs it for a few times and records the latencies in a ".profile" file. or could be added to the "convert" program.