ColdStartProxy¶
- class lightning.app.components.serve.auto_scaler.ColdStartProxy(proxy_url)¶
Bases:
objectColdStartProxy allows users to configure the load balancer to use a proxy service while the work is cold starting. This is useful with services that gets realtime requests but startup time for workers is high.
If the request body is same and the method is POST for the proxy service, then the default implementation of handle_request can be used. In that case initialize the proxy with the proxy url. Otherwise, the user can override the handle_request
- async handle_request(request)¶
This method is called when the request is received while the work is cold starting. The default implementation of this method is to forward the request body to the proxy service with POST method but the user can override this method to handle the request in any way.
