PythonServer¶

class lightning.app.components.serve.python_server.PythonServer(input_type=<class 'lightning.app.components.serve.python_server._DefaultInputData'>, output_type=<class 'lightning.app.components.serve.python_server._DefaultOutputData'>, **kwargs)¶

Bases: LightningWork, ABC

The PythonServer Class enables to easily get your machine learning server up and running.

Parameters

input_type¶ (type) –
Optional input_type to be provided. This needs to be a pydantic BaseModel class. The default data type is good enough for the basic usecases and it expects the data to be a json object that has one key called payload
```
input_data = {"payload": "some data"}
```
and this can be accessed as request.payload in the predict method.
```
def predict(self, request):
    data = request.payload
```
output_type¶ (type) –
Optional output_type to be provided. This needs to be a pydantic BaseModel class. The default data type is good enough for the basic usecases. It expects the return value of the predict method to be a dictionary with one key called prediction.
```
def predict(self, request):
    # some code
    return {"prediction": "some data"}
```
and this can be accessed as response.json()[“prediction”] in the client if you are using requests library

Example

>>> from lightning.app.components.serve.python_server import PythonServer
>>> from lightning.app import LightningApp
...
>>> class SimpleServer(PythonServer):
...
...     def setup(self):
...         self._model = lambda x: x + " " + x
...
...     def predict(self, request):
...         return {"prediction": self._model(request.image)}
...
>>> app = LightningApp(SimpleServer())

configure_layout()¶

Configure the UI of this LightningWork.

You can either :rtype: Optional[Frontend]

Return a single Frontend object to serve a user interface for this Work.
Return a string containing a URL to act as the user interface for this Work.
Return None to indicate that this Work doesn’t currently have a user interface.

Example: Serve a static directory (with at least a file index.html inside).

from lightning.app.frontend import StaticWebFrontend


class Work(LightningWork):
    def configure_layout(self):
        return StaticWebFrontend("path/to/folder/to/serve")

Example: Arrange the UI of my children in tabs (default UI by Lightning).

class Work(LightningWork):
    def configure_layout(self):
        return [
            dict(name="First Tab", content=self.child0),
            dict(name="Second Tab", content=self.child1),
            dict(name="Lightning", content="https://lightning.ai"),
        ]

If you don’t implement configure_layout, Lightning will use self.url.

Note

This hook gets called at the time of app creation and then again as part of the loop. If desired, a returned URL can depend on the state. This is not the case if the work returns a Frontend. These need to be provided at the time of app creation in order for the runtime to start the server.

abstract predict(request)¶

This method is called when a request is made to the server.

This method must be overriden by the user with the prediction logic. The pre/post processing, actual prediction using the model(s) etc goes here

Return type: Any

run(*args, **kwargs)¶

Run method takes care of configuring and setting up a FastAPI server behind the scenes.

Normally, you don’t need to override this method.

Return type: Any

setup(*args, **kwargs)¶

This method is called before the server starts. Override this if you need to download the model or initialize the weights, setting up pipelines etc.

Note that this will be called exactly once on every work machines. So if you have multiple machines for serving, this will be called on each of them.

Return type: None