Using the ML Model Base Class

Using Flask to deploy an ML model

Introduction

In previous blog posts I showed how to build a simple base class for abstracting machine learning models and how to create a python package that makes use of the base class. In this blog post I aim to use the ideas from the previous blog posts to build a simple application that uses the MLModel base class to deploy a model. I will be using the iris_model package built in this blog post.

When creating software, interacting with a component through an abstraction makes the code easier to understand and evolve.

By interacting with machine learning models through the MLModel abstraction, it becomes possible to build applications that can host any model that implements the MLModel interface. This way, simple model deployments become much faster since a custom-made application is not needed to put a model into production. The application I will show in this blog post takes advantage of this fact to allow a software engineer to install and deploy any number of models that implement the MLModel base class inside a web application.

Flask Web Application

One of the simplest ways to build a web application with python is to use the Flask framework. Flask makes it easy to set up a simple web application that serves web pages and a RESTful interface.

- model_service
- static ( folder containing the static web assets )
- templates ( folder for the html templates
- __init__.py
- config.py
- endpoints.py
- model_manager.py
- schemas.py
- views.py
- scripts ( folder containing scripts )
- tests ( folder containing the unit test suite)
- requirements.txt
- test_requirements.txt
app = Flask(__name__)
if os.environ.get(“APP_SETTINGS”) is not None:
app.config.from_object(os.environ[‘APP_SETTINGS’])
bootstrap = Bootstrap(app)

Model Manager Class

In order to use the iris_model class within the Flask application we are building, we need to have a way to manage the model object within the Python process. To do this we will create a ModelManager class that follows the singleton pattern. The ModelManager class will be instantiated one time at application startup. The ModelManager singleton instantiates MLModel classes from configuration, and returns information about the model objects being managed as well as references to the model objects.

class ModelManager(object):
_models = []
@classmethod
def load_models(cls, configuration):
for c in configuration:
model_module = importlib.import_module(c[“module_name”])
model_class = getattr(model_module, c[“class_name”])
model_object = model_class()

if isinstance(model_object, MLModel) is False:
raise ValueError(“The ModelManager can only hold references to objects of type MLModel.”)
cls._models.append(model_object)
pip install git+https://github.com/schmidtbri/ml-model-abc-improvements
>>> from model_service.model_manager import ModelManager>>> model_manager = ModelManager()>>> model_manager.load_models(configuration=[{“module_name”: “iris_model.iris_predict”,”class_name”: “IrisModel”}])>>> model_manager.get_models()[{‘display_name’: ‘Iris Model’, ‘qualified_name’: ‘iris_model’, ‘description’: ‘A machine learning model for predicting the species of a flower based on its measurements.’, ‘major_version’: 0, ‘minor_version’: 1}]
@app.before_first_request
def instantiate_model_manager():
model_manager = ModelManager()
model_manager.load_models(configuration=app.config[“MODELS”])

Flask REST Endpoints

To make use of the models hosted in the ModelManager object, we will first build a simple REST interface that will allow clients to find and make predictions. To define the data models that are returned by the REST interface we make use of the marshmallow schema package. Although it’s not strictly necessary to use it to build a web app, the marshmallow package provides a simple and quick way to build schemas and do serialization and deserialization.

@app.route(“/api/models”, methods=[‘GET’])
def get_models():
model_manager = ModelManager()
models = model_manager.get_models()
response_data = model_collection_schema.dumps(dict(models=models)).data
return response_data, 200

Flask Views

The Flask framework is also able to render web pages using Jinja templates, a great guide for learning about this can be found here. To add webpages rendered with Jinja templates to the web application I added the templates folder to the application package. In it I created the base html template, from which other templates inherit. The base template uses styles from the bootstrap package. To render the templates into views I also added the views.py module.

@app.route(‘/’, methods=[‘GET’])
def index():
model_manager = ModelManager()
models = model_manager.get_models()
return render_template(‘index.html’, models=models)
The index page of the web app.
The metadata page of the web app.

Dynamic Web Form

The last webpage of the application makes use of a view to render a webpage and the predict endpoint. The prediction web page for a model renders a dynamic form from the input json schema provided by the model, then accepts user input and sends it to the prediction REST endpoint when the user presses the “Predict” button, lastly it displays the prediction results from the model.

@app.route(“/models/<qualified_name>/predict”, methods=[‘GET’])
def display_form(qualified_name):
model_manager = ModelManager()
model_metadata = model_manager.get_model_metadata(qualified_name=qualified_name)
return render_template(‘predict.html’, model_metadata=model_metadata)
$(document).ready(function() {
$.ajax({
url: ‘/api/models/{{ model_metadata.qualified_name }}/metadata’,
success: function(data) {
var container = document.getElementById(‘prediction_form’);
var BrutusinForms = brutusin[“json-forms”];
bf = BrutusinForms.create(data.input_schema);
bf.render(container);
}
The prediction page of the web app

Documentation

To make the REST API easier to use we will produce documentation for it. A common way to document RESTful interfaces is the OpenAPI specification. In order to automatically create an OpenAPI document for the RESTful API that the model service provides, I used the python apispec package. The apispec package is able to automatically extract schema information from marshmallow Schema classes, and is able to extract endpoint specifications from Flask @app.route decorated functions.

spec = APISpec(
openapi_version=”3.0.2",
title=’Model Service’,
version=’0.1.0',
info=dict(description=__doc__),
plugins=[FlaskPlugin(), MarshmallowPlugin()],
)
spec.components.schema(“ModelSchema”, schema=ModelSchema)
spec.components.schema(“ModelCollectionSchema”, schema=ModelCollectionSchema)
spec.components.schema(“JsonSchemaProperty”, schema=JsonSchemaProperty)
spec.components.schema(“JSONSchema”, schema=JSONSchema)
spec.components.schema(“ModelMetadataSchema”, schema=ModelMetadataSchema)
spec.components.schema(“ErrorSchema”, schema=ErrorSchema)
with app.test_request_context():
spec.path(view=get_models)
spec.path(view=get_metadata)
spec.path(view=predict)
Swagger UI view of the OpenAPI specification created by the openapi.py script.

Conclusion

In this blog post I showed how to create a web application that is able to host any model that inherits from and follows the standards of the MLModel base class. By using an abstraction to deal with machine learning model code, it becomes possible to write an application that can deploy any model, instead of building applications that can deploy only one ML model.

Coder and machine learning enthusiast

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store