"Covid-19 AI demo in all-Docker" deployment including dockerised Flask, FastAPI, Tensorflow Serving and HA Proxy etc etc.
Full documentation is pulished here at Deploy ML/DL models into a consolidated AI demo service stack
As a jump start, we can simply use docker-compose to deploy the following dockerised components into an AWS Ubuntu server
- HAProxy - load balancer
- Gunicorn vs. Univorn - web gateway server
- Flask vs. FastAPI - application server
- Tensorflow-Serving vs. Tensorflow-Serving-gpu - application back-end servers for image etc classifications etc
- IRIS IntegratedML - consolidated App+DB AutoML with SQL interface
- Python3 in Jupyter Notebook to emulate a client for benchmarking
- Docker and docker-compose
- AWS Ubuntu 16.04 with a Tesla T4 GPU
Note: Tensorflow Serving with GPU is for demo purpose only - you can simply switch off the gpu related image (in a dockerfile) and the config (in the docker-compose.yml).
Out of scope or on next wish list:
- Nginx or Apache etc web servers are omitted in demo for now
- RabbitMQ and Redis - queue broker for reliable messaging that can be replace by IRIS or Ensemble.
- IAM (Intersystems API Manger) or Kong is on wish list
- SAM (Intersystems System Alert & Monitoring)
- ICM (Intersystems Cloud Manager) with Kubernetes Operator - always one of my favorites since its birth
- FHIR (Intesystems IRIS based FHIR R4 server and FHIR Sandbox for SMART on FHIR apps)
- CI/CD devop tools or Github Actions
Volumes Mapping & Directory Structure
Please refer to full documentation on section "Dockerised Components"
docker-compose up -d
docker-compose up --scale fastapi=2 --scale flask=2 -d
Sample application on AWS. Note: this service is on a temp AWS address and not up 24/07.
Functional Testing and API docs:
Please see section "2. Test demo APIs" within full documentation on section "Dockerised Components"
Please see section "3. Benchmark-test demo APIs" within full documentation on section "Dockerised Components"
Full documentation is pulished here
Tensorflow exported models are ommited in the models directory, since they are ~250M (larger than 100M by Github). I will upload these large files seperately.
The models are exported from Jupyter pipelines at here and here