MLOps Architecture Builder & Cheatsheet

psychologyModel Layer

A simple feed-forward network defined in PyTorch. The model's `state_dict` is saved for deployment.

model_setup.py

import torch
import torch.nn as nn
class SimpleNet(nn.Module):
    def __init__(self):
        super(SimpleNet, self).__init__()
        self.linear = nn.Linear(10, 1)
    def forward(self, x): return self.linear(x)
model = SimpleNet()
torch.save(model.state_dict(), "pytorch_model.pth")

layersServing Stack Layer

Use a high-performance framework like FastAPI for a custom server. For dedicated solutions, TorchServe is the native choice, while Kubeflow KServe, Ray Serve, and NVIDIA Triton offer powerful, managed abstractions.

cloud_queueOrchestration Layer

Package the application with a multi-stage Dockerfile and define its runtime with Kubernetes Deployment, Service, and HPA objects. Managed platforms like Vertex AI abstract this away.

memoryHardware Layer

CPUs: Suitable for small networks. GPUs: Essential for deep learning models. TPUs: Best for massive-scale inference on GCP.

psychologyModel Layer

A simple Keras model saved in TensorFlow's `SavedModel` format, which bundles the architecture and weights.

model_setup.py

import tensorflow as tf
model = tf.keras.Sequential([
    tf.keras.layers.Dense(10, activation='relu', input_shape=(10,)),
    tf.keras.layers.Dense(1)
])
model.save("tf_saved_model")

layersServing Stack Layer

TF Serving and Kubeflow KServe offer native, high-performance support for the `SavedModel` format. NVIDIA Triton is also highly optimized for TF models. A custom FastAPI server is another flexible option.

cloud_queueOrchestration Layer

The Kubernetes configuration is very similar to other frameworks. Ensure your Dockerfile copies the entire `tf_saved_model` directory.

memoryHardware Layer

CPUs: Good for smaller Keras models. GPUs: Highly recommended for deep learning models. TPUs: The premier choice for running TensorFlow models at scale on GCP.

psychologyModel Layer

A classic logistic regression model. Serialization is typically done with `joblib` for efficiency with NumPy structures.

model_setup.py

import joblib
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import make_classification
X, y = make_classification(n_features=4)
model = LogisticRegression().fit(X, y)
joblib.dump(model, "sklearn_model.joblib")

layersServing Stack Layer

FastAPI provides a simple and fast web server. Kubeflow KServe and Ray Serve also have native support for scikit-learn models. NVIDIA Triton is an option for CPU-optimized execution using its FIL backend.

cloud_queueOrchestration Layer

Standard Kubernetes setup. The Docker container will be lightweight as it only needs `scikit-learn`, `joblib`, and `fastapi` for a custom server.

memoryHardware Layer

CPUs: Almost always sufficient. There is no GPU acceleration for standard scikit-learn algorithms.

psychologyModel Layer

An XGBoost model saved in its native JSON or UBJ format, which is portable and efficient.

layersServing Stack Layer

Kubeflow KServe, Ray Serve, NVIDIA Triton (with FIL backend), and custom FastAPI servers are all excellent choices.

cloud_queueOrchestration Layer

Standard Kubernetes setup. The Dockerfile should include the `xgboost` library.

memoryHardware Layer

CPUs: Excellent performance. GPUs: XGBoost has optional GPU acceleration which can provide a significant speedup.

psychologyModel Layer

JAX models are often defined as pure functions with parameters handled separately. We save the parameters using a standard serialization library like Flax's `msgpack`.

layersServing Stack Layer

Ray Serve is an excellent fit for JAX's functional paradigm. A custom FastAPI server is also straightforward. Kubeflow KServe and NVIDIA Triton require a custom container approach wrapping the JAX logic.

cloud_queueOrchestration Layer

The Dockerfile needs to install `jax` and `jaxlib` corresponding to the target hardware (CPU, GPU, or TPU).

memoryHardware Layer

CPUs/GPUs/TPUs: JAX was designed for accelerators and excels on all of them due to its XLA-based compilation.

psychologyModel Layer

Large Language Models (e.g., Llama, Mistral) are based on the Transformer architecture. The key inference challenge is managing the KV Cache.

layersServing Stack Layer

Specialized serving toolkits like vLLM, SGLang, or NVIDIA Triton with its TensorRT-LLM backend are required for efficient inference, handling complexities like continuous batching and paged attention.

cloud_queueOrchestration Layer

Kubernetes (often with KubeRay) is used to manage GPU resources and schedule serving pods. Managed services like Vertex AI and SageMaker also provide optimized runtimes for popular LLMs.

memoryHardware Layer

GPUs: Essential. High-VRAM GPUs like NVIDIA A100 or H100 are required to fit the model weights and KV cache. TPUs: Viable for specific models, especially on GCP.

psychologyModel Layer

Visual Large Models (e.g., LLaVA, IDEFICS) combine a vision encoder (like ViT) with an LLM to process images and text.

layersServing Stack Layer

The stack must handle multi-modal inputs. Frameworks like vLLM and SGLang are adding native support for VLMs. A custom container is often needed to handle the specific image preprocessing logic.

cloud_queueOrchestration Layer

Similar to LLMs, requires robust orchestration to manage high-resource GPU pods and potentially large input payloads.

memoryHardware Layer

GPUs: High-VRAM GPUs are mandatory due to the combined size of the vision encoder, LLM, and KV cache.

psychologyModel Layer

Diffusion models (e.g., Stable Diffusion) generate images through an iterative denoising process, making latency a key challenge.

layersServing Stack Layer

Optimizations focus on reducing latency. Key tools include model compilers like TensorRT (often used with NVIDIA Triton), techniques like Latent Consistency Models (LCMs), and libraries like Diffusers, typically wrapped in a custom FastAPI container.

cloud_queueOrchestration Layer

Kubernetes or managed platforms are used to serve the GPU-intensive workload. Autoscaling is critical to handle bursty traffic patterns.

memoryHardware Layer

GPUs: High-end consumer or datacenter GPUs are needed for acceptable generation speeds. VRAM is the most critical resource, dictating max resolution and batch size.