Why This Playbook?

Generative AI, RAG, and Agentic AI are useful patterns, but not every AI problem starts with prompting an LLM.

Sometimes the right approach is a classic predictive model trained on labeled examples.

This playbook demonstrates that model engineering workflow with a small Node.js demo: given software change metadata, predict whether a change is low, medium, or high risk.

Repos:

Concept Primer: What Is Model Engineering?

Model engineering is the lifecycle around building and operating predictive models:

  1. Prepare labeled data.
  2. Engineer features.
  3. Train a model.
  4. Evaluate model behavior.
  5. Save a model artifact.
  6. Serve predictions through an API or application.
  7. Monitor and improve the model over time.

This differs from prompt-driven LLM workflows because the model learns from examples before prediction time.

Demo Scope

The demo predicts software delivery risk for a code change.

Inputs include:

  • files changed
  • lines added/deleted
  • test coverage percentage
  • complexity score
  • previous defect count
  • service criticality
  • database change flag
  • security change flag

Output:

{
  "risk": "high",
  "confidence": 0.9,
  "reasons": [
    "large change size",
    "low test coverage",
    "high complexity",
    "database/schema change",
    "security-sensitive change"
  ]
}

What It Demonstrates

  • training data with known labels
  • feature engineering from raw change metadata
  • model training
  • model evaluation
  • saved model artifacts
  • prediction API
  • browser demo
  • scenario runner and reports
  • manual implementation plus framework-backed v2

Manual vs Framework-Backed v2

The repo includes two model paths.

Manual path:

training examples -> feature vectors -> low/medium/high centroids -> nearest centroid prediction

This path is intentionally simple so the mechanics are easy to inspect.

Framework-backed v2:

training examples -> feature vectors -> ml-cart decision tree -> class prediction

The ml-cart v2 path keeps the same input fields, API shape, scenario runner, and UI flow, but replaces the manual centroid classifier with a decision tree model.

Flow

Training Data
    |
    v
Feature Engineering
    |
    v
Train Model -> Evaluate -> Save Artifact
    |
    v
Prediction API -> Browser UI / Scenario Runner
    |
    v
Risk + Confidence + Reasons

How Browser Prediction Works

The browser does not scan the source code automatically in this PoC.

The current app predicts risk from supplied change metadata. You can enter the values manually or load a scenario.

In a production version, those fields could be populated automatically from:

  • GitHub pull requests
  • local git diff
  • CI coverage reports
  • code complexity tools
  • Jira/GitHub defect history
  • service catalog metadata

The prediction flow is:

form values
   |
   v
POST /api/predict
   |
   v
feature engineering
   |
   v
model prediction
   |
   v
risk + confidence + reasons

Quickstart

git clone https://github.com/amiya-pattnaik/model-engineering-playbook.git
cd model-engineering-playbook/demo-app
npm install
npm run train
npm run train:v2
npm run dev
# open http://localhost:3000

Run and Evaluate

# manual model
npm run demo:scenarios

# ml-cart v2 decision tree
npm run demo:scenarios:v2

How This Complements the Other Playbooks

Generative AI      -> prompt/context -> structured output
RAG                -> documents -> grounded answer with citations
Agentic AI         -> scenario -> multi-step orchestration
Model Engineering  -> labeled data -> trained model -> prediction

Together, these repos show that AI engineering is not one pattern. The right design depends on whether you need generation, grounding, orchestration, prediction, or a combination of them.

Closing Thought

This playbook is intentionally small, but it covers the core model lifecycle: data, features, training, evaluation, artifact storage, serving, and repeatable scenarios.

That lifecycle is the foundation for taking predictive models from experiment to practical engineering workflows.