Operating Model on Systems & Engineering

Discovery And Delivery

Mon, 01 Jan 0001 00:00:00 +0000

Discovery and delivery are two kinds of work. They are distinguished by their main purpose, not by when they happen.

Evaluation And Production Learning

Mon, 01 Jan 0001 00:00:00 +0000

Status: Deferred note. Evaluation is kept separate from the current operating model and is not developed here yet.

7. Evals stop being “AI tests”

In a classical architecture, evals are often conceived as:

Operating Model

Mon, 01 Jan 0001 00:00:00 +0000

Principles and Decision Logic for AI Product Work

Purpose

This document explains how to reason about product work when we do not yet know which solution will work or how well it will behave.

Production Commitment

Mon, 01 Jan 0001 00:00:00 +0000

Production commitment: what users can rely on

A production commitment states what people may rely on in real use and what we accept responsibility for.

Reference Example Is Not Ground Truth

Mon, 01 Jan 0001 00:00:00 +0000

A reference example is not ground truth

An eval uses selected cases, criteria, and a person or system to examine a stated claim. It can provide useful evidence. It does not decide what a good product is.

Solution Utility Ladder

Mon, 01 Jan 0001 00:00:00 +0000

A solution utility ladder connects basic system behavior to a useful result for the user and to the change the solution is meant to produce.