Back to index

Thresholded Confusion Matrix

Matrix
Generates documents
Thresholded Confusion Matrix

Seen as a design

Take it with you

Copy and paste, or save a file and hand it over. For a machine with no network, one “Everything” file is enough.

Everything

confusion-matrix-everything.md

Everything in this panel, including the contents behind each URL, in one Markdown file

Prompt

1.6 KB 
Build an HTML document with "Thresholded Confusion Matrix". This layout runs, so there is no composition to rebuild — what it needs is content.

## What this layout does
Show true and false positive and negative counts at one threshold to assess operational miss and alert costs.

## What you have to supply
- Supply positive and negative classes, one threshold, and all four confusion counts.
- Precision, recall, specificity, and accuracy are derived from cell counts.

## The shape of the input
- `title` — string — 評価結論を伝える見出し
- `subtitle` — string? — 評価データ・期間
- `positiveLabel, negativeLabel` — string — 異なるクラス名
- `threshold` — number — 0〜1の単一分類しきい値
- `counts` — object (tp, fp, fn, tn) — 0以上の安全な整数。合計は1件以上

schema.ts holds the machine-checkable validation. Ask rather than guess when an input is missing.

## When not to use it
- Use an ROC curve to compare performance across thresholds.
- Use a calibration curve to compare predicted probabilities with observed rates.

## How to build it
Put the input in a JSON file and run:

```bash
node scripts/render.mjs confusion-matrix <input.json> --theme ink -o out.html
```

The output is one self-contained file and fetches nothing at render time.

## The source
- https://design-vault.pages.dev/designs/confusion-matrix/source/template.tsx
- https://design-vault.pages.dev/designs/confusion-matrix/source/schema.ts
- https://design-vault.pages.dev/designs/confusion-matrix/source/sample-data.json
- https://design-vault.pages.dev/designs/confusion-matrix/sample.html

Files

Template, input schema, sample data and metadata. This document was generated from those four. The generated document

template.tsx

5.8 KB

/designs/confusion-matrix/source/template.tsx

Show contents 
import type { Props } from "./schema";

const CSS = ".asset{font-family:var(--font-sans);color:var(--color-on-surface);background:var(--color-surface);max-width:var(--container-slide);margin:auto;padding:2rem 3rem;line-height:1.5}.asset *{box-sizing:border-box}.asset h1{font-size:var(--text-h1);line-height:1.3;margin:0}.asset .sub{color:var(--color-secondary);margin:.5rem 0 1rem}.asset .plot{display:block;width:100%;height:auto}.asset .class-definitions{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:.75rem;margin:.75rem 0}.asset .class-definitions div{min-width:0;padding:.75rem;background:var(--color-surface-subtle);border-top:2px solid var(--color-tertiary)}.asset .class-definitions dt{color:var(--color-secondary);font-size:var(--text-label)}.asset .class-definitions dd{margin:.25rem 0 0;overflow-wrap:anywhere}.asset .stats{display:grid;grid-template-columns:repeat(4,minmax(0,1fr));gap:.75rem;margin:1rem 0}.asset .stat{padding:.75rem;border-top:3px solid var(--color-tertiary);background:var(--color-surface-subtle)}.asset .stat dt{color:var(--color-secondary);font-size:var(--text-label)}.asset .stat dd{margin:.25rem 0 0;font:700 var(--text-h2) var(--font-mono)}.asset .note{color:var(--color-secondary);font-size:var(--text-label)}@media(max-width:640px){.asset{padding:1rem}.asset h1{font-size:var(--text-h2)}.asset .stats{grid-template-columns:repeat(2,minmax(0,1fr))}}";
const fmt = (n: number) => n.toLocaleString("ja-JP");
const pct = (n: number, denominator: number) => denominator ? (100*n/denominator).toFixed(1)+"%" : "—";

export default function Template(d: Props) {
  const { tp, fp, fn, tn } = d.counts;
  const total = tp + fp + fn + tn;
  const cells = [
    { id:"tp", x:360, y:210, count:tp, title:"真陽性", detail:"実際も予測も陽性", tone:"var(--color-tertiary)" },
    { id:"fn", x:660, y:210, count:fn, title:"偽陰性", detail:"見逃し", tone:"var(--color-secondary)" },
    { id:"fp", x:360, y:385, count:fp, title:"偽陽性", detail:"誤検知", tone:"var(--color-secondary)" },
    { id:"tn", x:660, y:385, count:tn, title:"真陰性", detail:"実際も予測も陰性", tone:"var(--color-tertiary)" },
  ];
  return <section className="asset asset-confusion-matrix"><style>{CSS}</style>
    <header><p>CLASSIFICATION / THRESHOLDED COUNTS</p><h1>{d.title}</h1>{d.subtitle && <p className="sub">{d.subtitle}</p>}</header>
    <svg className="plot" data-layout-still="true" width="1280" height="720" viewBox="0 0 1280 720" role="img" aria-label="実際のクラスと予測クラスの混同行列" fontFamily="sans-serif" fill="var(--color-on-surface)">
      <title>{d.title}</title><desc>陽性: {d.positiveLabel}。陰性: {d.negativeLabel}。しきい値{d.threshold}。陽性の再現率{pct(tp,tp+fn)}、偽陽性率{pct(fp,fp+tn)}。評価標本数{fmt(total)}。</desc>
      <rect x="0" y="0" width="1280" height="720" fill="var(--color-surface)"/>
      <text x="74" y="44" fontSize="14" fontWeight="700" letterSpacing="2" fill="var(--color-tertiary)">CONFUSION MATRIX / n={fmt(total)}</text>
      <text x="540" y="105" textAnchor="middle" fontSize="16" fontWeight="700">予測クラス</text>
      <text x="510" y="150" textAnchor="middle" fontSize="15">陽性</text><text x="810" y="150" textAnchor="middle" fontSize="15">陰性</text>
      <text x="250" y="300" textAnchor="middle" fontSize="16" fontWeight="700" transform="rotate(-90 250 300)">実際のクラス</text>
      <text x="310" y="292" textAnchor="end">陽性</text><text x="310" y="467" textAnchor="end">陰性</text>
      {cells.map(cell=><g key={cell.id} data-confusion-cell={cell.id}>
        <rect x={cell.x} y={cell.y} width="280" height="150" rx="4" fill="var(--color-surface)" stroke="var(--color-outline)" strokeWidth="2"/>
        <rect x={cell.x} y={cell.y} width="8" height="150" fill={cell.tone}/>
        <text x={cell.x+28} y={cell.y+35} fontSize="14" fill="var(--color-secondary)">{cell.title} · {cell.detail}</text>
        <text x={cell.x+28} y={cell.y+100} fontSize="38" fontWeight="700" fill={cell.tone}>{fmt(cell.count)}</text>
        <text x={cell.x+28} y={cell.y+132} fontSize="15" fill="var(--color-secondary)">{pct(cell.count,total)}</text>
      </g>)}
      <rect x="990" y="165" width="220" height="370" rx="4" fill="var(--color-surface)" stroke="var(--color-outline)"/>
      <text x="1016" y="205" fontSize="16" fontWeight="700">評価条件</text><text x="1016" y="250">しきい値</text><text x="1016" y="294" fontSize="25" fontWeight="700" fill="var(--color-tertiary)">{d.threshold}</text>
      <text x="1016" y="340">評価標本</text><text x="1016" y="384" fontSize="22" fontWeight="700">{fmt(total)} 件</text>
      <text x="1016" y="430">陽性判定の指標</text><text x="1016" y="462">再現率 {pct(tp,tp+fn)}</text><text x="1016" y="492">偽陽性率 {pct(fp,fp+tn)}</text>
      <text x="74" y="590" fontWeight="700">読み方</text><text x="74" y="625">偽陰性は見逃し、偽陽性は誤検知。件数を運用コストと照らしてしきい値を決める。</text><text x="74" y="660" fill="var(--color-secondary)">1つのしきい値における結果。全しきい値の性能比較にはROC曲線を使う。</text>
    </svg>
    <dl className="class-definitions"><div><dt>陽性クラス</dt><dd>{d.positiveLabel}</dd></div><div><dt>陰性クラス</dt><dd>{d.negativeLabel}</dd></div></dl>
    <dl className="stats"><div className="stat"><dt>適合率（TP / 予測陽性）</dt><dd>{pct(tp,tp+fp)}</dd></div><div className="stat"><dt>再現率（TP / 実陽性）</dt><dd>{pct(tp,tp+fn)}</dd></div><div className="stat"><dt>特異度（TN / 実陰性）</dt><dd>{pct(tn,tn+fp)}</dd></div><div className="stat"><dt>正解率</dt><dd>{pct(tp+tn,total)}</dd></div></dl>
    <p className="note">ゼロ除算となる率は「—」で表示。率だけでなく4セルの実件数も判断材料として確認する。</p>
  </section>;
}

schema.ts

1.5 KB

/designs/confusion-matrix/source/schema.ts

Show contents 
import { z } from "zod";

const text = z.string().trim().min(1).max(160);
const label = z.string().trim().min(1).max(64);
const count = z.number().int().nonnegative().safe().describe("混同行列の観測件数");

export const schema = z.object({
  title: text.describe("分類器の評価結果を伝える見出し"),
  subtitle: text.optional().describe("評価データ・期間・分割条件"),
  positiveLabel: label.describe("陽性の実クラスおよび予測クラス名"),
  negativeLabel: label.describe("陰性の実クラスおよび予測クラス名"),
  threshold: z.number().finite().min(0).max(1).describe("陽性判定に使った分類しきい値"),
  counts: z.object({
    tp: count.describe("真陽性: 実際も予測も陽性"),
    fp: count.describe("偽陽性: 実際は陰性だが予測は陽性"),
    fn: count.describe("偽陰性: 実際は陽性だが予測は陰性"),
    tn: count.describe("真陰性: 実際も予測も陰性"),
  }).strict().describe("単一しきい値での実クラス×予測クラスの4セル件数"),
}).strict().superRefine((data, ctx) => {
  if (data.positiveLabel === data.negativeLabel) ctx.addIssue({ code: "custom", path: ["negativeLabel"], message: "陽性・陰性ラベルは異なる必要がある" });
  if (data.counts.tp + data.counts.fp + data.counts.fn + data.counts.tn === 0) {
    ctx.addIssue({ code: "custom", path: ["counts"], message: "評価標本数は1件以上" });
  }
});

export type Props = z.infer<typeof schema>;

meta.yaml

2.1 KB

/designs/confusion-matrix/source/meta.yaml

Show contents 
id: confusion-matrix
name:
  ja: しきい値固定の混同行列
  en: Thresholded Confusion Matrix
purpose:
  ja: 単一しきい値での真陽性・偽陽性・偽陰性・真陰性の実件数を示し、見逃しと誤検知の運用影響を比較する。
  en: Show true and false positive and negative counts at one threshold to assess operational miss and alert costs.
status: published
still: ./still.png
category: analysis
catalogOrder: 980
jobs:
  - prioritize-diagnose
  - measure-change
useCases:
  - prioritization
  - kpi
layoutKinds:
  - matrix
tags:
  - confusion-matrix
  - classification-evaluation
  - offline
  - responsive
whenToUse:
  - ja: 固定した分類しきい値で、正しい判定と2種類の誤りの件数を確認する。
    en: Use to inspect correct predictions and both error types at a fixed threshold.
  - ja: 見逃しと誤検知を、実件数と運用コストに照らして判断する。
    en: Compare actual miss and alert counts against their operational costs.
avoidWhen:
  - ja: しきい値を動かしながら性能を比較する場合はROC曲線を使う。
    en: Use an ROC curve to compare performance across thresholds.
  - ja: 予測確率と実際の発生率の対応を見る場合は校正曲線を使う。
    en: Use a calibration curve to compare predicted probabilities with observed rates.
requiredInputs:
  - ja: 陽性・陰性クラス、しきい値、混同行列の4実件数を指定する。
    en: Supply positive and negative classes, one threshold, and all four confusion counts.
  - ja: 適合率・再現率・特異度・正解率はセル件数から導出する。
    en: Precision, recall, specificity, and accuracy are derived from cell counts.
relatedLayouts:
  - roc-curve
  - calibration-curve
dataSchema:
  title: string — 評価結論を伝える見出し
  subtitle: string? — 評価データ・期間
  positiveLabel, negativeLabel: string — 異なるクラス名
  threshold: number — 0〜1の単一分類しきい値
  counts: object (tp, fp, fn, tn) — 0以上の安全な整数。合計は1件以上
sources: []

sample-data.json

302 B

/designs/confusion-matrix/source/sample-data.json

Show contents 
{
  "title": "重大障害の見逃しと誤検知をしきい値別に確認する",
  "subtitle": "架空サンプル｜直近1か月の時間分割検証",
  "positiveLabel": "重大障害",
  "negativeLabel": "通常",
  "threshold": 0.72,
  "counts": { "tp": 86, "fp": 24, "fn": 18, "tn": 472 }
}