Inside Item Response Theory

An interactive walkthrough of what IRT does, and why it matters
Section 1

What does a test score actually mean?

Suppose three people take a 10-item test and all score 6 out of 10. In Classical Test Theory, they receive the same observed score and the same standard error of measurement. But look at their response patterns:

Response Patterns (✓ = correct, ✗ = incorrect)

Person A answered the easy items correctly and missed the hard ones — a sensible pattern. Person B shows the opposite: hard items correct, easy items wrong. Person C has a scattered pattern with no clear logic. CTT treats all three identically.

IRT asks: are these three people really equivalent? The response pattern matters, not just the sum. A theory that models each item individually can extract information that sum scores discard.
Section 2

Items have personalities

IRT's core idea: each item has its own Item Characteristic Curve (ICC) — an S-shaped function describing the probability of a correct response at each level of the latent trait $\theta$. Unlike CTT, which treats items as interchangeable units in a sum, IRT gives each item its own identity.

The 2-parameter logistic (2PL) model describes each item with two parameters: discrimination ($a$) controls the steepness — how sharply the item separates people near its threshold; difficulty ($b$) locates the curve along the trait axis.

$$P(\theta) = \frac{1}{1 + e^{-a(\theta - b)}}$$
2PL Item Characteristic Curve — probability of correct response at trait level $\theta$
Item Characteristic Curve
a 1.50
b 0.00
θ 0.00
P(θ)
0.500
Try it: Move $\theta$ along the trait axis and watch $P(\theta)$ change. At $\theta = b$, the probability is exactly 0.50. Increase $a$ and the curve steepens — the item becomes a sharper separator. Decrease $a$ toward 0.3 and the curve flattens — the item barely distinguishes anyone.
Section 3

Difficulty shifts, discrimination sharpens

One curve tells you about one item. Multiple curves reveal the test's architecture: where along the trait continuum each item operates, and how sharply each one discriminates.

Four Item Characteristic Curves
Try it: "Vary Difficulty" spreads items across the trait axis — each item targets a different ability level. "Vary Discrimination" keeps them at the same difficulty but changes how sharp each one is. Switch to "Custom" to design your own set.
Items with very low discrimination (a ≈ 0.5 or below — the flattest item in the "Vary Discrimination" set) are nearly flat: they contribute almost no measurement information. In practice, such items would be flagged for removal or revision.
Section 4

What CTT misses

In CTT, a test score is a simple sum: each item contributes one point, equally weighted. In IRT, each item's contribution depends on where on the trait continuum the person falls. An item centered at $\theta = 2$ provides almost no information about someone at $\theta = -2$, even though CTT counts that item the same as any other.

θ 0.00
CTT: Equal Item Weights
IRT: Information at θ = 0.00
CTT says each item contributes equally to the total score. IRT reveals that at any given trait level, some items contribute many times more information than others. The sum score treats a maximally informative item and a useless one the same.
Section 5

Item information

Each item's information function tells you exactly how much measurement precision it provides at each trait level. For the 2PL model, item information has a clean formula:

$$I_i(\theta) = a_i^2 \, P_i(\theta) \, Q_i(\theta)$$
Item information function, where $Q = 1 - P$

This is always a bell-shaped curve, peaking at $\theta = b$ with height $a^2/4$. High discrimination means a tall, narrow peak of information. Low discrimination means a short, wide bump — far less information near its own difficulty, though it holds its small value further out into the tails, where a sharply peaked item has already fallen away to nothing. Integrated across the whole trait range the area under this curve is exactly $a$, so the low-discrimination item does carry less information in total.

ICC and Item Information Function
a 1.50
b 0.00
Peak Info
0.563
Peak at θ
0.0
ICC: P(θ)
Information: I(θ)
Try it: Increase $a$ and watch the information peak grow taller and narrower — the item provides intense precision in a narrow band around its difficulty. Decrease $a$ and the peak flattens — the item spreads weak information over a broad range. Move $b$ to shift where the peak falls.
Section 6

From items to tests: the Test Information Function

A test's total information at any trait level is simply the sum of its items' information functions. This Test Information Function (TIF) is IRT's answer to reliability — but unlike a single reliability coefficient, the TIF tells you where the test is precise and where it isn't.

Test Parameters
Items 10
Avg a 1.50
Avg b 0.00
Spread 1.50
θ 0.00
I(θ)
—
SE(θ)
—
Individual item info
Test Information I(θ)
SE(θ)
Try it: Set Spread to 0 — all items cluster at the same difficulty, creating a sharp information peak. Increase Spread and the TIF broadens but its peak drops. This is the fundamental tradeoff: precision at one point vs. coverage across the range.
For a deeper exploration of $SE(\theta)$ and confidence intervals, see the IRT Precision visualization.
Section 7

The measurement precision illusion

This is the conceptual payoff. Classical Test Theory reports a single Standard Error of Measurement (SEM) for the entire test — it doesn't matter who you're measuring. IRT reveals the truth: measurement precision varies dramatically depending on where on the trait continuum a person falls.

θ 0.00
CTT: Constant SEM
IRT: Conditional SE(θ)
Try it: Select "Peaked Test" and move $\theta$ — the IRT SE stays low near the center but explodes at the extremes, while CTT's SEM stays flat. Select "Clinical Cutoff" — items are targeted around $\theta = 1.5$, so precision is excellent near the cutoff but poor elsewhere. CTT's single SEM hides this completely.
CTT says the test has a single SEM everywhere. IRT reveals that the SEM can range from excellent to useless depending on who you're measuring. For clinical decisions at specific cutoffs, this distinction is critical.
Section 8

Building a better test

IRT doesn't just describe tests — it guides test construction. Since each item's information function is known, you can choose items to shape the TIF to match your measurement goals. Click items in the bank below to add them to your test and watch the TIF update in real time.

Test Information Function (0 items selected)
Clinical screening tests need precision at a specific cutoff. Research instruments need broad coverage. Computerized Adaptive Testing (CAT) selects items in real time to maximize information at the test-taker's estimated ability.
Real test assembly must also consider content coverage, item exposure, and practical constraints. This demonstrates the psychometric logic only.
Section 9

What IRT makes possible

Return to Section 1's puzzle: three people, same sum score, different patterns. IRT doesn't just count correct answers — it uses the full response pattern and each item's parameters to estimate a trait level ($\theta$) and a standard error for each person individually.

Same Sum Score, Different Estimates

All three answered six items correctly — but not the same six. IRT weights each item by its discrimination, so what drives the estimate is not the raw count but the discrimination-weighted sum $\sum a_i u_i$: 10.1 for Person A, 7.7 for Person B, 5.9 for Person C. Person A's correct answers land on the sharply discriminating items (a = 2.4, 2.2, 2.3, 2.1), while Person C's land mostly on the weak ones (a = 0.5, 0.6, 0.7) — and their estimates end up more than a point and a half apart, from the same 6/10.

Now notice what the standard errors do not do. They do not track how orderly the pattern looks. Person B's reversed pattern is the most improbable of the three, yet it carries the smallest SE. Under the 2PL, test information $\sum a_i^2 P_i Q_i$ depends only on where $\hat\theta$ lands — the responses $u_i$ drop out of the second derivative entirely — so an aberrant pattern cannot widen the interval by itself. Catching genuinely odd responding takes a dedicated person-fit statistic ($l_z$, infit/outfit), not the standard error.

IRT doesn't just change how we think about measurement. It changes what's possible: adaptive testing that adjusts difficulty in real time, fair comparison across different test forms (equating), detection of items that function differently across groups (DIF), and honest reporting of where a test is precise and where it isn't.
This visualization uses the 2PL model throughout. The 3PL adds a guessing parameter, the GRM and GPCM handle polytomous (Likert-type) items, and multidimensional models allow multiple latent traits. The 2PL captures the core logic; extensions add flexibility for specific applications.
Citation

Persson, B. N. (2026). Inside Item Response Theory [Interactive visualization]. https://bjorn-persson.github.io/visualizations/irt-curve/

@misc{Persson2026irt,
  author = {Björn N. Persson},
  year = {2026},
  title = {Inside Item Response Theory},
  note = {Interactive visualization},
  url = {https://bjorn-persson.github.io/visualizations/irt-curve/}}