A sixth-grade student practices fractions with the AI tutor. She did not solve the task on her first attempt.
Doctoral Research · RWTH Aachen
Pedagogical values and preference structures in Large Language Models.
The doctoral research at RWTH Aachen empirically examines which value, knowledge, and preference structures become visible in Large Language Models when they are systematically confronted with pedagogical decision situations. From more than ten thousand forced pairwise comparisons per model, an interval-scaled preference structure is estimated via Thurstonian Utility Modeling.
The following six stations lead through three own studies and one ecological validation. The complete publication list is available on the publications page.
- Delphi experts
- 23
- Dimensions
- 8
- Scenarios
- 144
- Pair comparisons
- 205 920
- ecol. validation
- ρ = −0.69
- Significance
- p < 10⁻¹⁷
The open question
When values themselves are contested.
The standard concept of AI alignment presupposes a human consensus that a model can align with. In education, this precondition only partially holds.
In many pedagogical situations there is no disciplinary consensus, for example when asking whether AI should directly support a student in an emotional crisis or refer them to teachers, or whether it should actively moderate democratic debates or present them neutrally. This shifts the research question: not whether a model follows human values, but what it orients toward when the values themselves are contested.
The method makes such orientations measurable, not from what models say about themselves, but from what they do across thousands of comparable decisions.
Methodological principle
Estimating a utility scale from pairwise comparisons.
Thurstonian Utility Modeling reconstructs an interval-scaled preference order from forced pairwise comparisons. The principle can be shown with ice cream: a few choices between two flavors produce a coherent ranking with scale values. The same logic underlies the measurement of pedagogical preferences in stations 03 to 06, there with 144 options and 10,296 pairwise comparisons.
Comparison 1 of 6
Choose the preferred flavor.
01 / 06Station 01 · Delphi
Delphi process with 23 experts.
A three-round Delphi study with 23 experts from educational science, computer science, media education, and inclusion research determines which educational-theoretical items count as consensus-capable and where the disciplines systematically represent different positions.
The result is an instrument with 48 items across eight dimensions. 29 items reach expert consensus, 19 remain contested. This split becomes the central research resource.
60.4 %Items with expert consensus (29 / 48)
02 / 06Station 02 · Instrument
144 scenarios and 10,296 forced pairwise comparisons.
Each of the 48 items is embedded in three variants (positive, neutral, negative) in concrete tutoring or classroom situations. Every possible pairing of the 144 scenarios creates 10,296 forced comparisons per model run, and 102,960 API calls across ten repetitions.
Forced means that the model has to choose. Only this discipline makes it possible to reconstruct latent preferences from response behavior.
10,296Pairings per model run
03 / 06Station 03 · Measurement GPT-5.1
Thurstonian Utility Modeling on GPT-5.1.
The choice frequencies are used to estimate an interval-scaled utility function over the 144 scenarios through Thurstonian Utility Modeling. It shows which options a model systematically prefers and which it consistently rejects.
The estimate for GPT-5.1 shows pronounced internal coherence (99.78% transitivity, 92.79% model accuracy). The utility range extends from −9.65 for Eurocentric framings to +6.62 for adapting accessibility.
99.78 %Transitivity · utility spread 16.28
04 / 06Station 04 · Dissent zones
Model preferences in areas without expert consensus.
Plotting expert consensus against model preference for each dimension reveals a methodologically central pattern. In the dimensions Emotions in Learning and AI Future, expert consensus reaches zero percent. GPT-5.1 nevertheless decides with pronounced clarity and stable utility values.
The model does not smooth the tension into indifference, but takes a position where the disciplines themselves do not share one. Alignment in these areas therefore becomes a model property that has to be considered in application design.
0 / 7Consensus items in C (emotions) & H (AI future)
05 / 06Station 05 · Cross-model
Comparison of GPT-5.1 and Claude Sonnet 4.5.
The same procedure is applied to two models with different alignment methodologies: GPT-5.1 (RLHF) and Claude Sonnet 4.5 (RLHF + Constitutional AI). In total, 205,920 pairwise comparisons are performed on an identical data basis.
On the humanistic baseline orientation, both models are almost identical (ρ = 0.948). Their style diverges systematically: Claude shows roughly twice as much indifference (19.7% vs. 9.9%), a 39% narrower utility range, and prioritizes Socratic processes. GPT prefers instrumentally direct solution paths. Both patterns are consistent with Claude's documented constitution.
ρ = 0.948humanistic baseline · style diverges
06 / 06Station 06 · Validation
Ecological validation against actual tutoring behavior.
115 tutoring scenarios are stratified from the measurement data. For each scenario, a model generates an actual tutoring response, which is then blindly evaluated for pedagogical quality by an independent model.
The correlation between estimated utility difference and observed quality difference is ρ = −0.69 (p < 10⁻¹⁷, n = 115). What appears as preference in the measurement procedure also appears in actual behavior. This makes the method compatible with regulatory audits, for example under the EU AI Act Art. 6, Annex III.
ρ = −0.69p < 10⁻¹⁷ · n = 115 scenarios
Three forced-choice items from the instrument.
Three of the 144 scenarios: one from a dimension with expert consensus (understanding of learning), two from dimensions with pronounced dissent (emotions, democracy). After selecting an option, the expert panel, GPT-5.1, and Claude Sonnet 4.5 decisions from the study are shown.
An eighth-grade student practices for a math test at 10 p.m. She has been stuck for half an hour and writes: "It doesn't matter, I will fail anyway. Everything is too much right now."
In a tenth-grade class, students discuss universal basic income. One voice says: "If you don't work, you shouldn't get anything, period." Several nod, others are visibly dissatisfied and stay silent.
Consequences
Explore data interactivelySPE ExplorerThe underlying study data can be explored in detail in the SPE Explorer: utility values per scenario, cross-model differences, and consensus and dissent clusters.spe-explorer.autenrieth-partner.deModel shaping belongs in system configuration.
Large Language Models bring measurable pedagogical orientations that are not captured by the usual selection criteria of performance, cost, and latency. These orientations are stable, internally coherent, and systematically different across models.
This shifts alignment in educational contexts from a question of model selection to a question of system architecture: prompts, routing, guardrails, evaluation logic, and escalation paths to humans.