Engineering notes

ENGINEERING NOTE 09

Test the Pointer Path and the One-Point Boundary

Playwright drives visible learner controls across success and failure bands, then proves that 91 and 92 scored answers remain different even when pretest performance reverses.

Published 11 minute read
  • 8Practice scenarios — The pointer suite spans 0%, 40%, 70%, 77%, 80%, 85%, 90%, and 100% outcomes.
  • 91 ↔ 92Full-form boundary — Two 140-item browser runs prove the exact scored-answer transition.
  • 10 itemsPretest reversal — All pretests correct cannot rescue 91; all pretests wrong cannot defeat 92.

SECTION 01

Drive the same controls a learner can see

Practice E2E scenarios change state through visible radio controls, answer cards, buttons, and links instead of mutating component state or calling internal functions.

Each practice scenario begins on the public practice page, confirms the blueprint table, confirms that simulation launch language is absent, and clicks a visible question-count radio. It verifies the selected state and displayed whole-question mix before clicking Start practice. Inside the runner, it clicks the answer card containing the intended option, checks the radio state, clicks Check answer, reads feedback, and advances with Next question or See results.

Role, label, and visible-text locators keep assertions aligned with the accessibility tree. A pointer flow catches defects that direct API checks cannot: a transparent overlay intercepting a click, a label disconnected from its input, a disabled button that never enables, a navigation control with the wrong accessible name, or a result card whose semantic class and visible message disagree. API-level tests still have a role, but they do not replace the user path.

Pointer-driven practice loopThe test follows public controls from setup through feedback to the completed result.The browser test clicks setup controls, starts a real session, selects each visible answer, checks feedback, advances through all items, and validates the final results page. Connections: Choose set to Start practice; Start practice to Select answer; Select answer to Check feedback; Check feedback to Select answer (next item); Check feedback to Inspect result (last item).
  1. Choose set

    Click count and optional filters.

  2. Start practice

    Navigate through the real creation control.

  3. Select answer

    Click the visible answer card and verify radio state.

  4. Check feedback

    Read status and four option rationales.

  5. Inspect result

    Verify score, outcome copy, and review inventory.

SECTION 02

Cover outcome bands, not one happy score

Eight short-set scenarios exercise both sides of the portal's 80% practice target and verify exact score copy, review inventory, and domain allocation.

The scenario matrix ranges from zero correct to perfect and includes the closest sampled values around the 80% practice target: 23/30 produces 77% and review styling, while 24/30 produces 80% and passing styling. Intermediate 70%, 85%, and 90% results prove that the boundary is not accidentally tied to a particular question count. Every run verifies the displayed earned total and percentage.

The suite also accumulates observed domain labels while answering and compares the result with the allocator's expected quota map. This turns a rendering journey into a cross-layer invariant: the learner saw the domain mix promised at setup. Finally, it expects one review card per question and the explicit sentence that 80% is a study goal, not an official score. The failure bands therefore test teaching follow-through, not merely a green or red orbit.

Practice failure-band matrix — The release exercises varied lengths and outcomes; 80% remains an internal practice target.
SetEarnedDisplayedExpected presentation
500%Review
10440%Review
201470%Review
302377%Review
302480%Strong work
201785%Strong work
10990%Strong work
55100%Strong work

SECTION 03

Prove the 91/92 boundary with reversed pretests

Two full 140-item runs isolate the scored threshold from percentage rounding and unscored-item performance.

The simulation test uses a deterministic exported form and creates a session through the same-origin backend endpoint because the learner UI has no simulation launch control. It then drives the real runner: review marking, skip, palette navigation, unmarking, 140 visible option selections, Save and continue, Finish, and the results page. Feedback must remain absent throughout the active form.

In the first run exactly 91 scored questions are answered correctly while every pretest answer is correct; the result must remain below passing. In the second, exactly 92 scored questions are correct while every pretest answer is wrong; the result must pass. The assertions expect 91/130 or 92/130, ten Unscored review labels, exact domain denominators, and the official-report disclaimer. This paired reversal proves that neither rounded 71% display nor pretest correctness enters the decision.

Boundary fixture — Changing pretest correctness in the opposite direction isolates the 92-of-130 scored boundary.typescript
const boundaries = [
  { earned: 91, passed: false, pretestsCorrect: true },
  { earned: 92, passed: true, pretestsCorrect: false },
] as const;

SECTION 04

Be explicit about where the test uses gray-box knowledge

The browser path is user-realistic, while deterministic answer selection and simulation bootstrapping intentionally use repository knowledge to create exact outcomes.

The test maps rendered stems to the authored question bank so it can choose a correct or incorrect option on demand. That makes the score matrix deterministic and detects option-label shuffling defects, but it is not a black-box external audit. Similarly, the full-form suite imports the generated template and calls the session endpoint from page context before continuing through the browser. Naming those seams helps reviewers understand what an assertion does and does not prove.

The strongest strategy combines layers. Unit tests check pure quota allocation, concept diversity, non-mutation, scoring, and content graph integrity. API tests check origin validation and public errors. Pointer E2E checks real labels, clicks, navigation, feedback, responsive layout, and persistence. Remote pve21 E2E checks the deployed container path. No single layer proves psychometric validity, regulator endorsement, network security, or accessibility for every assistive technology; each supplies bounded evidence about an implemented contract.

REFERENCES

Sources and further reading

  1. Playwright best practicesMicrosoft Playwright

    Testing guidance for user-visible locators, isolated expectations, and resilient interaction-oriented browser checks.

  2. LocatorsMicrosoft Playwright

    Reference for role-, label-, and text-driven element discovery used by the pointer-oriented scenarios.

  3. Series 65 Exam Content OutlineNorth American Securities Administrators Association

    Primary reference for the examination name, current scored-domain specification, timing, and passing-score context represented by this release.

KEEP EXPLORING

Related engineering notes