Jithin T JosephCTDS

ML-PEST CTDS · 22 October 2026

Simulation and validation status

Review what the synthetic study demonstrated, where estimates failed, and what remains to be validated.

Simulation evaluated · independent clinical validation pending

The frozen CTDS estimator performed well under matching response-model assumptions, and 30 pulses improved average accuracy compared with 20. Broader response models and stress tests also exposed large errors that could pass the current review-readiness rule. CTDS remains an evaluation aid; independently confirm MT before treatment.

No patients or stimulators were involved. These are simulated performance measures, not clinical error rates or a clinical pass/fail assessment.

Study design

127,000Independent synthetic assessment histories
254,000Paired 20- and 30-pulse endpoint records
1,000Replicates per scenario

The stationary robustness grid contained 108 scenarios: true MT 20–90% MSO; true response spreads of 3%, 7% or 15% of MT; 0%, 2% or 5% random-response mixtures; and Gaussian or variance-matched logistic response curves. A further 19 scenarios tested drift, incorrect initial responses, ceilings, starting intensities and output boundaries.

The unchanged production estimator used a default first intensity of 37% MSO and fixed 7% model spread. Synthetic response probabilities were generated independently of the estimator. The paired plans shared their first 20 random draws; early stops were retained. Computational anchors were never counted as delivered pulses.

Definitions and analysis scope

The true MT was the known intensity giving a 50% response probability. A 5% random-response mixture creates a 2.5% low-intensity response floor; it does not mean a 5% overall error rate. “Within ±5 MSO points” differs from “within ±5% of true MT”: for true MT 50, an estimate of 55 differs by 5 MSO points but 10% of the true MT.

The grid was equally weighted and does not represent the prevalence of patient characteristics. The analysis plan was written before this expanded run, informed by an earlier pilot; it was not preregistered. The error margins are descriptive study measures, not validated clinical acceptance thresholds. Only the default 7% estimator spread was evaluated; custom spread settings are outside this study.

20 versus 30 pulses

Gaussian response curve, true spread 7% of MT, no random-response mixture.

Mean absolute error · lower is better
20 pulses0.992 MSO points30 pulses0.787 MSO points
All runs retained; raw estimates, including incomplete or blocked runs where an estimate exists
Outcome20 pulses30 pulses
Mean absolute error (MSO points)0.990.79
Signed bias (MSO points)-0.06-0.03
95th percentile absolute error (MSO points)2.862.24
Within ±5 MSO points · all runs99.60%99.92%
Within ±5% of true MT · all runs97.55%99.55%
Ready for review · all runs97.48%97.52%
Ready AND within ±5 MSO points · all-run yield97.15%97.48%
Within ±5 MSO points · among ready runs99.66%99.97%
Ready with error >10 MSO points · count00

At 30 pulses, the Monte Carlo 95% interval for the within-5-point rate was 99.84%–99.99%. Monte Carlo intervals describe repeated-simulation precision for this design; they are not patient-level MT confidence intervals. Ten extra pulses reduced robustness-grid MAE by 0.301 MSO points (paired interval 0.296–0.306).

Review readiness did not guarantee accuracy.

At 30 pulses, 2,247 of 108,000 robustness runs were marked ready despite errors greater than 5 MSO points. Of these, 144 had errors greater than 10 points: 0.133% of all robustness runs, or 0.140% of ready runs. The largest ready-run error was an underestimation of 40.33 MSO points.

Accuracy across every stationary scenario
30-pulse accuracy across six true thresholds, three response spreads, three lapse mixtures, and Gaussian or logistic curves. Exact values for every cell are supplied in the downloadable study package.
Each cell contains 1,000 histories. High true thresholds, broad response curves and lapses produced the hardest cases. All-run accuracy is shown, without excluding non-ready results.

Explore a recorded simulation

Use the pulse slider to inspect delivered intensity, response and the evolving estimate. These are fixed synthetic study records; viewing them does not add observations to your assessment.

Simulated true MT: 90% MSOFinal estimate: 49.67% MSO · error -40.33 MSO points · marked ready for review
0255075100% MSO1102030Actual delivered pulse number

Blue line: estimated MT · dashed line: true MT · green circles: positive responses · brown squares: negative responses.

Pulse30 / 30
Delivered intensity50% MSO
ResponsePositive
Estimated MT49.67% MSO

This is the largest observed review-ready error, not a representative case. True MT 90; Gaussian response spread 15%; random-response mixture 5%. An early positive at 37% MSO redirected sampling downwards. Two positives in the last ten pulses allowed the run to pass the 0/1/9/10 response-imbalance rule.

Stress-test results

Stress scenarios were analyzed separately and should not be pooled into stationary accuracy. The table shows 30-pulse plans; the report includes each of the 19 scenarios and both budgets.

Scenario groupHistoriesMean absolute error
(MSO points)
Ready for review
Abrupt threshold drift4,0005.7623.18%
Deliberately incorrect first response2,00011.730.00%
Clinician ceiling of 60% MSO2,00010.000.00%
Alternative starting intensities9,0000.7897.32%
True MT near output boundaries2,0000.9991.45%

A stationary model lagged abrupt threshold changes. Wrong initial responses caused persistent errors but all 2,000 such runs were rejected by the readiness gate. All 2,000 ceiling-limited runs stopped after one pulse; no above-ceiling pulse was simulated. Safe stopping can leave a reachable threshold unresolved.

What the MTAT checks establish

37 of 37 isolated scalar reference cases matched. These retained numerical checks used original MTAT 2.1 scalar routines with environment stubs. They did not execute the full MTAT application or verify its interface, initialization pathways, stopping rules or readiness behavior end to end.

The 127,000-history simulation evaluated CTDS against known synthetic thresholds. It was not a head-to-head performance comparison with MTAT. Numerical agreement in a limited set of cases does not establish similar clinical performance or full-application equivalence.

Independent clinical validation is pending

Further work requires stronger estimate-quality checks, evaluation on held-out scenarios, complete original-application comparisons and a prospective clinician-supervised EMG study against an independent reference assessment. No clinical acceptance margin was established here. Visual-movement, active-MT and custom-spread performance were not validated.

Full report and reproducible study

Study conducted on website version 42. The estimator is unchanged in this publication update. The package includes the frozen code, analysis plan, seeds, raw endpoints, scenario summaries and pulse replays. Nine simulation checks passed; the retained 37 scalar checks also matched.

Source identity and method references

Tested source commit: 539405967be8691ceb59cc1bd6721754f342a487
Estimator SHA-256: 46b65b52c3b09b312f558947c57e323b2184b7b5263e2f81b469bde2c6261d7b

The published studies support the method’s context; they do not clinically validate CTDS.

Return to ML-PEST CTDS →

Copy your report

Clipboard access is unavailable. Select and copy the text below, or download it.