ANNOUNCE: Nonparametric ROC software for IBM

Nonparametric Receiver Operating Characteristic Analysis

Stephen Vida, M.D., F.R.C.P.(C)

Department of Psychiatry

Montreal General Hospital

McGill University

Receiver operating characteristic analysis is a leading tool for

the analysis of the sensitivity and specificity of

- signal detection devices, e.g. radar receivers.

- diagnostic tests, e.g. laboratory tests.

- diagnostic instruments, e.g. rating scales.

- radiologic instruments, e.g. x-ray machines, CT scanners,

and MRI scanners.

- diagnosticians, e.g. clinicians.

Sensitivity and specificity are key measures of test performance.

For tests yielding numerical scores, sensitivity and specificity

usually vary inversely over the range of theoretically possible cutoff

scores, complicating the task of quantifying and comparing the

diagnostic accuracy of tests.

Receiver Operating Characteristic analysis (ROC) approaches this

problem by plotting the curve of sensitivity, or true positive rate,

versus 1 - specificity, or false positive rate, for all possible cutoff

scores of the test. The area under the ROC curve (AUC) can be used to

describe the diagnostic accuracy of the test. Parametric and

nonparametric methods exist that allow the calculation of the AUC and

the comparison of tests. A disadvantage of parametric formulations is

the assumption of a normal or Gaussian distribution of test scores.

Parametric ROC formulations, based on normal or Gaussian

distributional assumptions, have existed for some time. While to some

degree they are robust to non-normality of underlying data, some

authors have reported non-normal datasets for which parametric

formulations either failed to compute solutions or computed erroneous

solutions.

Nonparametric ROC formulations have more recently evolved that are

distribution-free, in that they do not depend on the distribution of the

test scores being analyzed but rather on their ranking. Although

nonparametric, statistical efficiency comparable to parametric approaches

has been reported.

Nonparametric ROC may be useful when

1. a distribution-free method is preferred because of

non-normality of data.

2. parametric methods have failed to compute or have yielded

erroneous or implausible results.

3. corroboration or comparison of parametric and nonparametric

methods is desired.

A Nonparametric Receiver Operating Characteristic Program is

available for IBM-compatible computers that

- plots from one to three ROC curves on the same graph.

- calculates the area under the ROC curve, its standard error,

and its 95% confidence limits.

- calculates the sensitivity and specificity, with 95%

confidence limits, at all possible cutoff scores of the

instrument being analyzed.

- statistically compares two ROC curves from independent

samples.

- statistically compares two or three ROC curves simultaneously

from correlated samples.

- exports ROC curve coordinates to Harvard graphics templates,

included.

- performs statistical power analysis to estimate sample sizes

required for ROC studies.

- is accompanied by a comprehensive user's manual that includes

a review of the theory of ROC analysis and detailed

instructions for the use of the program.

For further information about ROC or the program described above,