> Does anyone know a ROC calculating and plotting software?
If you have access to the World Wide Web, you can download any or all of
our software for ROC analysis without charge from
http://www.***.com/ ;Alternatively,
you can download the software by anonymous FTP, for which instructions
follow. ***If you download the software by FTP, please be sure to send
us the information requested in the "Read Me" file.***
I'm appending below a reading list on ROC methodology in case it may be
helpful.
Charles Metz
Charles E. Metz, Ph.D.
Professor of Radiology
The University of Chicago
___________________________________________
INSTRUCTIONS FOR DOWNLOADING UNIVERSITY OF CHICAGO ROC
SOFTWARE BY ANONYMOUS FTP VIA THE INTERNET
Our ROC software can be downloaded by FTP via the Internet. Type:
ftp://random.bsd.uchicago.edu (web-browser software)
OR
ftp random.bsd.uchicago.edu (command-line users)
user: anonymous
password: <yourname>
DOS, Macintosh, and generic UNIX/workstation/mainframe versions of our
software package are available. The software is located in an "roc"
directory, which contains the subdirectories:
ibmpc -- pkzip compressed/archived versions of the DOS
software
mac -- lha compressed/archived versions of the Mac
software
src -- generic Fortran source code for workstations and
mainframes (no screen prompts)
data -- sample data for testing the workstation/mainframe
programs
man -- simple-minded man pages for UNIX workstation/
mainframe users.
README files in the "roc" directory and in the "ibmpc" and "mac"
subdirectories explain how to get the files you want. The basic
procedure is:
1. decide which version(s) of the software package
you need
2. ftp and login into the computer (as above)
3. type "binary" at the ftp prompt (this forces
binary transfers)
4. cd to the appropriate directory
5. mget all the files
6. type "bye" to leave.
=========================
Example of an FTP session
=========================
% ftp random.bsd.uchicago.edu
login (ben:random.bsd.uchicago.edu): anonymous
password: ben herman
FTP> binary
binary mode set
FTP> cd roc/mac
cd successful
FTP> prompt
interactive file transfer canceled
FTP> mget *.*
getting file ###### as ######
.
.
.
FTP> bye
connection closed
%
=========================
End of example
=========================
If you have any questions or encounter any problems in downloading our
software, please contact my graduate student, Ben Herman, at
Charles Metz
_______________________________________________________
Readings in ROC Analysis, with Emphasis on Medical Applications
Prepared by Charles E. Metz
Department of Radiology
The University of Chicago
BACKGROUND:
Egan JP. Signal detection theory and ROC analysis. New York: Academic
Press, 1975.
Fryback DG, Thornbury JR. The efficacy of diagnostic imaging. Med
Decis Making 1991; 11: 88-94.
Griner PF, Mayewski RJ, Mushlin AI, Greenland P. Selection and
interpretation of diagnostic tests and procedures: principles and
applications. Annals Int Med 1981; 94: 553-592.
Lusted LB. Signal detectability and medical decision-making. Science
1971; 171: 1217-1219.
McNeil BJ, Adelstein SJ. Determining the value of diagnostic and
screening tests. J Nucl Med 1976; 17: 439-448.
McNeil BJ, Keeler E, Adelstein SJ. Primer on certain elements of
medical decision making. New Engl J Med 1975; 293: 211-215.
Robertson EA, Zweig MH, Van Steirtghem AC. Evaluating the clinical
efficacy of laboratory tests. Am J Clin Path 1983; 79: 78-86.
Zweig MH, Campbell G. Receiver-operating characteristic (ROC) plots: a
fundamental evaluation tool in clinical medicine. Clinical Chemistry
1993; 39: 561-177. [Erratum published in Clinical Chemistry 1993; 39:
1589.]
GENERAL:
Hanley JA. Alternative approaches to receiver operating characteristic
analysis. Radiology 1988; 168: 568-570.
Hanley JA. Receiver operating characteristic (ROC) methodology: the
state of the art. Critical Reviews in Diagnostic Imaging 1989; 29:
307-335.
King JL, Britton CA, Gur D, Rockette HE, Davis PL. On the validity of
the continuous and discrete confidence rating scales in receiver
operating characteristic studies. Invest Radiol 1993; 28: 962-963.
Metz CE. Basic principles of ROC analysis. Seminars in Nucl Med 1978;
8: 283-298.
Metz CE. ROC methodology in radiologic imaging. Invest Radiol 1986;
21: 720-733.
Metz CE. Some practical issues of experimental design and data analysis
in radiological ROC studies. Invest Radiol 1989; 24: 234-245.
Metz CE, Shen J-H. Gains in accuracy from replicated readings of
diagnostic images: prediction and assessment in terms of ROC analysis.
Med Decis Making 1992; 12: 60-75.
Rockette HE, Gur D, Metz CE. The use of continuous and discrete
confidence judgments in receiver operating characteristic studies of
diagnostic imaging techniques. Invest Radiol 1992; 27: 169-172.
Swets JA. ROC analysis applied to the evaluation of medical imaging
techniques. Invest Radiol 1979; 14: 109-121.
Swets JA. Indices of discrimination or diagnostic accuracy: their ROCs
and implied models. Psychol Bull 1986; 99: 100-117.
Swets JA. Measuring the accuracy of diagnostic systems. Science 1988;
240: 1285-1293.
Swets JA. Signal detection theory and ROC analysis in psychology and
diagnostics: collected papers. Mahwah, NJ; Lawrence Erlbaum Associates,
1996.
Swets JA, Pickett RM. Evaluation of diagnostic systems: methods from
signal detection theory. New York: Academic Press, 1982.
BIAS:
Begg CB, Greenes RA. Assessment of diagnostic tests when disease
verification is subject to selection bias. Biometrics 1983; 39:
207-215.
Begg CB, McNeil BJ. Assessment of radiologic tests: control of bias and
other design considerations. Radiology 1988; 167: 565-569.
Gray R, Begg CB, Greenes RA. Construction of receiver operating
characteristic curves when disease verification is subject to selection
bias. Med Decis Making 1984; 4: 151-164.
Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating
the efficacy of diagnostic tests. New Engl J Med 1978; 299: 926-930.
CURVE FITTING:
Dorfman DD, Alf E. Maximum likelihood estimation of parameters of
signal detection theory and determination of confidence intervals
rating method data. J Math Psych 1969; 6: 487-496.
Grey DR, Morgan BJT. Some aspects of ROC curve-fitting: normal and
logistic models. J Math Psych 1972; 9: 128-139.
Hanley JA. The robustness of the "binormal" assumptions used in fitting
ROC curves. Med Decis Making. 1988; 8: 197-203.
Metz CE, Herman BA, Shen J-H. Maximum-likelihood estimation of ROC
curves from continuously-distributed data. Statistics in Medicine (in
press).
Metz CE, Pan X. Proper binormal ROC curves: theory and
maximum-likelihood estimation. J. Math. Psych. (in press).
Pan X, Metz CE. The "proper" binormal model: parametric ROC curve
estimation with degenerate data. Academic Radiol 1997; 4: 380-389.
Swets JA. Form of empirical ROCs in discrimination and diagnostic
tasks: implications for theory and measurement of performance. Psychol
Bull 1986; 99: 181-198.
STATISTICS:
Agresti A. A survey of models for repeated ordered categorical response
data. Statistics in Medicine 1989; 8; 1209-1224.
Bamber D. The area above the ordinal {*filter*} graph and the area below
the receiver operating graph. J Math Psych 1975; 12: 387-415.
DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two
or more correlated receiver operating characteristic curves: a
nonparametric approach. Biometrics 1988; 44: 837-845.
Dorfman DD, Berbaum KS, Metz CE. ROC rating analysis: generalization to
the population of readers and cases with the jackknife method. Invest
Radiol 1992; 27: 723-731.
Dorfman DD, Berbaum KS, Metz CE, Lenth RV, Hanley JA, Dagga HA. Proper
ROC analysis: the bigamma model. Academic Radiol 1997; 4: 138-149.
Dorfman DD, Metz CE. Multi-reader multi-case ROC analysis: comments on
Beggs commentary. Academic Radiol 1995; 2 (Supplement 1): S76-S78.
Hanley JA, McNeil BJ. The meaning and use of the area under a receiver
operating characteristic (ROC) curve. Radiology 1982; 143: 29-36.
Hanley JA, McNeil BJ. A method of comparing the areas under receiver
operating characteristic curves derived from the same cases. Radiology
1983; 148: 839-843.
Jiang Y, Metz CE, Nishikawa RM. A receiver operating characterisitc
partial area index for highly sensitive diagnostic tests. Radiology
1996; 201: 745-750.
Ma G, Hall WJ. Confidence bands for receiver operating characteristic
curves. Med Decis Making 1993; 13: 191-197.
McClish DK. Analyzing a portion of the ROC curve. Med Decis Making
1989; 9: 190-195.
McClish DK. Determining a range of false-positive rates for which ROC
curves differ. Med Decis Making 1990; 10: 283-287.
McNeil BJ, Hanley JA. Statistical approaches to the analysis of
receiver operating characteristic (ROC) curves. Med Decis Making 1984;
4: 137-150.
Metz CE. Statistical analysis of ROC data in evaluating diagnostic
performance. In: Multiple regression analysis: applications in the
health sciences (D Herbert and R Myers, eds.). New York: American
Institute of Physics, 1986, pp. 365-384.
Metz CE. Quantification of failure to demonstrate statistical
significance: the usefulness of confidence intervals. Invest Radiol
1993; 28: 59-63.
Metz CE, Herman BA, Roe CA. Statistical comparison of two ROC curve
estimates obtained from partially-paired datasets. Med. Decis. Making
1998; 18: 110-121.
Metz CE, Kronman HB. Statistical significance tests for binormal ROC
curves. J Math Psych 1980; 22: 218-243.
Metz CE, Wang P-L, Kronman HB. A new
...
read more »
ROC calculating software