The total operating characteristic to measure diagnostic ability for multiple thresholds

References

• Blough, D.S., 1967. Stimulus generalization as signal detection in pigeons. Science, 158 (3803), 940–941.
   PubMed Web of Science ®Google Scholar
• Bradley, A.P., 1997. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition, 30 (7), 1145–1159.
   Web of Science ®Google Scholar
• Cain, W.S., 1977. Differential sensitivity for smell: ‘noise’ at the nose. Science, 195 (4280), 796–798.
   PubMed Web of Science ®Google Scholar
• Camacho Olmedo, M.T., Paegelow, M., and Mas, J.F., 2013. Interest in intermediate soft-classified maps in land change model validation: suitability versus transition potential. International Journal of Geographical Information Science, 27 (12), 2343–2361. 
   Web of Science ®Google Scholar
• Carterette, E.D. and Jones, M.H., 1967. Visual and auditory information processing in children and adults. Science, 156 (3777), 986–988.
   PubMed Web of Science ®Google Scholar
• Chen, Y., et al., 2013. Modeling urban land-use dynamics in a fast developing city using the modified logistic cellular automaton with a patch-based simulation strategy. International Journal of Geographical Information Science, 28 (2), 234–255. 
   Web of Science ®Google Scholar
• Conway, T.M. and Wellen, C.C., 2011. Not developed yet? Alternative ways to include locations without changes in land use change models. International Journal of Geographical Information Science, 25 (10), 1613–1631.
   Web of Science ®Google Scholar
• Cook, N.R., 2007. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation, 115 (7), 928–935.
   PubMed Web of Science ®Google Scholar
• Dodd, L.E. and Pepe, M.S., 2003. Partial AUC estimation and regression. Biometrics, 59 (3), 614–623.
   PubMed Web of Science ®Google Scholar
• Eastman, J.R., Van Fossen, M., and Solorzano, L.A., 2005. Transition potential modeling for land cover change. In: D. Maguire, M. Batty, and M.F. Goodchild, eds. GIS, Spatial analysis and modeling. Redlands, CA: ESRI Press, 339–368.
   Google Scholar
• Egan, J.P., 1975. Signal detection theory and ROC analysis. Series in cognition and perception. New York: Academic Press.
   Google Scholar
• Fawcett, T., 2006. An introduction to ROC analysis. Pattern Recognition Letters, 27 (8), 861–874.
   Web of Science ®Google Scholar
• Fielding, A.H. and Bell, J.F., 1997. A review of methods for the assessment of prediction errors in conservation presence/absence models. Environmental Conservation, 24 (1), 38–49.
   Web of Science ®Google Scholar
• Fleming, S.M., et al., 2010. Relating introspective accuracy to individual differences in brain structure. Science, 329 (5998), 1541–1543.
   PubMed Web of Science ®Google Scholar
• Gao, J. and Zhang, Y., 2012. Incorporating spectral data into logistic regression model to classify land cover: a case study in Mt. Qomolangma (Everest) National Nature Preserve. International Journal of Geographical Information Science, 26 (10), 1–18.
   Google Scholar
• Gerçek, D., Toprak, V., and Strobl, J., 2011. Object-based classification of landforms based on their local geometry and geomorphometric context. International Journal of Geographical Information Science, 25 (6), 1011–1023.
   Web of Science ®Google Scholar
• Golicher, D., et al., 2012. Pseudo-absences, pseudo-models and pseudo-niches: pitfalls of model selection based on the area under the curve. International Journal of Geographical Information Science, 26 (11), 2049–2063.
   Web of Science ®Google Scholar
• Green, D.M. and Swets, J.A., 1966. Signal detection theory and psychophysics. New York: Wiley Press.
   Google Scholar
• Hand, D.J. and Till, R.J., 2001. A simple generalization of the area under the ROC curve to multiple class classification problems. Machine Learning, 45 (2), 171–186.
   Web of Science ®Google Scholar
• Hanley, J.A. and McNeil, B.J., 1982. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143 (1), 29–36.
   PubMed Web of Science ®Google Scholar
• Hirzel, A.H., et al., 2006. Evaluating the ability of habitat suitability models to predict species presences. Ecological Modeling, 199 (2), 142–152.
   Web of Science ®Google Scholar
• Jansen, L.J.M. and Veldkamp, T., 2011. Evaluation of the variation in semantic contents of class sets on modelling dynamics of land-use changes. International Journal of Geographical Information Science, 26 (4), 717–746.
   Web of Science ®Google Scholar
• Jeong, M.-H., et al., 2013. Decentralized and coordinate-free computation of critical points and surface networks in a discretized scalar field. International Journal of Geographical Information Science, 28 (1), 1–21. 
   Web of Science ®Google Scholar
• Jiang, Y.L., Metz, C.E., and Nishikawa, R.M., 1996. A receiver operating characteristic partial area index for highly sensitive diagnostic tests. Radiology, 201 (3), 745–750.
   PubMed Web of Science ®Google Scholar
• Johnson, D.S., et al., 2007. Genome-wide mapping of in vivo protein-DNA interactions. Science, 316 (5830), 1497–1502.
   PubMed Web of Science ®Google Scholar
• Johnson, J.M., et al., 2003. Genome-wide survey of human alternative pre-mRNA splicing with exon junction microarrays. Science, 302 (5653), 2141–2144.
   PubMed Web of Science ®Google Scholar
• Jolliffe, I.T. and Stephenson, D.B., 2003. Forecast verification a practitioner’s guide in atmospheric science. Chichester: Wiley.
   Google Scholar
• Knaus, W.A., Wagner, D.P., and Lynn, J., 1991. Short-term mortality predictions for critically ill hospitalized adults: science and ethics. Science, 254 (5030), 389–394.
   PubMed Web of Science ®Google Scholar
• Kolb, M., Mas, J.-F., and Galicia, L., 2013. Evaluating drivers of land-use change and transition potential models in a complex landscape in Southern Mexico. International Journal of Geographical Information Science, 27 (9), 1804–1827. 
   Web of Science ®Google Scholar
• Leitão, P.J., Moreira, F., and Osborne, P.E., 2011. Effects of geographical data sampling bias on habitat models of species distributions: a case study with steppe birds in southern Portugal. International Journal of Geographical Information Science, 25 (3), 439–454.
   Web of Science ®Google Scholar
• Li, L., Wang, J., and Leung, H., 2010. Using spatial analysis and Bayesian network to model the vulnerability and make insurance pricing of catastrophic risk. International Journal of Geographical Information Science, 24 (12), 1759–1784.
   Web of Science ®Google Scholar
• Lin, Y., et al., 2010. Predictive ability of logistic regression, auto-logistic regression and neural network models in empirical land-use change modeling – a case study. International Journal of Geographical Information Science, 25 (1), 65–87.
   Web of Science ®Google Scholar
• Lobo, J.M., Jimenez-Valverde, A., and Real, R., 2008. AUC: a misleading measure of the performance of predictive distribution models. Global Ecology and Biogeography, 17 (2), 145–151.
   Web of Science ®Google Scholar
• Lusted, L.B., 1971. Signal detectability and medical decision-making. Science, 171 (3977), 1217–1219.
   PubMed Web of Science ®Google Scholar
• Manzo, G., et al., 2012. GIS techniques for regional-scale landslide susceptibility assessment: the Sicily (Italy) case study. International Journal of Geographical Information Science, 27 (7), 1433–1452.
   Web of Science ®Google Scholar
• Mas, J.-F., et al., 2013. A suite of tools for ROC analysis of spatial models. ISPRS International Journal of Geo-Information, 2 (3), 869–887.
   Google Scholar
• Mateo Sánchez, M.C., Cushman, S.A., and Saura, S., 2013. Scale dependence in habitat selection: the case of the endangered brown bear (Ursus arctos) in the Cantabrian Range (NW Spain). International Journal of Geographical Information Science. doi:10.1080/13658816.2013.776684 
   Web of Science ®Google Scholar
• Miller, J. and Franklin, J., 2002. Modeling the distribution of four vegetation alliances using generalized linear models and classification trees with spatial dependence. Ecological Modeling, 157 (2), 227–247.
   Web of Science ®Google Scholar
• Nevin, J.A., 1965. Decision theory in studies of discrimination in animals. Science, 150 (3699), 1057.
   PubMed Web of Science ®Google Scholar
• Nuechterlein, K.H., Parasuraman, R., and Jiang, Q., 1983. Visual sustained attention: image degradation produces rapid sensitivity decrement over time. Science, 220 (4594), 327–329.
   PubMed Web of Science ®Google Scholar
• Overmars, K.P. and Verburg, P.H., 2005. Analysis of land use drivers at the watershed and household level: linking two paradigms at the Philippine forest fringe. International Journal of Geographical Information Science, 19 (2), 125–152.
   Web of Science ®Google Scholar
• Paegelow, M. and Olmedo, M.T.C., 2005. Possibilities and limits of prospective GIS land cover modelling – a compared case study: Garrotxes (France) and Alta Alpujarra Granadina (Spain). International Journal of Geographical Information Science, 19 (6), 697–722.
   Web of Science ®Google Scholar
• Pérez-Vega, A., Mas, J.-F., and Ligmann-Zielinska, A., 2012. Comparing two approaches to land use/cover change modeling and their implications for the assessment of biodiversity loss in a deciduous tropical forest. Environmental Modelling & Software, 29 (1), 11–23.
   Web of Science ®Google Scholar
• Peters, J., et al., 2011. Synergy of very high resolution optical and radar data for object-based olive grove mapping. International Journal of Geographical Information Science, 25 (6), 971–989.
   Web of Science ®Google Scholar
• Peterson, A.T., Papes, M., and Soberón, J., 2008. Rethinking receiver operating characteristic analysis applications in ecological niche modeling. Ecological Modeling, 213 (1), 63–72.
   Web of Science ®Google Scholar
• Pontius, R.G. and Batchu, K., 2003. Using the relative operating characteristic to quantify certainty in prediction of location of land cover change in India. Transactions in GIS, 7 (4), 467–484.
   Google Scholar
• Pontius, R.G. and Pacheco, P., 2004. Calibration and validation of a model of forest disturbance in the Western Ghats, India 1920–1990. GeoJournal, 61 (4), 325–334. 
   Google Scholar
• Reed, A.V., 1973. Speed-accuracy trade-off in recognition memory. Science, 181 (4099), 574–576.
   PubMed Web of Science ®Google Scholar
• Saatchi, S., et al., 2008. Modeling distribution of Amazonian tree species and diversity using remote sensing measurements. Remote Sensing of Environment, 112 (5), 2000–2017.
   Web of Science ®Google Scholar
• Smith, E.L.D., et al., 1982. Color vision is altered during the suppression phase of binocular rivalry. Science, 218 (4574), 802–804.
   PubMed Web of Science ®Google Scholar
• Stephan, C., et al., 2003. Comparison of eight computer programs for receiver-operating characteristic analysis. Clinical Chemistry, 49 (3), 433–439.
   PubMed Web of Science ®Google Scholar
• Swets, J.A., 1961. Is there a sensory threshold? Science, 134 (3473), 168–177.
   PubMed Web of Science ®Google Scholar
• Swets, J.A., 1963. Information retrieval systems. Science, 141 (3577), 245–250.
   PubMed Web of Science ®Google Scholar
• Swets, J.A., 1973. The relative operating characteristic in psychology. Science, 182 (4116), 990–1000.
   PubMed Web of Science ®Google Scholar
• Swets, J.A., 1988. Measuring the accuracy of diagnostic systems. Science, 240 (4857), 1285–1293.
   PubMed Web of Science ®Google Scholar
• Swets, J.A., et al., 1979. Assessment of diagnostic technologies. Science, 205 (4408), 753–759.
   PubMed Web of Science ®Google Scholar
• Wang, J. and Mountrakis, G., 2010. Developing a multi-network urbanization model: a case study of urban growth in Denver, Colorado. International Journal of Geographical Information Science, 25 (2), 229–253.
   Web of Science ®Google Scholar
• Wang, N., et al., 2013. Comparative performance of logistic regression and survival analysis for detecting spatial predictors of land-use change. International Journal of Geographical Information Science, 27 (10), 1960–1982. 
   Web of Science ®Google Scholar
• Wiley, E.O., et al., 2003. Niche modeling and geographic range predictions in the marine environment using machine-learning algorithm. Oceanography, 16 (3), 120–127.
   Google Scholar