Research on research evaluation

References

• Abelson , P. 1990 . ‘Mechanisms for evaluating scientific information and the role of peer review’ . Journal of the American Society for Information Science, , 41 (3) : 216 – 222 .
   Web of Science ®Google Scholar
• Albert , R. S. 1975 . ‘Toward a behavioral definition of genius’ . American Psychologist, , 35 : 140 – 151 .
   Web of Science ®Google Scholar
• Allardt , E. , Lysgaard , S. and Bottger Sorensen , A. 1988 . Sociologin i Sverige. Vetenskap, miljö och organisation. Utvärdering av svensk sociologi (Sociology in Sweden. Science, environment and organization. Evaluation of Swedish sociology) , Uppsala : Ord & Form AB .
   Google Scholar
• Allwood , C. M. and Montgomery , H. 1982 . ‘Detection of errors in statistical problem solving’ . Scandinavian Journal of Psychology, , 23 : 131 – 139 .
   Web of Science ®Google Scholar
• Averch , H. 1991 . ‘The practice of research evaluation in the United States’ . Research Evaluation, , 1 (3) : 130 – 136 .
   Google Scholar
• Barron , F. 1963 . “ ‘The disposition toward originality’ ” . In Scientific Creativity: Its Recognition and Development, , Edited by: Taylor , C.W. and Barron , F. 139 – 152 . New York : John Wiley .
   Google Scholar
• Bayer , A. E. and Folger , J. 1966 . ‘Some correlates of a citation measure of productivity in science’ . Sociology of Education, , 39 : 381 – 390 .
   Web of Science ®Google Scholar
• Blume , S. S. and Sinclair , R. 1973 . ‘Chemists in British universities: a study of the reward system in science’ . American Sociological Review, , 38 : 126 – 138 .
   Web of Science ®Google Scholar
• Boor , M. 1982 . ‘The citation impact factor : another dubious index of journal quality ‘ . American Psychologist, , 37 : 975 – 977 .
   Web of Science ®Google Scholar
• Boutrey , G. A. 1970 . ‘Quantity versus quality in scientific research (II) : The paper explosion’ . Impact of Science on Society, , 20 : 195 – 206 .
   Google Scholar
• Boye , Y. , Andersson , T. L. , Lindström , S. , Montgomery , H. and Stankiewicz , R. 1987a . Sektorsforskning ihögskolemiljö. lakttagelser och slutsatser om sektorsforskning från intervjuer med högskoleforskare (Sectoral research in university settings. Observations and conclusions on sectoral research based on university researcher interviews) , Stockholm : Högsektprojektet, delrapport 2 .
   Google Scholar
• Boye , Y. , Nydén , M. , Hugard , Å. and Karlsson , A.‐C. 1987b . Hur mår högskolan av sektorsforskning? Resultat frân en enkät riktad till högskoleforskarna (Which effects do sectoral research have on the universities? Results of a questionnaire to university researchers) , Stockholm : Högsektprojektet, delrapport 1 .
   Google Scholar
• Brante , T. 1984 . Vetenskapens sociala grunder (The social base of science) , Stockholm : Rabén & Sjögren .
   Google Scholar
• Brockhaus , W. L. and Mickelsen , J. F. 1976 . ‘A worldwide investigation of writings and publications concerning the Delphi methodology’ . Journal of the International Society for Technology Assessment, , 2 : 5 – 34 .
   Google Scholar
• Brohult , M. 1987 . ‘Humaniora i kris eller på uppgång?’ ('Are the humanities in a crisis or on a rise?') . Svenska Dagbladet, , 26 May
   Google Scholar
• Brooks , T. A. 1985 . ‘Private acts and public objects: an investigation of citer motivations’ . Journal of the American Society for Information Science, , 36 : 223 – 229 .
   Web of Science ®Google Scholar
• Brooks , T. A. 1986 . ‘Evidence of complex citer motivations’ . Journal of the American Society for Information Science, , 37 : 34 – 36 .
   Web of Science ®Google Scholar
• Callon , M. , Courtial , J.‐P. and Laville , F. 1989 . Co‐word Analysis: A Tool for the Evaluation of Public Research Policy. The Case of Polymers, , Paris : Centre de Sociologie de l'Innovation .
   Google Scholar
• Capron , H. ‘Measuring the economic impact of R&D: An introduction’ . Proceedings of the Workshop on Quantitative Evaluation of the Impact of R&D Programmes, . January 23–24 1992 . Edited by: Capron , H. pp. 1 – 10 . Brussels : Commission of the European Communities, Brussels .
   Google Scholar
• Cattell , R. B. and Drevdahl , J. E. 1955 . ‘A comparison of the personality profile (16 p.f.) of eminent researchers with that of eminent teachers and administrators, and of the general population ‘ . British Journal of Psychology, , 46 : 248 – 261 .
   PubMed Web of Science ®Google Scholar
• Cawkell , A. E. 1978 . ‘Evaluating scientific journals with journal citation reports—a case study in acoustics’ . Journal of the American Society for Information Science, , 29 : 41 – 46 .
   Web of Science ®Google Scholar
• Chase , J. M. 1970 . ‘Normative criteria for scientific publication’ . The American Sociologist, , 5 : 262 – 265 .
   Google Scholar
• Christensen , A. 1985 . ‘Släpp in människorna’ ('Let the people in') . Dagens Nyheter, Humanvetenskapliga dagar 1985 (bilaga), , 7 May : 23
   Google Scholar
• Christensen‐Sz Alanski , J. J. J. and Beach , L. R. 1984 . ‘The citation bias : fad and fashion in the judgment and decision literature’ . American Psychologist, , 39 : 75 – 78 .
   Web of Science ®Google Scholar
• Christiansen , J. and Christiansen , L. Research on research: evaluation of evaluations in the Nordic countries . A paper from the research project ‘Effektiv forskning ‘, prepared for the United Nations Seminar on Evaluation in the Management of R & D. Madrid . April 3–7 1989 . Center for Public Organization and Management, Copenhagen School of Economics and Social Science .
   Google Scholar
• Christiansen , K. and Foss Hansen , H. 1993 . Forskningsevaluering i teorri og praksis (Research evaluation in theory and practice) , Frederiksberg : Samfundslitteratur .
   Google Scholar
• Chubin , D. E. and Hackett , E. J. 1990 . Peerless Science. Peer Review and U.S. Science Policy, , Albany : State University of New York .
   Google Scholar
• Chubin , D. E. and Moitra , S. D. 1975 . ‘Content analysis of references : adjunct or alternative to citation counting?’ . Social Studies of Science, , 5 : 423 – 441 .
   Web of Science ®Google Scholar
• Clark , K. E. 1975 . America's Psychologists. A Survey of a Growing Profession, , Washington, DC : American Psychological Association .
   Google Scholar
• Cole , J. R. 1979 . Fair Science. Women in the Scien'‐ific Community, , New York : The Free Press .
   Google Scholar
• Cole , J. R. and Cole , S. 1971 . ‘Measuring the quality of sociological research: problems in the use of Science Citation Index’ . American Sociologist, , 6 : 23 – 29 .
   Google Scholar
• Cole , J. R. and Cole , S. 1973 . ‘Citation analysis’ . Sciences, , 183 : 32 – 33 .
   Web of Science ®Google Scholar
• Cole , S. and Cole , J. R. 1987 . ‘Testing the Ortega hypothesis: milestone or millstone?’ . Scientometrics, , 23 : 345 – 353 .
   Web of Science ®Google Scholar
• Cole , S. , Cole , J. R. and Simon , G. A. 1981 . ‘Chance and consensus in peer review . Science, , 214 20 November : 881 – 885 .
   PubMed Web of Science ®Google Scholar
• Cole , S. , Rubin , L. and Cole , J. R. 1977 . ‘Peer review and the support of science’ . The Scientific American, , 237 : 34 – 41 .
   PubMed Web of Science ®Google Scholar
• Cox , W. M. and Catt , V. 1979 . ‘Productivity in publication and the quality of psychology departments: Reply to Schaeffer and Sulyma’ . American Psychologist, , 34 : 443 – 444 .
   Web of Science ®Google Scholar
• Crane , D. 1969 . ‘Social structure in a group of scientists: a test of the “Invisible college” hypothesis’ . American Sociological Review, , 34 : 335 – 352 .
   Web of Science ®Google Scholar
• Crawford , S. and Stucki , L. 1990 . ‘Peer review and the changing research record’ . Journal of the American Society for Information Science, , 41 (3) : 223 – 228 .
   Web of Science ®Google Scholar
• Croft , G. 1983 . ‘An investigation into the applicability of science citation index as a tool to measure the usefulness of publications of the International Tin Research Institute’ . Aslib Proceedings, , 35 : 249 – 257 .
   Google Scholar
• Cronin , B. 1984 . The Citation Process. The Role and Significance of Citations in Scientific Communication, , London : Taylor Graham .
   Google Scholar
• Croom , D. L. 1970 . ‘Dangers in the use of science citation index’ . Nature, , 227 : 1173
   PubMed Web of Science ®Google Scholar
• Davidson Frame , J. and Carpenter , M. P. 1979 . ‘International research collaboration’ . Social Studies of Science, , 9 : 481 – 497 .
   Web of Science ®Google Scholar
• De Mey , M. 1982 . The Cognitive Paradigm, , Dordrecht : D. Reidel Publishing Company .
   Google Scholar
• Dieks , D. and Chang , H. 1976 . ‘Differences in impact of scientific publications: some indices derived from a citation analysis’ . Social Studies of Science, , 6 : 247 – 267 .
   Web of Science ®Google Scholar
• Dolan , W. P. 1976 . “ Preface by W. A. Arrowsmith, Evaluation of higher education committee of the study commission on undergraduate education and the education of teachers ” . In The Ranking Game: The Power of the Academic Elite, Lincoln, Nebraska
   Google Scholar
• Drew , D. E. and Karpf , R. S. 1975 . Evaluating Science Departments: A New Index, , Santa Monica, CA : The Rand Corporation .
   Google Scholar
• Edge , D. 1979 . ‘Quantitative measures of communication in science: a critical review’ . History of Science, , 17 : 102 – 134 .
   PubMedGoogle Scholar
• Eliassen , K. A. , Pedersen , M. N. and forskningsrÅdet , Humamstisk‐samhÄllsvetenskapliga . 1984 . Svensk samhällsorganisation och förvaltning. En inventering av svenskforskning (Swedish societal organization and administration. An inventory of Swedish research) , Uppsala : Ord och Form AB .
   Google Scholar
• Elzinga , A. 1988 . ‘The consequences of evaluation for academic research’ . Science Studies, , 1 : 5 – 14 .
   Google Scholar
• Elzinga , A. and Jamison , A. 1984 . “ ‘Forskningssociologi’ (Research sociology) ” . In Forskning om forskning eller konsten att beskriva en elefant (Research on research or the art of describing an elephant) , Edited by: Bärmark , J. Lund : Natur och Kultur .
   Google Scholar
• Endler , N. S. 1977 . ‘Research productivity and scholarly impact of Canadian psychology departments (1975)’ . American Psychologist, , 33 : 106 – 108 .
   Web of Science ®Google Scholar
• Farina , C. and Gibbons , M. 1979 . ‘A quantitative analysis of the science research council's policy of “selectivity and concentration"’ . Research Policy, , 8 : 306 – 308 .
   Web of Science ®Google Scholar
• Folly , G. , Hajtman , B. , Nagy , J. I. and Rufi , I. 1981 . ‘Some methodological problems in ranking scientists by citation analysis . Scientometrics, , 3 : 135 – 147 .
   Web of Science ®Google Scholar
• Foss Hansen , H. 1988 . ‘Control and organization of research: a case study of humanities departments in Denmark’ . Science Studies, , 1 : 15 – 24 .
   Google Scholar
• Fox , M. F. 1983 . ‘Publication productivity among scientists: a critical review’ . Social Studies of Science, , 13 : 285 – 305 .
   Web of Science ®Google Scholar
• Fridjonsdottir , K. 1983 . Vetenskap och politik. En kunskapssociologisk studie (Science and politics. A study in the sociology of knowledge) , Malmö : Förlaget Akademilitteratur AB .
   Google Scholar
• Garfield , E. 1977 . ‘The 250 most‐cited prirmary authors, 1961–1975. Part II. The correlation between citedness, Nobel‐prizes and academy memberships’ . Current Contents, , 9 : 5 – 15 .
   Google Scholar
• Garfield , E. 1979a . Citation Indexing: Its Theory and Application in Science, Technology and Humanities , New York : John Wiley .
   Google Scholar
• Garfield , E. 1979b . ‘Is citation analysis a legitimate evaluation tool?’ . Scientometrics, , 1 : 359 – 375 .
   Web of Science ®Google Scholar
• Garfield , E. 1983a . ‘How to use citation analysis for faculty evaluations, and when is it relevant ?’ . Current Contents, , 31 October : 5 – 13 . Part 1
   Google Scholar
• Garfield , E. 1983b . ‘How to use citation analysis for faculty evaluations, and when is it relevant?’ . Current Contents, , 7 November : 5 – 13 . Part 2
   Google Scholar
• Garfield , E. 1990 . ‘The most‐cited papers of ;all times, SCI 1945–1988. Part 1A. The SCI top 100—will the Lowry method ever be obliterated?’ . Current Contents, , 12 February : 3 – 15 .
   Google Scholar
• Garfield , E. , Malin , M. V. and Small , L. 1978 . “ Citation data as science indicators ” . In Toward a Metric of Science. The Adven of Science Indicators, , Edited by: Elkana , Y. , Lederberg , J. , Merton , R. K. , Thackray , A. and Zuckerman , H. 179 – 207 . New York : John Wiley .
   Google Scholar
• Gillet , R. 1989 . ‘Research performance indicators based on peer review: a critical analysis’ . Higher Education Quarterly, , 43 (1) : 21 – 38 .
   Web of Science ®Google Scholar
• Gottfredson , S. D. 1978 . ‘Evaluating psychological research reports. Dimensions, reliability, and correlates of quality judgment’ . American Psychologist, , 33 : 920 – 934 .
   Web of Science ®Google Scholar
• Hamilton , D. P. 1991 . ‘Research papers: who's uncited now?’ . Science, , 251 4 January : 25
   PubMed Web of Science ®Google Scholar
• Hargens , L. L. , Mullins , N. C. and Hecht , P. K. 1980 . ‘Research areas and stratification processes in science’ . Social Studies of Science, , 10 : 55 – 74 .
   Web of Science ®Google Scholar
• Harnad , S. 1985 . ‘Rational disagreement in peer review’ . Science, Technology, & Human Values, , 10 : 55 – 62 .
   PubMed Web of Science ®Google Scholar
• HÄrnqvist , K. and Gustafsson , J. E. 1978 . International references to Swedish educational research: a citation study, Reports from The Institute of Education , University of Göteborg .
   Google Scholar
• Hartmann , I. and Neidhardt , F. 1990 . ‘Peer review at the Deutsche Forschungsgemeinschaft’ . Scientometrics, , 19 (5–6) : 419 – 425 .
   Web of Science ®Google Scholar
• Hemlin , S. and Montgomery , H. 1985 . Forskning om utvärdering av forskning: Vad har gjorts och vad kan göras? (Research on evaluation of research: what has been done and what can be done?) , Department of Psychology, University of Goteborg . Report No. 1
   Google Scholar
• Hemlin , S. and Montgomery , H. 1988 . ‘Att bedöma vetenskaplig kvalitet: En undersökning av sakkunnigutlåtanden vid professorstillsättningar’ (To judge scientific quality: an investigation of professorial committee evaluations) . VEST. Tidskrift för vetenskapsstudier, , : 8 – 15 .
   Google Scholar
• Hemlin , S. and Montgomery , H. 1990 . ‘Scientists’ conceptions of scientific quality’ . Science Studies, , 1 : 73 – 81 .
   Google Scholar
• Hemlin , S. and Montgomery , H. 1991 . “ ‘Research on scientific quality assessment: what has been done and what can be done?’ ” . In Gothenburg Psychological Reports, , 21 Gothenburg University . no. 2
   Google Scholar
• Hemlin , S. and Montgomery , H. 1993 . ‘Peer judgments of scientific quality: a cross‐disciplinary document analysis of professorship candidates’ . Science Studies, , 6 (1) : 19 – 27 .
   Google Scholar
• Hempel , C. G. 1952 . Fundamentals of Concept Formation in Empirical Science, , Chicago : The University of Chicago Press .
   Google Scholar
• Hobday , M. 1987 . The SPRU evaluation of the Alvey programme, SPRU , University of Sussex .
   Google Scholar
• Holton , G. 1978 . “ ‘Can science be measured?’ ” . In Toward a Metric of Science. The Advent of Science Indicators, , Edited by: Elkana , Y. , Lederberg , J. , Merton , R. K. , Thackray , A. and Zuckerman , H. 39 – 68 . New York : John Wiley .
   Google Scholar
• Howard , G. S. , Cole , D. A. and Maxwell , S. E. 1987 . ‘Research productivity in psychology based on publication in the journals of the American Psychological Association’ . American Psychologist, , 42 : 975 – 986 .
   Web of Science ®Google Scholar
• Inhaber , H. and Przednowek , K. 1976 . ‘Quality of research and the Nobel Prizes’ . Social Studies of Science, , 6 : 33 – 50 .
   Web of Science ®Google Scholar
• Jackson , D. N. and Rushton , J. P. , eds. 1987 . Scientific Excellence. Origins and Assessment, , London : Sage .
   Google Scholar
• Kahn , F. D. 1983 . ‘Big ideas for big projects’ . The Times Higher Education Supplement, , 29 April : 12
   Google Scholar
• Kasperson , C.J. 1978 . ‘An analysis of the relationship between information sources and creativity in scientists and engineers’ . Human Communication Research, , 4 : 111 – 119 .
   Google Scholar
• Knorr , K. D. , Mittermeier , R. , Aichholzer , G. and Waller , G. 1979a . “ Leadership and group performance: a positive relationship in academic research units ” . In Scientific Productivity. The Effectiveness of Research Groups in Six Countries, , Edited by: Andrews , F.M. 95 – 120 . Cambridge : Cambridge University Press .
   Google Scholar
• Knorr , K. D. , Mittermeier , R. , Aichholzer , G. and Waller , G. 1979b . “ ‘Individual publication productivity as a social position effect in academic and industrial research units’ ” . In Scientific productivity. The Effectiveness of Research Groups in Six Countries, , Edited by: Andrews , F. M. 55 – 94 . Cambridge : Cambridge University Press .
   Google Scholar
• Kochen , M. 1978 . “ ‘Models of scientific output’ ” . In Toward a Metric of Science. The Advent of Science Indicators, , Edited by: Elkana , Y. , Lederberg , J. , Merton , R. K. , Thackray , A. and Zuckerman , H. 97 – 136 . New York : John Wiley .
   Google Scholar
• Kostoff , R. N. 1994 . ‘Assessing research impact. US Government retrospective and quantitative approaches’ . Science and Policy, , 21 (1) : 13 – 22 .
   Google Scholar
• Kowalewski , Z. 1974 . “ ‘Bureaucratic trends in the organization and institutionalization of scientific activity’ ” . In Social Processes of Scientific Development, , Edited by: Whitley , R. 269 – 280 . London : Routledge & Kegan Paul .
   Google Scholar
• Kruglanski , A. W. , Baldwin , M. W. and Towson , S. M.J. 1983 . “ ‘The lay‐epistemic process in attribution‐making’ ” . In Attribution Theory. Social and Functional Extensions, , Edited by: Hewstone , M. 81 – 95 . Oxford : Blackwells .
   Google Scholar
• Kuhn , T. S. 1970 . The Structure of Scientific Revolutions, , 2nd edn , Chicago : The University of Chicago Press .
   Google Scholar
• Kuhn , T. S. 1977 . The Essential Tension, , Chicago : The University of Chicago Press .
   Google Scholar
• LaFollette , M. 1994 . ‘Measuring equity. The U.S. General Accounting Office study of peer review’ . Science Communication, , 16 (2) : 211 – 220 .
   Web of Science ®Google Scholar
• Lakatos , I. 1976 . ‘Review of S. Toulmin's Human Understanding’ . Minerva, , 14 : 128 – 129 .
   Google Scholar
• Laudan , L. 1984 . Science and Values, , Berkeley : University of California Press .
   Google Scholar
• Lawani , S. M. 1977 . ‘Citation analysis and the quality of scientific productivity’ . Bio Science, , 27 : 26 – 31 .
   Web of Science ®Google Scholar
• Lawani , S. M. 1982 . ‘On the relationship between quantity and quality of a country's research productivity’ . Journal of Information Science, , 5 : 143 – 145 .
   Web of Science ®Google Scholar
• Lewin , A. Y. and Zwany , A. 1976 . ‘Peer nominations: a model, literature critique and paradigm for research’ . Personell Psychology, , 29 : 423 – 447 .
   Web of Science ®Google Scholar
• Lichtenstein , S. and Fischoff , B. 1980 . ‘Training for calibration’ . Organizational Behavior and Human Performance, , 26 : 149 – 171 .
   Web of Science ®Google Scholar
• Lindsey , D. 1980 . ‘Production and citation measures in the sociology of science: the problem of multiple authorship’ . Social Studies of Science, , 10 : 145 – 162 .
   Web of Science ®Google Scholar
• Line , M. B. 1981 . ‘The structure of social science literature as shown by a large‐scale citation analysis’ . Social Science Information Studies, , 1 : 67 – 87 .
   Google Scholar
• Luukkonen‐Gronow , T. Analytical review of methods and criteria used for different purposes in the evaluation of R & D activities . Paper presented at the ad hoc meeting on methodologies used in the evaluation of R & D activities . June 24—28 1985 , Helsinki, Finland. United Nations Economic and Social Council .
   Google Scholar
• MacRoberts , M. H. and MacRoberts , B. R. 1986 . ‘Quantitative measures of communication in science: a study of the formal level’ . Social Studies of Science, , 16 : 151 – 172 .
   Web of Science ®Google Scholar
• MacRoberts , M. H. and MacRoberts , B. R. 1987 . ‘Testing the Ortega hypothesis: facts and artifacts’ . Scientometrics, , 12 : 293 – 295 .
   Web of Science ®Google Scholar
• Maestre , C. and Pavitt , K. 1972 . Analytical Methods in Government Science Policy. An Evaluation, , Paris : OECD .
   Google Scholar
• Mahoney , M.J. 1979 . ‘Psychology of the scientist: an evaluative review’ . Social Studies of Science, , 9 : 349 – 375 .
   Web of Science ®Google Scholar
• Mahoney , M.J. 1985 . ‘Open exchange and eoistemic progress’ . American Psychologist, , 40 : 29 – 39 .
   Web of Science ®Google Scholar
• Margolis , J. 1967 . ‘Citation indexing and evaluation of scientific papers’ . Science, , 155 : 1213 – 1219 .
   PubMed Web of Science ®Google Scholar
• Martin , B. R. Establishing science policies and priorities—the role of foresight. Paper presented to the International Seminar on Priority Setting in Science and Technology . October 18–20 1994 , Wellington, New Zealand. SPRU, University of Sussex .
   Google Scholar
• Martin , B. R. and Irvine , J. 1983a . ‘Assessing basic research. Some partial indicators of scientific progress in radio astronomy’ . Research Policy, , 12 : 61 – 90 .
   Web of Science ®Google Scholar
• Martin , B. R. and Irvine , J. 1983b . ‘Internal criteria for scientific choice. An evaluation of research in high‐energy physics using electron accelerators’ . Minerva, , 19 : 409 – 432 .
   Web of Science ®Google Scholar
• Martin , B. R. and Irvine , J. 1983c . Assessing basic research: the case of the Isaac Newton telescope . Social Studies of Science, , 13 : 49 – 86 .
   Web of Science ®Google Scholar
• Martin , B. R. , Skea , J. E. F. and Ling , E. N. 1992 . “ Performance indicators for academic scientific research ” . In Report submitted to the Advisory Board of the Research Councils and to the Economic and Social Research Council , SPRU, University of Sussex .
   Google Scholar
• Matter , G. A. and Broms , H. 1983 . ‘The myth of Garfield and citation indexing’ . Tidskrift för Dokumentation, , 39 : 1 – 8 .
   Google Scholar
• Matthews , K. A. , Helmreich , R. L. , Beane , W. E. and Lucker , G. W. 1980 . ‘Pattern A, achievement striving, and scientific merit: does pattern A help or hinder?’ . Journal of Personality and Social Psychology, , 39 : 962 – 967 .
   PubMed Web of Science ®Google Scholar
• Mazlish , B. 1982 . ‘The quality of “The quality of science”: an evaluation’ . Science, Technology, and Human Values, , 7 : 42 – 52 .
   Google Scholar
• McClelland , D. C. 1963 . “ ‘The calculated risk: an aspect of scientific performance’ ” . In Scientific Creativity: Its Recognition and Development, , Edited by: Taylor , C. W. and Barron , F. 139 – 152 . New York : John Wiley .
   Google Scholar
• McGuire , W.J. 1989 . “ A perspectivist approach to the strategic planning of programmatic scientific research ” . In Psychology of Science. Contributions to Metascience, , Edited by: Gholson , B. , Shadish , W. R. Jr. , Neimeyer , F. A. and Houts , A. C. Cambridge : Cambridge University Press .
   Google Scholar
• McMullin , E. 1983 . “ ‘Values in science’ ” . In Philosophy of Science Association 1982 (2) , Edited by: Asquith , P. and Nickles , T. 3 – 28 . East Lansing, MI : Philosophy of Science Association .
   Google Scholar
• Merton , R. K. 1938 . Science, Technology and Soc ety in Seventeenth Century England, , New York : Harper & Row .
   Google Scholar
• Merton , R. K. 1973 . “ ‘"Recognition” and “excellence”: instructive ambiguities’ ” . In The Sociology of Science. Theoretical and Empirical Investigations, , Edited by: Storer , N. 419 – 459 . Chicago : The University of Chicago Press .
   Google Scholar
• Mitroff , I. I. 1974 . ‘Norms and counter‐norms in a select group of the Apollo moon scientists: a case study of the ambivalence of scientists’ . American Sociological Review, , 39 : 579 – 595 .
   Web of Science ®Google Scholar
• Mok , A. and Westerdiep , A. 1974 . “ ‘Societal influences on the choice of research topics of biologists’ ” . In Social Processes of Scientific Development, , Edited by: Whitley , R. 211 – 223 . London : Routledge & Kegan Paul .
   Google Scholar
• Moravcsik , M.J. and Murugesan , P. 1975 . ‘Some results on the function and quality of citations’ . Social Studies of Science, , 5 : 86 – 92 .
   Web of Science ®Google Scholar
• Mulkay , M. 1979 . Science and the Sociology of Knowledge, , London : George Allen & Unwin .
   Google Scholar
• Myers , C. R. 1970 . ‘Journal citations and scientific eminence in contemporary psychology’ . American Psychologist, , 25 : 1941 – 1948 .
   Web of Science ®Google Scholar
• Narin , F. 1987 . ‘Citation analysis. Bibliometric techniques in the evaluation of research programs’ . Science and Public Policy, , 14 : 99 – 106 .
   Google Scholar
• National Audit Bureau (Riksrevisionsverket, RRV) . 1982 . Styrning av högskolans forskning (Directing university research) Stockholm (Dnr 1982: 189)
   Google Scholar
• National Science Board (NSB) . 1991 . Science & Engineering Indicators—1991, , 10th edn , Washington : NSB .
   Google Scholar
• Nederhof , A. J. 1988 . “ ‘The validity and reliability of evaluation of scholarly performance’ ” . In Handbook of Quantitative Studies of Science and Technology, , Edited by: Van Raan , A. F. J. 193 – 228 . Amsterdam : Elsevier .
   Google Scholar
• Nederhof , A.J. and Van Raan , A. F.J. 1987 . ‘Peer review and bibliometric indicators of scientific performance: a comparison of cum laude doctorates with ordinary doctorates in physics’ . Scientometrics, , 11 : 333 – 350 .
   Web of Science ®Google Scholar
• Nelson , D. M. 1977 . Methods of citation analysis in the Fine Arts . Special Libraries, , 68 : 390 – 395 .
   Google Scholar
• Niiniluoto , I. 1990 . ‘Science and epistemic values’ . Science Studies, , 3 : 21 – 25 .
   Google Scholar
• Nilstun , T. 1988 . Expertbedömningar. Om teori, ideal och verklighet vid utvärdering av sektors FoU (Peer reviews. On theories, ideals, and reality when evaluating sectoral research) , BVN skriftserie 1988: 1 Stockholm : The Scientific Advisory Board of the Swedish Council for Building Research .
   Google Scholar
• Noble , J. H. Jr. 1974 . ‘Peer review: quality control of applied social research’ . Science, , 185 : 916 – 921 .
   PubMed Web of Science ®Google Scholar
• Persson , O. Produktion och inflytande hos en grupp svenska psykologer. En bibliometrisk metodstudie (Production and influence of a group of Swedish psychologists. A bibliometric methodological study) , Inforsk. Report no. 17 1982 : Department of Sociology, University of Umeå .
   Google Scholar
• Peters , D. P. and Ceci , S. J. 1982 . ‘Peer‐review practices of psychological journals: the fate of published articles, submitted again’ . The Behavioral and Brain Sciences, , 5 : 187 – 195 .
   Web of Science ®Google Scholar
• Porter , A. L. 1977 . ‘Citation analysis: queries and caveats’ . Social Studies of Science, , 7 : 257 – 267 .
   Web of Science ®Google Scholar
• Premfors , R. 1986 . Forskningsmiljön i högskolan. En kunskapsöversikt (The research environment in universities. A literature review) , Report no. 36 Department of Political Science, University of Stockholm .
   Google Scholar
• Presser , S. 1980 . ‘Collaboration and the quality of research’ . Social Studies of Science, , 10 : 95 – 101 .
   Web of Science ®Google Scholar
• Rodgers , R. C. and Maranto , C. L. 1989 . ‘Causal models of publishing productivity in psychology’ . Journal of Applied Psychology, , 74 : 636 – 649 .
   Web of Science ®Google Scholar
• Rossi , P. H. and Freeman , H. E. 1993 . Evaluation. A Systematic Approach, , London : Sage .
   Google Scholar
• Roy , R. 1985 . ‘Commentary: peer review is alive and well in the United States’ . Science, Technology, & Human Values, , 10 : 73 – 81 .
   Google Scholar
• Rudd , E. 1977 . ‘The effect of alphabetic order of author listing on the careers of scientists’ . Social Studies of Science, , 7 : 268 – 269 .
   Web of Science ®Google Scholar
• Rudd , E. 1983 . The problem of allocation of funds for university research and the assessment of its quality , Colchester : University of Essex .
   Google Scholar
• Rudd , E. 1988 . ‘The evaluation of the quality of research’ . Studies in Higher Education, , 13 (19) : 45 – 57 .
   Web of Science ®Google Scholar
• Rushton , J. P. 1984 . ‘Evaluating research eminence in psychology: the construct validity of citation counts’ . Bulletin of the British Psychological Society, , 37 : 33 – 36 .
   Google Scholar
• Rushton , J. P. and Endler , N. S. 1977 . The scholarly impact and research productivity of departments of psychology in the United Kingdom . Bulletin of the British Psychological Society, , 30 : 369 – 373 .
   Google Scholar
• Rushton , J. P. , Murray , H. G. and Paunonen , S. V. 1983 . ‘Personality, research creativity, and teaching effectiveness in university professors’ . Scientometrics, , 5 : 93 – 116 .
   Web of Science ®Google Scholar
• Schaeffer , D. L. and Sulyma , I. M. 1979a . ‘Productivity in publication and the quality of psychology departments’ . American Psychologist, , 34 : 441 – 443 .
   Web of Science ®Google Scholar
• Schaeffer , D. L. and Sulyma , I. M. 1979b . ‘Citation rates and the quality of Canadian psychology departments’ . Canadian Psychology Review/Psychology Canadienne, , 20 : 22 – 37 .
   Google Scholar
• Schubert , A. and Braun , T. 1993 . ‘Reference standards for citation based assessments’ . Scientometrics, , 26 (1) : 21 – 35 .
   Web of Science ®Google Scholar
• Seglen , P.O. 1992a . “ Evaluations of scientists by journal impact ” . In Representations of Science and Technology. Proceedings of the International Conference on Science and Technology Indicators, Bielefeld, Federal Republic of Germany, 10–12 June 1990, , Edited by: Weingart , P. , Sehringer , R. and Winterhager , M. 240 – 252 . Leiden University : DSWO Press .
   Google Scholar
• Seglen , P. O. 1992b . ‘The skewness of science’ . Journal of the American Society for Information Science, , 43 (9) : 628 – 638 .
   Web of Science ®Google Scholar
• Seglen , P. O. 1994 . ‘Causal relationship between article citedness and journal impact’ . Journal of the American Society for Information Science, , 45 (1) : 1 – 11 .
   Web of Science ®Google Scholar
• Shadish , W. R. Jr. 1989 . “ ‘Perception and evaluation of quality in science’ ” . In Psychology of Science. Contributions to Metascience, , Edited by: Gholson , B. , Shadish , W. R. Jr. , Neimeyer , R. A. and Houts , A. C. Cambridge : Cambridge University Press .
   Google Scholar
• Shaw , B. T. 1967 . The Use of Quality and Quantity of Publication as Criteriafor Evaluating Scientists. , Washington, DC : Agricultural Research Service, United States Department of Agriculture .
   Google Scholar
• Sigelmann , L. and Whicker , M. 1987 . ‘Some implications of bias in peer review: a simulation‐based analysis’ . Social Science Quarterly, , 68 : 494 – 509 .
   Web of Science ®Google Scholar
• Simonton , D.K. 1988 . ‘Age and outstanding achievement: what do we know after a century of research ?’ . Psychological Bulletin, , 104 : 251 – 267 .
   PubMed Web of Science ®Google Scholar
• Smigel , E. O. and Ross , H. L. 1970 . ‘Factor in the editorial decision’ . The American Sociologist, , 5 : 19 – 21 .
   Google Scholar
• StÅhle , B. , ed. Evaluation of Research. Nordic Experiences. Proceedings of a Nordic workshop in Saltsjöbaden . September 3–5 1986 , Sweden. Nordic Science Policy Council, Copenhagen . (FPR Publication no. 5)
   Google Scholar
• Stankiewicz , R. 1980 . “ Leadership and the performance of research groups ” . Lund : RPI Research Policy Institute, University of Lund. Studentlitteratur . Doctoral dissertation
   Google Scholar
• Swedish Board of Universities and Colleges (Universitets‐ och högskoleämbetet, UHÄ) (1983), Utvärdering av forskning (Evaluation of research). Reg. no. 459–3797–83, Stockholm
   Google Scholar
• Terman , L. W. 1955 . ‘Are scientists different?’ . Scientific American, , 192 : 25 – 29 .
   Web of Science ®Google Scholar
• Törnebohm , H. 1983 . Studier av kunskapsutveckling (Studies of the development of knowledge) , Lund : Doxa .
   Google Scholar
• Toulmin , S. 1964 . ‘The complexity of scientific choice: stocktaking’ . Minerva , 2 : 343 – 359 .
   Web of Science ®Google Scholar
• Van Raan , A. F. J. , ed. 1988 . Handbook of Quantitative Studies of Science and Technology , Amsterdam : North‐Holland .
   Google Scholar
• Vinkler , P. 1987 . ‘A quasi‐quantitative citation model’ . Scientometrics, , 12 : 47 – 72 .
   Web of Science ®Google Scholar
• Wade , N. 1975 . ‘Citation analysis: a new tool for science administrators’ . Science, , 188 2 May : 429 – 432 .
   PubMed Web of Science ®Google Scholar
• Wallmark , J. T. and Sedig , K. G. 1986 . ‘Quality of research measured by citation method and by peer review—a comparison’ . IEEE Transactions on Engineering Management, , 33 : 218 – 222 .
   Web of Science ®Google Scholar
• Wallmark , J. T. , Eckerstein , S. , Langered , B. and Holmqvist , H, E. S. 1973 . ‘The increase in efficiency with size of research teams’ . IEEE Transactions on Engineering Management, , 20 : 80 – 86 .
   Web of Science ®Google Scholar
• Weinberg , A. M. 1963 . ‘Criteria for scientific choice’ . Minerva, , 1 : 159 – 171 .
   Web of Science ®Google Scholar
• Weingart , P. , Sehringer , R. and Winterhager , M. , eds. Representations of Science and Technology . Proceedings of the International Conference on Science and Technology Indicators . June 10–12 1990 , Bielefeld, Federal Republic of Germany. DSWO Press, Leiden University .
   Google Scholar
• Westbrook , J. H. 1960 . ‘Identifying significant research’ . Science, , 132 28 October : 1229 – 1234 .
   PubMed Web of Science ®Google Scholar
• Zusne , L. 1976 . ‘Age and achievement in psychology: the harmonic mean as a model’ . American Psychologist, , 31 : 805 – 807 .
   Web of Science ®Google Scholar