• Image field 67

  • About this instrument: The initial structure is derived from an evaluation instrument for quantitative research[1] and amended to incorporate criteria identified by Glanzel (1996)[2]. This document is intended for quite different purposes than those presumed by Glanzel, whose concern it is to support standardization in methodology and terminology between bibliometric researchers. This tool is meant to support critical evaluation of bibliometric research for non-experts in this subspecialty area, with the purpose of practical application of the results within the framework of Evidence-based Library and Information Practice.

    I would be pleased to receive comments and suggestions for the improvement of this evaluative tool. I've added a comments box at the very end for that purpose. Please feel free to contact me at cperryman@mail.twu.edu.


    Carol Perryman, Ph.D.
    Assistant Professor, School of Library and Information Studies
    Texas Woman's University
    P.O. Box 425438
    Denton, TX 76204-5438
    Phone: (940) 898-2612
    Fax: (940) 898-2611
    Email: CPerryman@mail.twu.edu
    Research page: cperryman.com

    [1] Evaluation Tool for Quantitative Research Studies, published by the Health Care Research and Development Unit (HCPRDU), University of Salford, 2005. Available http://www.fhsc.salford.ac.uk/hcprdu/quantitative.htm

    [2] Glanzel W. (1996). The need for standards in bibliometric research and technology. Scientometrics, 35(2):167-176.


    I am most grateful to Lorie Kloda, editor of the Evidence Summaries section of the open access journal, Evidence Based Library and Information Practice (https://ejournals.library.ualberta.ca/index.php/EBLIP), for having shared this document with colleague Vincent Larivière, who in turn suggested additions.

    • Section 1: Basic Information 
    • Stated objective(s) of research: Is there a key question being asked by the authors?*
    • Section 2: Background information: building on knowledge 
    • Is the literature review relevant to the resources, study objectives, and methodology used for the present research?*
    • Are included materials up to date and comprehensive?
    • Do the authors appear to have evaluated the quality of the cited materials?
    • If a bibliographic database is used, have the authors considered published or unpublished evaluations of the database as a way to understand problems they may encounter?
    • Does the literature review clearly support the need for the present research?
    • Section 3: Data set: Acquisition and analysis 
    • Do exclusion/inclusion criteria seem relevant to the research question(s)? *
    • Are retrieval methods described in sufficient detail to replicate the process?*
    • Are retrieval methods appropriate to the question?*
    • Are the limitations of the data source(s) considered?*
    • Are sufficient examples (tables, figures, etc.) provided to help you understand the data handling processes?*
    • If there are discrepancies or contradictions in the data, are they accounted for (i.e., missing or incomplete information)?*
    • Are analysis or evaluation methods appropriate to the objectives?*
    • Is the process for organizing the data logical and clearly explained? *
    • If categories or themes are assigned as a way to group information about the data, are they
    • Section 4: Statistical analysis 
    • Is the method chosen for statistical analysis appropriate for the question and the data?*
    • Are the parameters for statistical significance established and explained? *
    • Are outliers (anomalies in the findings) discussed in terms of cause and effect?*
    • Section 5: Findings, contributions, and generalizability 
    • Does the study achieve its original objective(s)?*
    • If the study builds on prior research, does the present study validate, refute, or add to the earlier findings?*
    • If there are discrepancies or contradictions in the findings, are they discussed?*
    • Section 6: Your own conclusions 
    • Can the study by generalized beyond the setting or data examined in this study?*
    • Can the study be replicated using available information? *
    • Can the findings from this study be used in your setting?
    • Can the methods used in this study be used in your setting?

    • By clicking on 'Email document' below, you will be leaving the form, and won't be able to print your responses. If you want to print, select 'print form' before you email. Of course, you can also print the email.

    •  
    • Should be Empty: