Research@Lincoln
    • Login
     
    View Item 
    •   Research@Lincoln Home
    • Faculty of Environment, Society and Design
    • Department of Informatics and Enabling Technologies
    • View Item
    •   Research@Lincoln Home
    • Faculty of Environment, Society and Design
    • Department of Informatics and Enabling Technologies
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Robust statistical methods for empirical software engineering

    Kitchenham, B.; Madeyski, L.; Budgen, D.; Keung, J.; Brereton, P.; Charters, Stuart; Gibbs, Shirley F.; Pohthong, A.
    Abstract
    © 2016 The Author(s) There have been many changes in statistical theory in the past 30 years, including increased evidence that non-robust methods may fail to detect important results. The statistical advice available to software engineering researchers needs to be updated to address these issues. This paper aims both to explain the new results in the area of robust analysis methods and to provide a large-scale worked example of the new methods. We summarise the results of analyses of the Type 1 error efficiency and power of standard parametric and non-parametric statistical tests when applied to non-normal data sets. We identify parametric and non-parametric methods that are robust to non-normality. We present an analysis of a large-scale software engineering experiment to illustrate their use. We illustrate the use of kernel density plots, and parametric and non-parametric methods using four different software engineering data sets. We explain why the methods are necessary and the rationale for selecting a specific analysis. We suggest using kernel density plots rather than box plots to visualise data distributions. For parametric analysis, we recommend trimmed means, which can support reliable tests of the differences between the central location of two or more samples. When the distribution of the data differs among groups, or we have ordinal scale data, we recommend non-parametric methods such as Cliff’s δ or a robust rank-based ANOVA-like method.... [Show full abstract]
    Keywords
    empirical software engineering; statistical methods; robust methods; robust statistical methods; Software Engineering
    Fields of Research
    080309 Software Engineering; 0104 Statistics; 010405 Statistical Theory; 010406 Stochastic Analysis and Modelling
    Date
    2016-06-16
    Type
    Journal Article
    Collections
    • Department of Informatics and Enabling Technologies [119]
    Thumbnail
    View/Open
    KitchenhamMadeyskiBudgenKeungBreretonChartersGibbsPohthongRobuststatistical2016.pdf
    Share this

    on Twitter on Facebook on LinkedIn on Reddit on Tumblr by Email

    DOI
    https://doi.org/10.1007/s10664-016-9437-5
    Metadata
     Expand record
    © The Author(s) 2016. This article is published with open access at Springerlink.com
    Citation
    Kitchenham et al. (2016). Robust statistical methods for empirical software engineering. Empirical Software Engineering, 1-52. doi 10.1007/s10664-016-9437-5
    This service is maintained by Learning, Teaching and Library
    • Archive Policy
    • Copyright and Reuse
    • Deposit Guidelines and FAQ
    • Contact Us
     

     

    Browse

    All of Research@LincolnCommunities & CollectionsTitlesAuthorsKeywordsBy Issue DateThis CollectionTitlesAuthorsKeywordsBy Issue Date

    My Account

    LoginRegister

    Statistics

    View Usage Statistics
    This service is maintained by Learning, Teaching and Library
    • Archive Policy
    • Copyright and Reuse
    • Deposit Guidelines and FAQ
    • Contact Us