1,375
Views
1
CrossRef citations to date
0
Altmetric
COMPUTER SCIENCE

An initial investigation of the effect of quality factors on Agile test case quality through experts’ review

ORCID Icon, & | (Reviewing editor)
Article: 2082121 | Received 16 Jun 2021, Accepted 15 May 2022, Published online: 06 Jun 2022

References

  • Aamir, M., & Khan, M. N. A. (2017). Incorporating quality control activities in scrum in relation to the concept of test backlog. Sādhanā, 42(7), 1051–26. https://doi.org/10.1007/s12046-017-0688-7
  • Adlemo, A., Tan, H., & Tarasov, V. (2018). Test case quality as perceived in Sweden. 5th International Workshop on Requirements Engineering and Testing (RET’18), 9–12. https://doi.org/10.1145/3195538.3195541
  • Afzal, W. (2007). Metrics in software test planning and test design processes. Issue January. Blekinge Institute of Technology.
  • Ahmad, A., Leifler, O., & Sandahl, K. (2019). Empirical analysis of factors and their effect on test flakiness-practitioners’ perceptions. ArXiv Preprint ArXiv:1906.00673. https://doi.org/10.48550/arXiv.1906.00673
  • Ahmed, I., Gopinath, R., Brindescu, C., Groce, A., & Jensen, C. (2016). Can testedness be effectively measured? Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, 547–558.
  • Ali, M. J. (2006). Metrics for requirements engineering. UMEA University.
  • Allala, S. C., Sotomayor, J. P., Santiago, D., King, T. M., & Clarke, P. J. (2019). Towards transforming user requirements to test cases using MDE and NLP. 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC), 350–355. https://doi.org/10.1109/COMPSAC.2019.10231
  • Athanasiou, D., Nugroho, A., Visser, J., & Zaidman, A. (2014). Test code quality and its relation to issue handling performance. IEEE Transactions on Software Engineering, 40(11), 1100–1125. https://doi.org/10.1109/TSE.2014.2342227
  • Atmadja, M., & Shuai, R. Z. (2011). Chapter 3: Testing. In D. C. Rajapakse (Ed.), A fresh graduate’s guide to software development tools and technologies (pp. 1–21). School of Computing, National University of Singapore. https://www.comp.nus.edu.sg/~seer/book/2e/
  • Aziz, Y. (2017). Exploring a keyword driven testing framework: a case study at Scania IT. Uppsala Universitet.
  • Ball, T., Kim, J.-M., Porter, A. A., & Siy, H. P. (1997). If your version control system could talk. ICSE Workshop on Process Modelling and Empirical Studies of Software Engineering, 11.
  • Barraood, S. O., Mohd, H., Baharom, F., & Omar, M. (2018). Quality factors of test cases: a systematic literature review. Knowledge Management International Conference (KMICe), Universiti Utara Malaysia.
  • Barraood, S. O., Mohd, H., & Baharom, F. (2021a). Test case quality factors: content analysis of software testing websites. Webology, 18(Special Issue on Artificial Intelligence in Cloud Computing), 75–87. https://doi.org/10.14704/WEB/V18SI01/WEB18007
  • Barraood, S. O., Mohd, H., & Baharom, F. (2021b). Test case quality factors. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12(3), 1683–1694. https://doi.org/10.17762/turcomat.v12i3.990
  • Barraood, S. O., Mohd, H., & Baharom, F. (2021c). Test case quality: issues and limitations in Agile software development. Knowledge Management International Conference (KMICe) 2021c, February 1 2021, 80–87. http://www.kmice.cms.net.my/
  • Beck, K. (2003). Test-driven development: By example. Addison-Wesley Professional.
  • Beer, A., Junker, M., Femmer, H., & Felderer, M. (2017). Initial investigations on the influence of requirement smells on test-case design. 2017 IEEE 25th International Requirements Engineering Conference Workshops (REW), 323–326.
  • Beer, A., & Felderer, M. (2018). Measuring and improving testability of system requirements in an industrial context by applying the goal question metric approach. 5th International Workshop on Requirements Engineering and Testing Measuring, 25–32.
  • Beniwal, R. (2015). Analysis of testing metrics for object oriented applications. 2015 IEEE International Conference on Computational Intelligence & Communication Technology, 41–46. https://doi.org/10.1109/CICT.2015.34
  • Berberyan, H., & Ali, S. (2019, November). Automation regression testing for Sas.Am website. In CS & IT Conference Proceedings (Vol. 9, pp. 13). CS & IT Conference Proceedings.
  • Boghdady, P. N., Badr, N. L., Hashem, M., & Tolba, M. F. (2011). A proposed test case generation technique based on activity diagrams. International Journal of Engineering & Technology IJET-IJENS, 11(3), 37–57.
  • Bowes, D., Hall, T., Petrić, J., Shippey, T., & Turhan, B. (2017). How good are my tests? In International Workshop on Emerging Trends in Software Metrics, WETSoM (pp. 9–14). IEEE. https://doi.org/10.1109/WETSoM.2017.2
  • Bucaioni, A., Cicchetti, A., Ciccozzi, F., Kodali, M., & Sjödin, M. (2018). Alignment of requirements and testing in Agile: An industrial experience. In S. Latifi (Ed.), Information Technology-New Generations,Advances in Intelligent Systems and Computing 738 (pp. 225–232). Springer.
  • Causevic, A., Punnekkat, S., & Sundmark, D. (2012). Quality of test design in test driven development. 2012 Eighth International Conference on the Quality of Information and Communications Technology, 224. http://urn.kb.se/resolve?urn=urn:nbn:se:mdh:diva-18773
  • Causevic, A., Sundmark, D., & Punnekkat, S. (2012). Test case quality in test driven development : A study design and a pilot experiment. The EASE 2012, 223–227. https://doi.org/10.1049/ic.2012.0029
  • Chauhan, N. (2010). Software testing: principles and practices. Oxford university press.
  • Chernak, Y. (2001). Validating and improving test-case effectiveness. IEEE Software, 18(1), 81–86. https://doi.org/10.1109/52.903172
  • Collins, E., Dias-Neto, A., & de Lucena, V. F., Jr. (2012). Strategies for Agile software testing automation : An industrial experience. Computer software and Applications Conference Workshops (COMPSACW), 2012 IEEE 36th Annual, 440–445.
  • Craig, R. D., & Jaskiel, S. P. (2002). Systematic software testing. Artech House.
  • Crispin, L., & Gregory, J. (2009). Agile testing: A practical guide for testers and agile teams (1st ed.). Pearson Education, Inc.
  • Daka, E., & Fraser, G. (2014). A survey on unit testing practices and problems. 2014 IEEE 25th International Symposium on Software Reliability Engineering, 201–211.
  • Do, H. (2016). Recent Advances in Regression Testing Techniques. Advances in Computers, 103, 53–77. https://doi.org/10.1016/bs.adcom.2016.04.004
  • Eldh, S., Hansson, H., & Punnekkat, S. (2011). Analysis of mistakes as a method to improve test case design. Fourth IEEE International Conference on Software Testing, Verification and Validation Analysis, 70–79. https://doi.org/10.1109/ICST.2011.52
  • Fischbach, J., Femmer, H., Mendez, D., Fucci, D., & Vogelsang, A. (2020). What makes agile test artifacts useful? An activity-based quality model from a practitioners’ perspective. ESEM, ’20. https://arxiv.org/abs/2009.01722
  • Garousi, V., Felderer, M., Kuhrmann, M., & Herkilo, K. (2017). What industry wants from academia in software testing ? Hearing practitioners ’ opinions. EASE’17, 65–69. https://doi.org/10.1145/3084226.3084264
  • Gay, G., Rajan, A., Staats, M., Whalen, M., & Heimdahl, M. P. E. (2016). The effect of program and model structure on the effectiveness of mc/dc test adequacy coverage. ACM Transactions on Software Engineering and Methodology (TOSEM), 25(3), 1–34. https://doi.org/10.1145/2934672
  • Goeschl, S., Herp, M., & Wais, C. (2010). When agile meets OO testing: A case study. Proceedings of the 1st Workshop on Testing Object-Oriented Systems, 10.
  • Gómez, O. S., Monte, B., & Monte, B. (2016). Impact of CS programs on the quality of test cases generation : An empirical study categories and subject descriptors. ICSE ’16 Companion, 374–383. https://doi.org/10.1145/2889160.2889190
  • Grano, G., Scalabrino, S., Gall, H. C., & Oliveto, R. (2018). An empirical investigation on the readability of manual and generated test cases. Proceedings of the 26th Conference on Program Comprehension, 348–351.
  • Grano, G. (2019). A new dimension of test quality : Assessing and generating higher quality unit test cases. The 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA ’19), July 15–19. https://doi.org/10.1145/3293882.3338984
  • Gupta, N., Yadav, V., & Singh, M. (2018). Automated regression test case generation for web application: A survey. ACM Computing Surveys (CSUR), 51(4), 87. https://doi.org/10.1145/3232520
  • Hallowell, M. R., & Gambatese, J. A. (2009). Qualitative research: Application of the Delphi method to CEM research. Journal of Construction Engineering and Management, 136(1), 99–107. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000137
  • Harichandan, S., Panda, N., & Acharya, A. A. (2014). Scrum testing with backlog management in Agile development environment. International Journal of Computer Science and Engineering, 2(3), 187–192. http://www.ijcseonline.org/pub_paper/38-IJCSE-00144.pdf
  • Hayhurst, K. J. (2001). A practical tutorial on modified condition/decision coverage. DIANE Publishing.
  • Horváth, F., Vancsics, B., Vidács, L., & Beszédes, Á. (2015). Test suite evaluation using code coverage based metrics. SPLST’15, 46–60.
  • Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation (Adobe reader). Pearson Education.
  • Hussain, A., Mkpojiogu, E. O. C., & Kamal, F. M. (2016). The role of requirements in the success or failure of software projects. International Review of Management and Marketing, 6(S7), 306–311.
  • Hynninen, T., Kasurinen, J., Knutas, A., & Taipale, O. (2018). Software testing: Survey of the industry practices. 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), 1449–1454. https://doi.org/10.23919/MIPRO.2018.8400261
  • IEEE 829. (2008). IEEE Standard for Software and System Test Documentation. IEEE Computer Society.
  • ISO-IEC 25010:2011. (2011). ISO-IEC 25010: 2011. In Systems and software engineering-systems and software quality requirements and evaluation (SQuaRE)-System and software quality models. ISO. https://doi.org/10.3403/30215101
  • ISO/IEC 9126-1. 2000. Information technology — Software product quality — Part 1: Quality model. In Iso/Iec Fdis 9126-1Vol. 2000 1–26.ISO; IEC. http://www.cse.unsw.edu.au/~cs3710/PMmaterials/Resources/9126-1Standard.pdf
  • ISO/IEC25010. (2011). ISO/IEC25010: 2011 Systems and software engineering–Systems and software Quality Requirements and Evaluation (SQuaRE)–System and software quality models. International Organization for Standardization. https://www.iso.org/obp/ui/#iso:std:iso-iec:25010:ed-1:v1:en
  • ISO/IEC25023. (2016). INTERNATIONAL STANDARD ISO/IEC 25023. 2016.
  • Javed, K., Khan, A. H., & Tubbassum, L. (2019). Critical analysis of software development methodologies based on project risk management. International Journal of Academic Research in Business and Social Sciences, 9(12), 248–261. http://dx.doi.org/10.6007/IJARBSS/v9-i12/6720
  • Jovanovikj, I., Narasimhan, V., Engels, G., & Sauer, S. (2018). Context-specific quality evaluation of test cases. Modelsward, 594–601. https://doi.org/10.5220/0006724405940601
  • Juan, Z., Lizhi, C., Weiqing, T., & Song, Y. (2010). Test case reusability metrics model. Icctd, 294–298. https://doi.org/10.1109/ICCTD.2010.5645869
  • Juhnke, K., Tichy, M., & Houdek, F. (2018). Quality Indicators for Automotive Test Case Specifications. SEERTS 2018: Workshop on Software Engineering for Applied Embedded RealTime Systems @ SE18, 96–100.
  • Juhnke, K., Tichy, M., & Houdek, F. (2021). Challenges concerning test case specifications in automotive software testing: Assessment of frequency and criticality. Software Quality Journal, 29(1), 39–100. https://doi.org/10.1007/s11219-020-09523-0
  • Kamde, P. M., Nandavadekar, V. D., & Pawar, R. G. (2006). Value of test cases in software testing. Management of Innovation and Technology, 2006 IEEE International Conference On, 2, 668–672. https://doi.org/10.1109/ICMIT.2006.262303
  • Kaner, C. (2003). What is a good test case ? Software Testing Analysis & Review Conference (STAR) East., 1–16.
  • Kao, C. H., Lin, C. C., & Lu, H. T. (2016). Toward automatic performance testing for rest-based web applications. The Eleventh International Conference on Software Engineering Advances-ICSEA, 81–87.
  • Kārkliņa, K., & Pirta, R. (2018). Quality metrics in Agile software development projects (pp. 21). Information Technology & Management Science (RTU Publishing House).
  • Kayes, I., Sarker, M., & Chakareski, J. (2016). Product backlog rating : A case study on measuring test quality in scrum. Innovations in Systems and Software Engineering, 12(4), 303–317. https://doi.org/10.1007/s11334-016-0271-0
  • Kochhar, P. S., Xia, X., & Lo, D. (2019). Practitioners’ views on good software testing practices. 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), 61–70. https://doi.org/10.1109/ICSE-SEIP.2019.00015
  • Kumar, S., & Ranjan, P. (2017). A phase wise approach for fault identification. Journal of Information and Optimization Sciences, 39(1), 223–237. https://doi.org/10.1080/02522667.2017.1380420
  • Lai, S.-T. (2017). Test case quality management procedure for enhancing the efficiency of IID continuous testing. Journal of Software, 12(10), 794–806. https://doi.org/10.17706/jsw.12.10.794-806
  • Lai, S.-T. (2019). Applying continuous integration for increasing the maintenance quality and efficiency of web app. International Journal of Software Engineering & Applications (IJSEA), 10(1), 37–50. https://doi.org/10.5121/ijsea.2019.10103
  • Leffingwell, D. (2010). Agile software requirements: Lean requirements practices for teams, programs, and the enterprise. Addison-Wesley Professional.
  • Lewis, W. E. (2009). Software testing and continuous quality improvement (3rd ed.). Taylor & Francis Group.
  • Li, B., Vendome, C., Linares-Vásquez, M., Poshyvanyk, D., & Kraft, N. A. (2016). Automatically documenting unit test cases. 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST), 341–352.
  • Limaye, M. G. (2009). Software testing. Tata McGraw-Hill Education.
  • Lin, C., Tang, K., & Kapfhammer, G. M. (2014). Test suite reduction methods that decrease regression testing. Information And Software Technology, 56(10), 1322–1344. https://doi.org/10.1016/j.infsof.2014.04.013
  • Lynn, M. R. (1986). Determination and quantification of content validity. Nursing Research.
  • Mannion, M., & Keepence, B. (1995). SMART requirements. ACM SIGSOFT Software Engineering Notes, 20(2), 42–47. https://doi.org/10.1145/224155.224157
  • Matharu, G. S., Mishra, A., Singh, H., & Upadhyay, P. (2015). Empirical study of agile software development methodologies: A comparative analysis. ACM SIGSOFT Software Engineering Notes, 40(1), 1–6. https://doi.org/10.1145/2693208.2693233
  • Meszaros, G., Smith, S. M., & Andrea, J. (2003). The test automation manifesto. Conference on Extreme Programming and Agile Methods, 73–81.
  • Meszaros, G. (2007). xUnit test patterns: Refactoring test code. Pearson Education.
  • Mohamed, S. F. P. (2015). A process based approach software certification model for agile and secure environment. Universit Utara Malaysia.
  • Mohi-aldeen, S. M., Deris, S., & Mohamad, R. (2014). Systematic mapping study in automatic test case generation. IOS Press, 703–720. https://doi.org/10.3233/978-1-61499-434-3-703
  • Nawaz, A., & Malik, K. M. (2008). Software testing process in agile development. In Computer Science Master Thesis, Blekinge Institute of Technology. Citeseer.
  • Olausson, M., Rossbreg, J., Ehn, J., & Sköld, M. (2013). Pro team foundation service. In illustrate, Apress (pp. 404). Apress.
  • Parveen, T., Tilley, S., & Gonzalez, G. (2007). A case study in test management. Proceedings of the 45th Annual Southeast Regional Conference, 82–87.
  • Penmetsa, J. R. (2017). Agile testing. In H. Mohanty, J. R. Mohanty, A. K. Balakrishnan (Eds.), Trends in software testing (pp. 19–33). Springer Science+Business Media.https://doi.org/10.1007/978-981-10-1415-2
  • Petunova, O., & Bērziša, S. (2017). Test case review processes in software testing. Information Technology and Management Science, 20(1), 48–53. https://doi.org/10.1515/itms-2017-0008
  • Pfaller, C., Wagner, S., Universit, T., & Wiemann, M. (2008). Multi-dimensional measures for test case quality. Software Testing Verification and Validation Workshop, 2008. ICSTW’08. IEEE International Conference On, 364–368.
  • Pressman, R. S. (2005). Software engineering: A practitioner’s approach. Palgrave Macmillan.
  • Pressman, R. S. (2010). CS605-Software Engineering Practitioner’s Approach. https://doi.org/10.1017/CBO9781107415324.004
  • Qamar, N., & Malik, A. A. (2019). Evaluating the impact of pair testing on team productivity and test case quality – A controlled experiment. Pakistan Journal of Engineering and Applied Sciences, 25(7), 80–88. https://journal.uet.edu.pk/ojs_old/index.php/pjeas/article/view/1766/328
  • Quadri, S. M. K., & Farooq, S. U. (2010). Software testing – goals, principles, and limitations. International Journal of Computer Applications, 6(9), 7–10. https://doi.org/10.5120/1343-1448
  • Rajaram, H. K., Loane, J., Macmahon, S. T., & Caffery, F. M. (2021). Expert review of taxonomy based testing : A testing framework for medical device software. The 16th International Conference on Evaluation OfNovel Approaches to Software Engineering (ENASE 2021), Enase, 331–339. https://doi.org/10.5220/0010458503310339
  • Rajasekhar, P., & Shafi, R. M. (2014). Agile software development and testing: Approach and challenges in advanced distributed systems. Global Journal of Computer Science and Technology, 14(1). http://computerresearch.org/index.php/computer/article/view/38
  • Rava, M., & Daengdej, A. P. D. J. (2014). Measuring test case reusability based on simplicity and Independence. ACIT 2014, 66–71. https://doi.org/10.3850/978-981-07-8859-9_15
  • Rogers, M. R., & Lopez, E. C. (2002). Identifying critical cross-cultural school psychology competencies. Journal of School Psychology, 40(2), 115–141. https://doi.org/10.1016/S0022-4405(02)00093-6
  • Romli, R., Sarker, S., Omar, M., & Mahmod, M. (2020). Automated test cases and test data generation for dynamic structural testing in automatic programming assessment using MC/DC. International Journal on Advanced Science, Engineering and Information Technology, 10(1), 120. https://doi.org/10.18517/ijaseit.10.1.10166
  • Salamah, S., Gates, A., Roach, S., & Mondragon, O. (2005). Verifying pattern-generated LTL formulas: A case study. International SPIN Workshop on Model Checking of Software, 200–220.
  • Sanaa, H., Afifi, W., & Darwish, N. (2016). The goal questions metrics for Agile business intelligence. Egyptian Computer Science Journal, 40(2), 24–42.
  • Schedlbauer, M. (2012). Testing web applications (version 1.). Northeastern University, CCIS.
  • Setiani, N., Ferdiana, R., & Hartanto, R. (2020). Test case understandability model. IEEE Access, 8, 169036–169046. https://doi.org/10.1109/ACCESS.2020.3022876
  • Shamshiri, S., Rojas, J. M., Galeotti, J. P., Walkinshaw, N., & Fraser, G. (2018). How do automatically generated unit tests influence software maintenance? 2018 IEEE 11th international conference on software testing, verification and validation (ICST), 250–261.
  • Shrivastava, D. P., & Jain, R. C. (2010). Metrics for test case design in test driven development. International Journal of Computer Theory and Engineering, 2(6), 952–956. https://doi.org/10.7763/IJCTE.2010.V2.269
  • Sundmark, D., Pettersson, A., & Thane, H. (2005). Regression testing of multi-tasking real-time systems : A problem statement. ACM SIGBED Review, 2(2), 31–34. https://doi.org/10.1145/1121788.1121798
  • Tran, H. K. V., Ali, N. B., Börstler, J., & Unterkalmsteiner, M. (2019). Test-case quality–understanding practitioners’ perspectives. International Conference on Product-Focused Software Process Improvement, 37–52.
  • Tran, H. K. V., Unterkalmsteiner, M., Börstler, J., & Bin Ali, N. (2021). Assessing test artifact quality—A tertiary study. Information and Software Technology, 106620, 1–27. https://doi.org/10.1016/j.infsof.2021.106620
  • Tudjarova, S., Chorbev, I., & Joksimoski, B. (2017). Software quality metrics while using different development methodologies. International Conference on ICT Innovations, 240–250.
  • Waltz, C. F., Strickland, O. L., & Lenz, E. R. (2010). Measurement in nursing and health research. Springer publishing company.
  • Yadav, H. B., & Yadav, D. K. (2015). A fuzzy logic based approach for phase-wise software defects prediction using software metrics. Information and Software Technology, 63, 44–57. https://doi.org/10.1016/j.infsof.2015.03.001
  • Yamaura, T. (1998). How to design practical test cases. IEEE Software, 15(6), 30–36. https://doi.org/10.1109/52.730835
  • Yan, M., Xia, X., Zhang, X., Xu, L., Yang, D., & Li, S. (2019). Software quality assessment model: A systematic mapping study. Science China Information Sciences, 62(9), 1–18.
  • Yu, J. (2018). Design and application on agile software exploratory testing model. 2018 2nd IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), 2082–2088.
  • Zander-Nowicka, J., Mosterman, P. J., & Schieferdecker, I. (2008). Quality of test specification by application of patterns. PLoP08 - 15th Conference on Pattern Languages of Programs, Proceedings, January. https://doi.org/10.1145/1753196.1753200
  • Zeiss, B., Vega, D., Schieferdecker, I. K., Neukirchen, H., & Grabowski, J. (2007). Applying the ISO 9126 quality model to test specifications – exemplified for ttcn-3 test specifications. Software Engineering, 15(6), 231–242.
  • Zhu, H. (1995). Axiomatic assessment of control flow-based software test adequacy criteria. Software Engineering Journal, 10(5), 194–204. https://doi.org/10.1049/sej.1995.0025