A tuned feed-forward deep neural network algorithm for effort estimation

References

• Aquino, W., Bunting, G., Miller, S. T., & Walsh, T. F. (2019). A gradient-based optimization approach for the detection of partially connected surfaces using vibration tests. Computer Methods in Applied Mechanics and Engineering, 345(3), 323–335. https://doi.org/10.1016/j.cma.2018.11.002
   Web of Science ®Google Scholar
• Aravkin, A., Burke, J. V., Chiuso, A., & Pillonetto, G. (2011, December). Convex vs nonconvex approaches for sparse estimation: lasso, multiple kernel learning and hyperparameter lasso. In 2011 50th IEEE Conference on Decision and Control and European Control Conference pp.(156–161). Orlando, FL, USA: IEEE.
   Google Scholar
• Azzeh, M., Nassif, A. B., Banitaan, S., & Almasalha, F. (2016). Pareto efficient multi-objective optimization for local tuning of analogy-based estimation. Neural Computing & Applications, 27(8), 2241–2265. https://doi.org/10.1007/s00521-015-2004-y
   Web of Science ®Google Scholar
• Basgalupp, M. P., Barros, R. C., da silva, T. S., & de carvalho, A. C. 2013, March.Software effort prediction: A hyper-heuristic decision-tree based approach. In Proceedings of the 28th Annual ACM Symposium on Applied Computing, pp.1109–1116. https://doi.org/10.1145/2480362.2480571
   Google Scholar
• Benediktsson, O., Dalcher, D., Reed, K., & Woodman, M. (2003). COCOMO-based effort estimation for iterative and incremental software development. Software Quality Journal, 11(4), 265–281. https://doi.org/10.1023/A:1025809010217
   Web of Science ®Google Scholar
• Bengio, Y. (2000). Gradient-based optimization of hyperparameters. Neural Computation, 12(8), 1889–1900. https://doi.org/10.1162/089976600300015187
   PubMed Web of Science ®Google Scholar
• Bergstra, J., & Bengio, Y. (2012, February). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13(1), 281–305. https://doi.org/10.5555/2188385.2188395
   Web of Science ®Google Scholar
• Bergstra, J. S., Bardenet, R., Bengio, Y., & Kégl, B. (2011). Algorithms for hyper-parameter optimization. In Advances in Neural Information Processing Systems, (Vol. 24, pp.2546–2554). https://papers.nips.cc/paper/2011/hash/86e8f7ab32cfd12577bc2619bc635690-Abstract.html
   Google Scholar
• Burgess, C. J., & Lefley, M. (2001). Can genetic programming improve software effort estimation? A comparative evaluation. Information and Software Technology, 43(14), 863–873. https://doi.org/10.1016/S0950-5849(01)00192-6
   Web of Science ®Google Scholar
• Chan, S., Treleaven, P., & Capra, L. (2013, October). Continuous hyperparameter optimization for large-scale recommender systems. In 2013 IEEE International Conference on Big Data pp.(350–358). Santa Clara, CA, USA: IEEE.
   Google Scholar
• Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321–357. https://doi.org/10.1613/jair.953
   Web of Science ®Google Scholar
• Cockett, R., Kang, S., Heagy, L. J., Pidlisecky, A., & Oldenburg, D. W. (2015). SimPEG: an open source framework for simulation and gradient based parameter estimation in geophysical applications. Computers & Geosciences, 85, 142–154. https://doi.org/10.1016/j.cageo.2015.09.015
   Web of Science ®Google Scholar
• Dam, H. K., Pham, T., Ng, S. W., Tran, T., Grundy, J., Ghose, A., & Kim, C. J. (2018). A deep tree-based model for software defect prediction. arXiv Preprint arXiv:1802.00921. https://arxiv.org/pdf/1802.00921.pdf?source=post_page---------------------------
   Google Scholar
• Du, W. L., Ho, D., & Capretz, L. F. (2015). Improving software effort estimation using neuro-fuzzy model with SEER-SEM. arXiv Preprint arXiv:1507.06917. https://arxiv.org/ftp/arxiv/papers/1507/1507.06917.pdf
   Google Scholar
• Duan, M., Li, K., & Li, K. (2017). An ensemble cnn2elm for age estimation. IEEE Transactions on Information Forensics and Security, 13(3), 758–772. https://doi.org/10.1109/TIFS.2017.2766583
   Web of Science ®Google Scholar
• Duan, M., Li, K., Liao, X., Li, K., & Tian, Q. (2019). Features-enhanced multi-attribute estimation with convolutional tensor correlation fusion network. Transactions on Multimedia Computing, Communications, and Applications (TOMM), 15 (3s), 1–23. ACM. doi:10.1145/3355542
   Google Scholar
• Eggensperger, K., Feurer, M., Hutter, F., Bergstra, J., Snoek, J., Hoos, H., & Leyton-Brown, K. 2013, December.Towards an empirical foundation for assessing bayesian optimization of hyperparameters. In NIPS Workshop on Bayesian Optimization in Theory and Practice, Vol.10. https://www.cs.ubc.ca/~hoos/Publ/EggEtAl13.pdf
   Google Scholar
• El-Fakdi, A., & Carreras, M. (2013). Two-step gradient-based reinforcement learning for underwater robotics behavior learning. Robotics and Autonomous Systems, 61(3), 271–282. https://doi.org/10.1016/j.robot.2012.11.009
   Web of Science ®Google Scholar
• Ensor, K. B., & Glynn, P. W. (1997). Stochastic optimization via grid search. Lectures in Applied Mathematics-American Mathematical Society, 33, 89–100. https://web.stanford.edu/~glynn/papers/1997/EnsorG97.pdf
   Google Scholar
• Falcini, F., Lami, G., & Costanza, A. M. (2017). Deep learning in automotive software. IEEE Software, 34(3), 56–63. https://doi.org/10.1109/MS.2017.79
   Web of Science ®Google Scholar
• Gu, X., Zhang, H., Zhang, D., & Kim, S. (2016, November). Deep API learning. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering pp.(631–642). Seattle, WA, USA: ACM.
   Google Scholar
• Gupta, R., Pal, S., Kanade, A., & Shevade, S. (2017, February). Deepfix: Fixing common c language errors by deep learning. In Thirty-First AAAI Conference on Artificial Intelligence. San Francisco, California, USA
   Google Scholar
• Han, S., & Dally, B. (2017). Efficient methods and hardware for deep learning. University Lecture. https://stacks.stanford.edu/file/druid:qf934gh3708/EFFICIENT%20METHODS%20AND%20HARDWARE%20FOR%20DEEP%20LEARNING-augmented.pdf
   Google Scholar
• He, Y., Qin, Y., Wang, S., Wang, X., & Wang, C. (2019). Electricity consumption probability density forecasting method based on LASSO-quantile regression neural network. Applied Energy, 1, 565–575. https://doi.org/10.1016/j.apenergy.2018.10.061
   Google Scholar
• Held, D., Thrun, S., & Savarese, S. 2016, October.Learning to track at 100 fps with deep regression networks. In European Conference on Computer Vision. pp.749–765. Cham. DOI:10.1007/978-3-319-46448-0_45
   Google Scholar
• Hellendoorn, V. J., & Devanbu, P. (2017, August). Are deep neural networks the best choice for modeling source code? In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering pp.(763–773).  Paderborn, Germany : ACM.
   Google Scholar
• Hjort, N. L., & Glad, I. K. (1995). Nonparametric density estimation with a parametric start. The Annals of Statistics, 23(3), 882–904. https://doi.org/10.1214/aos/1176324627
   Web of Science ®Google Scholar
• Hosni, M., Idri, A., Abran, A., & Nassif, A. B. (2018). On the value of parameter tuning in heterogeneous ensembles effort estimation. Soft Computing, 22(18), 5977–6010. https://doi.org/10.1007/s00500-017-2945-4
   Web of Science ®Google Scholar
• Hsu, C. J., & Huang, C. Y. (2011). Comparison of weighted grey relational analysis for software effort estimation. Software Quality Journal, 19(1), 165–200. https://doi.org/10.1007/s11219-010-9110-y
   Web of Science ®Google Scholar
• Hu, X., Li, G., Xia, X., Lo, D., & Jin, Z. (2018, May). Deep code comment generation. In Proceedings of the 26th Conference on Program Comprehension pp.(200–210).  Gothenburg, Sweden: ACM.
   Google Scholar
• Hu, X., Li, G., Xia, X., Lo, D., & Jin, Z. (2019). Deep code comment generation with hybrid lexical and syntactical information. Empirical Software Engineering, 25, 1–39. DOI:10.1007/s10664-019-09730-9
   Web of Science ®Google Scholar
• Huang, X., Ho, D., Ren, J., & Capretz, L. F. (2007). Improving the COCOMO model using a neuro-fuzzy approach. Applied Soft Computing, 7(1), 29–40. https://doi.org/10.1016/j.asoc.2005.06.007
   Web of Science ®Google Scholar
• Hubin, A., Storvik, G., & Frommlet, F. (2018). Deep bayesian regression models. arXiv Preprint arXiv:1806.02160. https://arxiv.org/abs/1806.02160
   Google Scholar
• Hutter, F., Lücke, J., & Schmidt-Thieme, L. (2015). Beyond manual tuning of hyperparameters. KI-Künstliche Intelligenz, 29(4), 329–337. https://doi.org/10.1007/s13218-015-0381-0
   Web of Science ®Google Scholar
• Hyndman, R. J., & Fan, Y. (1996). Sample quantiles in statistical packages. The American Statistician, 50(4), 361–365. https://www.jstor.org/stable/pdf/2684934.pdf
   Web of Science ®Google Scholar
• Idri, A., Abnane, I., & Abran, A. (2018). Evaluating pred (p) and standardized accuracy criteria in software development effort estimation. Journal of Software: Evolution and Process, 30(4), e1925. DOI:10.1002/smr.1925
   Web of Science ®Google Scholar
• Idri, A., Hosni, M., & Abran, A. (2016). Systematic literature review of ensemble effort estimation. Journal of Systems and Software, 118, 151–175. https://doi.org/10.1016/j.jss.2016.05.016
   Web of Science ®Google Scholar
• Jiang, W., Ma, H., & Chen, Y. (2012, April). Gradient based fast mode decision algorithm for intra prediction in HEVC. In 2012 2nd International Conference on Consumer Electronics, Communications and Networks (CECNet) pp.(1836–1840). Beijing, China: IEEE.
   Google Scholar
• Kemerer, C. F. (1987). An empirical validation of software cost estimation models. Communications of the ACM, 30(5), 416–429. https://doi.org/10.1145/22899.22906
   Web of Science ®Google Scholar
• Kendall, A., & Gal, Y. (2017). What uncertainties do we need in bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems, pp.5574–5584. https://papers.nips.cc/paper/2017/file/2650d6089a6d640c5e85b2b88265dc2b-Paper.pdf
   Google Scholar
• Kocaguneli, E., Menzies, T., & Keung, J. W. (2011). On the value of ensemble effort estimation. IEEE Transactions on Software Engineering, 38(6), 1403–1416. https://doi.org/10.1109/TSE.2011.111
   Web of Science ®Google Scholar
• Kocaguneli, E., Menzies, T., & Mendes, E. (2015). Transfer learning in effort estimation. Empirical Software Engineering, 20(3), 813–843. https://doi.org/10.1007/s10664-014-9300-5
   Web of Science ®Google Scholar
• Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. ICommunications of the ACM, , 60(6),84-90. doi.10.1145/3065386
   Google Scholar
• Li, C., Bovik, A. C., & Wu, X. (2011). Blind image quality assessment using a general regression neural network. IEEE Transactions on Neural Networks, 22(5), 793–799. https://doi.org/10.1109/TNN.2011.2120620
   PubMed Web of Science ®Google Scholar
• Li, L., Feng, H., Zhuang, W., Meng, N., & Ryder, B. (2017, September). Cclearner: A deep learning-based clone detection approach. In 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME) pp.(249–260). Shanghai, China: IEEE.
   Google Scholar
• Li, X., Jiang, H., Liu, D., Ren, Z., & Li, G. (2018, May). Unsupervised deep bug report summarization. In Proceedings of the 26th Conference on Program Comprehension pp.(144–155). New York, United States: ACM.
   Google Scholar
• Li, Z., Zou, D., Xu, S., Ou, X., Jin, H., Wang, S., & Zhong, Y. (2018). VulDeePecker: A deep learning-based system for vulnerability detection. arXiv Preprint arXiv:1801.01681. https://arxiv.org/abs/1801.01681
   Google Scholar
• Malgonde, O., & Chari, K. (2019). An ensemble-based model for predicting agile software development effort. Empirical Software Engineering, 24(2), 1017–1055. https://doi.org/10.1007/s10664-018-9647-0
   Web of Science ®Google Scholar
• Märtens, M., & Izzo, D. (2019, July). Neural network architecture search with differentiable cartesian genetic programming for regression. In Proceedings of the Genetic and Evolutionary Computation Conference Companion pp.(181–182). Prague, Czech Republic.
   Google Scholar
• Mensah, S., Keung, J., MacDonell, S. G., Bosu, M. F., & Bennin, K. E. (2018). Investigating the significance of the Bellwether effect to improve software effort prediction: Further empirical study. IEEE Transactions on Reliability, 67(3), 1176–1198. https://doi.org/10.1109/TR.2018.2839718
   Web of Science ®Google Scholar
• Minku, L. L. (2019). A novel online supervised hyperparameter tuning procedure applied to cross-company software effort estimation. Empirical Software Engineering, 24, 1–52. DOI:10.1007/s10664-019-09686-w
   Web of Science ®Google Scholar
• Nassif, A. B., Azzeh, M., Capretz, L. F., & Ho, D. (2016). Neural network models for software development effort estimation: A comparative study. Neural Computing & Applications, 27(8), 2369–2381. https://doi.org/10.1007/s00521-015-2127-1
   Web of Science ®Google Scholar
• Paquette, C., Lin, H., Drusvyatskiy, D., Mairal, J., & Harchaoui, Z. (2017). Catalyst acceleration for gradient-based non-convex optimization. arXiv Preprint arXiv:1703.10993. https://arxiv.org/abs/1703.10993
   Google Scholar
• Pradel, M., & Sen, K. (2018). DeepBugs: A learning approach to name-based bug detection. Proceedings of the ACM on Programming Languages, 2(OOPSLA), 147. Tucson, Arizona, USA.
   Google Scholar
• Qian, X., & Dougherty, E. R. (2016). Bayesian regression with network prior: optimal bayesian filtering perspective. IEEE Transactions on Signal Processing, 64(23), 6243–6253. https://doi.org/10.1109/TSP.2016.2605072
   PubMed Web of Science ®Google Scholar
• Sarro, F., & Petrozziello, A. (2018). Linear programming as a baseline for software effort estimation. ACM Transactions on Software Engineering and Methodology (TOSEM), 27(3), 12. https://doi.org/10.1145/3234940
   Web of Science ®Google Scholar
• Sengupta, B., Friston, K. J., & Penny, W. D. (2016). Gradient-based MCMC samplers for dynamic causal modelling. NeuroImage, 125, 1107–1118. https://doi.org/10.1016/j.neuroimage.2015.07.043
   PubMed Web of Science ®Google Scholar
• Song, L., Minku, L. L., & Yao, X. (2018, October). A novel automated approach for software effort estimation based on data augmentation. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering pp.(468–479). Lake Buena Vista, FL, USA
   Google Scholar
• Song, L., Minku, L. L., & Yao, X. (2013, October). The impact of parameter tuning on software effort estimation using learning machines. In Proceedings of the 9th international conference on predictive models in software engineering p.(9). Baltimore, Maryland, USA: ACM.
   Google Scholar
• Štrumbelj, E. (2017). Bayesian Lasso and multinomial logistic regression on GPU. PloS One, 12(6), e0180343. https://doi.org/10.1371/journal.pone.0180343
   PubMed Web of Science ®Google Scholar
• Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., & Manzagol, P. A. (2010, December). Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research, 11(12), 3371–3408. https://www.jmlr.org/papers/volume11/vincent10a/vincent10a.pdf?source=post_page---------------------------
   Web of Science ®Google Scholar
• Vono, M., Dobigeon, N., & Chainais, P. (2018, September). Sparse bayesian binary logistic regression using the split-and-augmented gibbs sampler. In 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP) pp.(1–6). Aalborg, Denmark: IEEE.
   Google Scholar
• White, M., Tufano, M., Martinez, M., Monperrus, M., & Poshyvanyk, D. (2019, February). Sorting and transforming program repair ingredients via deep learning code similarities. In 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER) pp.(479–490). Hangzhou, China: IEEE.
   Google Scholar
• White, M., Tufano, M., Vendome, C., & Poshyvanyk, D. (2016, August). Deep learning code fragments for code clone detection. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering pp.(87–98). Singapore: ACM.
   Google Scholar
• White, M., Vendome, C., Linares-Vásquez, M., & Poshyvanyk, D. (2015, May). Toward deep learning software repositories. In Proceedings of the 12th Working Conference on Mining Software Repositories pp.(334–345). Florence, Italy : IEEE Press.
   Google Scholar
• Xia, T., Krishna, R., Chen, J., Mathew, G., Shen, X., & Menzies, T. (2018). Hyperparameter optimization for effort estimation. arXiv Preprint arXiv:1805.00336. https://arxiv.org/abs/1805.00336
   Google Scholar
• Xiaomeng, W., Tao, Z., Runpu, W., Wei, X., & Changyu, H. (2018, August). CPGVA: Code property graph based vulnerability analysis by deep learning. In 2018 10th International Conference on Advanced Infocomm Technology (ICAIT) pp.(184–188). Stockholm, Sweden: IEEE.
   Google Scholar
• Yamada, S., Hishitani, J., & Osaki, S. (1993). Software-reliability growth with a Weibull test-effort: A model and application. IEEE Transactions on Reliability, 42(1), 100–106. https://doi.org/10.1109/24.210278
   Web of Science ®Google Scholar
• Yang, J., & Ma, J. (2019). Feed-forward neural network training using sparse representation. Expert Systems with Applications, 116, 255–264. https://doi.org/10.1016/j.eswa.2018.08.038
   Web of Science ®Google Scholar
• Yin, B., Wu, M., Cavallaro, J. R., & Studer, C. (2014, December). Conjugate gradient-based soft-output detection and precoding in massive MIMO systems. In 2014 IEEE Global Communications Conference pp.(3696–3701). Austin, TX, USA: IEEE.
   Google Scholar
• Zhan, H., Gomes, G., Li, X. S., Madduri, K., & Wu, K. (2018). Efficient online hyperparameter optimization for kernel ridge regression with applications to traffic time series prediction. arXiv Preprint arXiv:1811.00620. https://arxiv.org/abs/1811.00620
   Google Scholar
• Zhang, W., Yang, Y., & Wang, Q. (2015). Using bayesian regression and EM algorithm with missing handling for software effort prediction. Information and Software Technology, 58, 58–70. https://doi.org/10.1016/j.infsof.2014.10.005
   Web of Science ®Google Scholar
• Zhang, Z., Lai, Z., Xu, Y., Shao, L., Wu, J., & Xie, G. S. (2017). Discriminative elastic-net regularized linear regression. IEEE Transactions on Image Processing, 26(3), 1466–1481. https://doi.org/10.1109/TIP.2017.2651396
   PubMed Web of Science ®Google Scholar