Reply to the Discussion of “Application of neural network and adaptive neuro-fuzzy inference systems for river flow prediction” by O. Kisi

The authors wish to thank Dr Kisi for his discussion paper (Kisi, 2010) showing interest in the results of the research methodologies of our article (Pramanik & Panda, 2009). In this reply, the authors have tried to elucidate the comments given by the discusser in a systematic manner, paragraph numbers corresponding to those in the discussion.

1. The parameters of the membership functions of the fuzzy interference system (FIS) are optimized using the hybrid learning algorithm, which is efficient and commonly used training algorithm, reported to produce promising results in several previous studies (Chang & Chang, 2006; Firat & Gungor, 2007; Elabd & Schlenkhoff, 2009). The hybrid learning algorithm is a combination of the gradient descent technique and the least-squares technique. The gradient descent technique was used to optimize the nonlinear input parameters (a_i , b_i , c_i ), whereas the least-square method was employed to identify the linear output parameters (p_i , q_i , r_i ). Moreover, the hybrid learning approach is beneficial due to the fact that the convergence is much faster since the dimensions of the search space is reduced in the original pure back-propagation method used in ANN technique. In addition to the gradient decent algorithm, many other learning algorithms like LM, CG and CGX can be used to update the ANFIS parameter values for better results as suggested by the discusser. But the authors have not attempted these algorithms for ANFIS modelling in the present study.

2. In Pramanik & Panda (2009), ANN model training was carried out using the trial-and-error method. About one hundred epochs were found sufficient to train the ANN models to attain a close agreement between the computed error and the present error goal. Since the results of the ANN models are not reproducible due to random initialization of the weights, the best results were selected based on the highest modelling efficiency and the lowest RMSE value. The learning rate and momentum constant used in our study to produce the best results are 0.005 and 0.9, respectively.

3. The authors do not agree with the discusser's view that the RMSE obtained during testing cannot be lower than the RMSE of training for the training done using RMSE criterion. There are quite a good number of ANN applications in past studies showing that RMSE obtained during testing is lower than the training RMSE, even though the RMSE criterion was used to optimize the ANN architectures (Nayak et al., 2004; Chen et al., 2006; Jaksa et al., 2008). The authors wish to express that the values of these RMSEs obtained during model training and testing depends on the nature of the fitting data sets. In Pramanik & Panda (2009), lower values of RMSE during testing may have been due to satisfactory training attained for the kind of training data sets used and thus it cannot be said that the trained network failed to investigate the river flow processes.

4. In the paper under discussion, the comparisons among all the models were based on the RMSE values. In this regard, according to the discusser, the performance of Model 4 is the best during testing but not during training. However, in the published paper, the authors mentioned that the performance of Model 4 during testing is better than training based on the modelling efficiency. However, Model 4 can be said to be well-fitted or merely under-fitted, but cannot be called over-fitted as pointed out by the discussers. We agree with the discusser's remark on the statement “Model 4 produced better results during model training but failed to yield better results in testing. This may be due to over-fitting of the training data sets and poor generalization of the input-output data …” and would request the readers to refer the discussion paper.

5. The performance of the CGF algorithm in the study is slightly better than LM; this is due to the capability of the algorithm's search technique, which performs along the conjugate direction with a faster rate of convergence. For this case study, the CGF yielded comparable results with LM yielding the modelling efficiency slightly better than LM. We agree with the discusser's view that in most of the applications but not all, the LM algorithm has shown the best performance (Cigizoglu & Kisi, 2005; Kisi, 2005, 2007).

6. The present study of using ANFIS for river flow prediction was the first study for the authors on the application of ANFIS. The authors tried to obtain the best results by training the ANFIS architecture, deleting certain rules in each run. Over-fitting is a common issue in data-driven modelling and attempts were made to overcome it by using only the selected rules. In each run, the authors used less than 128 rules by deleting a few rules. In the Matlab GUI, we used the option to delete rules in each run and finally fewer than 128 rules would have been used in the ANFIS parameter optimization. Therefore, we think that the presented results are the optimum outcome of the ANFIS model training, and it is expected that the ANFIS architecture is least affected by over-fitting.

7. We strongly agree with the discussers' view that normalizing the data sets in the range 0.2 to 0.8 could improve model training. The study being the first attempt for the authors to use ANN in river flow prediction, the existing Matlab function “premnx” was used to scale the data in the range –1 to +1. However, in subsequent studies, the authors have investigated the effect of different scales of the data sets on model prediction accuracy (Singh et al., 2009; Panda et al., 2010).

8. The authors used the term “marginal” to indicate least improvement in the value of RMSE in the case of GDX and CGF. Improvement in the RMSE in the discussion of the original published paper was used in a positive sense, meaning that there is slight decrease in the values of RMSEs in the case of GDX and CGF, whereas the decrease is more significant in case of LM and ANFIS.