5.3.1 Direct (CG-MLE) Stability

The LR maximum likelihood equations are not quadratic forms as defined in Section 2.1. Therefore nonlinear CG is required instead of the linear CG used in our IRLS, above. We have chosen to use the Polak-Ribiére direction update due to its consistently good performance in experiments with our implementation of CG. As discussed in Section 2.2, nonlinear CG needs occasional restarts of the search direction to the current gradient. For all of our CG-MLE experiments we use two restart criteria. The simplest is restarting after $M$ iterations are performed, where $M$ is the number of attributes in our dataset. This is only likely to occur for our narrowest dataset, ds1.10pca. The second is Powell restarts, described in Section 2.2. Powell restarts are incorporated into the Polak-Ribiére direction update formula, a combination we refer to as modified Polak-Ribiére direction updates. The proposed stability parameters for our CG-MLE implementation are explained in sections that follow, and summarized in Table 5.24. We will continue updating our LR description tree, and Figure 5.12 indicates the branch related to CG-MLE.

We recommend that the reader not compare results in this section to those of 5.2. Chapter 6 compares times and AUC scores for the final version of our IRLS implementation, using both cgeps and cgdeveps, the final version of our CG-MLE implementation, and three other popular classifiers.