kNaUO[IfA

E L S E V I E R Analytica Chimica Acta 321 (1996) 97-103

Quantitative structure-property relationships for colour reagents and their colour reactions with ytterbium using regression

analysis and computational neural networks

H u a L i i, L u X u *, Y i q i u Y a n g , Q i a n g S u

Clumgchun I~timte of Applied Chem~fry, Chbvese Academy of Sciences, Clmngchun 130022. Cl~.a

Received 17 May 1995; revised 13 September 1995; accepted 23 October 1995

Abstract

In this paper, the new topological indices A~t-A,o suggested in our laboratory and molecular connectivity indices have been applied to multivariate analysis in structure-property studies. The topological indices of twen|y asymmetrical phosphono bisazo derivatives of chromotmpic acid have been calculated. The stnvcture-pm~rty relationships between colour reagents and their colour reactions with ytterbium have been studied by A xt-A i3 indices and molecular conne~vily indices with satisfactory results. Multiple regression analysis and neural networks were employed simultaneons~ in this study.

Keywords: Quantilalive structure-property relationships; Neural networks; Regression analysis; Topological indices; Chememeuic~ Ytterbium

1. Introduction

The rare earth elements are widely used in metal- lurgy and ceramics indusu'ies and in the processing of electronic and luminescent materials because they posses special physico-chemical properties. For these reasons, studies of the rare earth elements have become important in recent years. Many methods have been developed for the determination of rare

" CGn~ponding author. 1 Present addre~: Department of Chemistry, Henan University,

Kaifer, g 475001, Henan, ~ .

earth elements, and one of the most important of these is spectrophotometry [1-4]. A key step in the spectropholometrie method is the selection of a sen- sitive, highly selective colour reagent and of suitable analytical conditions. Therefore, many of the colour reagents, such as asymmetrical phosphono binszo derivatives of chromotropic acid have been synthe- sized in China.

Correlations between the structures of the colour reagents and its reactivifies and physicochemical properties are important because they can be used to guide the colour reagent design.

Ouantitative structure-activiW/propeny relation- ship studies (QSAR/ QSPR) have been exploited

0G03-2670/96/$15.00 @ 1996 Elsevier Sci©ncc B.V, All rights reserved S~DI 0 0 0 3 - 2 6 7 0 ( 9 5 ) 0 0 5 4 9 - 8

98 H. Liet al. /Analyrica Chimica A cra 321 (1996) 97-103

extensively in the design of drugs and pesticides, but few such studies have been applied to the design of colour reagents. The method of topological indexing of molecular structures has also been widely used in recent years in connection with QSAR/OSPR. The key step in the development of a topological index is the selection of a graph invariant, which i,o a quantity that can be derived from the graph and is not af- fected by its node numbering. Over a hundred topo- logical indices have been described, such as the Wiener index W [5], Randic index ID [6], Hosoya index Z [7], Balaban index J [8] and the general a N index [9]. One significant development has been the increasingly widespread use of topological indices, a trend which has become of growing importance in recent years. The topological indices Axl-Ax3 based on the augmented distance matrices devised recently by our laboratory have been successfully employed tctL, e ',~udies on structure-activity relationships for alkanes, alcohols and barbiturates [10]. In this study, we further utilized these indices and the molecular connectivity indices [11] to the structure-property relationships between colour reagents and their colonr reactions with ytterbium. The results demonstrate the feasibility and effectiveness of the method.

2. Neural network algorithm

Computational neural networks have been devel- oped to simulate the functions of neurons in the human brain. The way in which the computer-based network works is to accept a set of facts, transform those facts, and generate an associated set of output facts. Through an iterative "learning" process, the network refines the information derived from the input values (descriptors) in order to reproduce an associated set of target values (molar absorptivities). Once a network has been ualned to recog, ize the underlying theme for a given set of input/target pairs, it may be used to predict an output value corresponding to a new group of input values.

A neural network consists in general of an input layer, an output layer, and any number of intermedi- ate layers, called hidden layers. Each unit in the network is influenced by those units to which it is connected, the degree of influence being dictated by the values of the links or connections. The input

signals are weighted as they are transmitted to the nodes of the second layer, the hidden layer. The hidden layer neurons process the data and send a signal to the neurons of the output layer. The output layer provides the predicted value, in this work the molar absorptivities of the coiour reaction~,. A neural network is trained to relate certain inputs (descriptor values) to target outputs (molar absorptivities). To accomplish th~s, a variety of neura, t network learning algorithms can be used. In this study, back-propa- gation (BP) and quasi-Newton methods were used. We found that the quasi-Newton method required fewer training cycles than did the back-propagation algorithm.

2.1. The back-propagation training algorithm

Each individual neuron in a network computes the weighted sum of inputs, anti, using the equation:

netj = Ew~jx~ + oj (1) Where wtj is the weights connecting neuron j with each neuron in the previous layer, x{ is the outputs for neurons in the preceding layer, and 0j is the bias for neuron j. A nonlinear activation function is then performed on the weighted sum of inputs to compute the output value for neuron j to be sent to the following layer of neurons:

O~ = 1 / ( 1 + e . . . . *) (2)

During training, the BP algorithm minimizes the mean square error where the error, E, is

E - E(tp--Op) 2 (3) p

Where p is an index for training observations. The target value for pattern p is tp and the computed value is Op.

The connection weights and biases in the network are adjusted sequentiaBy to reduce the error. The BP algorithm uses a gradient descent method to adjust all the weights and biases in the network sequen- tially. In this method, the partial derivative of the error function is used to deteimine each weight adjustment, Awlj. If the neuron of interest is con- t~ned in the output layer, then the error is calculated from the difference between the output value and the target value multiplied by the derivative of the out-

H. Li et at./A:mlytica Chimica Acta 321 (1996) 97-103 99

put value. The error terms for the hidden-layer neu- rons are more complicated because the target values do not exist and must be calculated recursively from neurons already modified. To improve the training time of the BP algorithm and avoid the hazards of oscillating or stagnating along the error surface, or becoming trapped in total minima, a momentum term is added to the weight adjustment equation. The full details of BP training have been published previ- ously [12].

2.2. Quasi-Newton (BFGS) training algorithm

The BFGS (Broyden-Fletcher-Goldfarb-Shanno [13-17]) quasi-Newton optimization method is an alternative way to minimize the sum-squared-errur of Eq. 3. The major difference between the BFGS and back-propagation learning algorithms is how the weights are adjusted, BFGS networks possess the ability to recognize the shape of the error function with respect to each weight and consequently adjust or step to value of the weight which wilt minimize the erlDr.

The basis of all quasi-Newton methods is that in cycle k + 1 of the optimization, the error E and gradient gk+t ate assumed to be expressible as truncated Taylor series in the parameters x:

E(xk+l) ~ E(Xk) 4- gTAx k + 1 / 2 A X [ H k A X t

(4)

gk*~ = gk + HkAXk (5)

Where AX k - -Xs+ n - X k is the change in the pa- rameters from cycle k to cycle k + 1, and Hk is the Hessian matrix of cycle k. The Hessian is defined as the matTix of the second derivatives of the error function with respect to the parameters. For a neural network, the parameters are just the weights and biases.

If Xk+ t is to correspond to a minimum of the e r r o r functional, then g t + n = O. This leads to the "Newton" step formula

AX k = - -Hk lgk (6)

In the quasi-Newton method, the inverse Hessian matrix Hk t is never computed directly. Instead, it is iteratively estimated and updated as the optimization proceeds. The initial estimate of the inverse Hessian

matrix G is obtained from the gradient for two different sets of weights and biases, each weight and bias is changed by a small amount and the gradient is computed a second time. The c o i f i n g change in g is used to estimate the diagonal ekmenls of G:

G,i -= ,~w,j/as~ (7)

With an estimate Gut of the inverse Hessian matrix, the steps in a BFGS cycle are as follows: 1. Chcose search direction d~, according to d k z

--Gkg~ 2. Determine the scalar a k to minimiTe E(Xk+

a d s ) 3. Let xk+ n = x t + adk 4. Compute the gradient gk+, corresponding to the

parameters xk . z 5. Update the inverse Hessian matrix G by the

BFGS method 6 . Iterate

The line minimization parameter a k can be esti- mated using a parabolic fit of the error along dk:

E(xh + adk) -- E ( x k ) + a a +/~a 2 (g)

Two pieces of data are needed to find rite cou- stunts a and b. Satisfactoxy results are obtained by using (a) the slope, ~E /0a at x k, which is given by d~gk) and (b) the error E(x k + sd l ) , where s is an appropriately chosen step size. The minimum of this cure occurs when a = - a / ( 2 b ) .

When using any iterative training procedure, a criterion must be available for deciding when to stop the iterations. Three approaches were used in this study: (a) the weights were adjusted for each obser- vation until the sum-squared-error reached an accept- able value for the entire training set; (b) the number of training cycles was limited, and training was stopped after a fixed number of training cycles had been reached; (c) training was stopped when the minimum test set error was achieved.

3. The topolosical indices

On the basis of the distance matrix, Ax~-Ax3 have been derived in our laboratory. To facilitate the understanding of A~z-Ax3, the method is briefly introduced here. The three topological indices are

100 14. Li el a t /Ana ly t i ea Chimica Acre 321 fi996) 97-103

generated from path matrices A, B, and C, respec- tively. These three matrices are defined as follows:

{10 P a t h = l A = (81i j ) , ~llij = others

2 path = 2 B = (bij), bij = 0 others

3 path = 3 C = (eij), e 0 = 0 others

Augmented path matrices G t - G a are obtained by adding two columns into matrices A, B, and C, respectively. The elements in the first column of matrices G t - G 3 are square roots of vertex 0¢~ee~, and the elements in the second column represent the square roots of the van der Waals radii of atoms. From matrices G t -G3 , we can obtain matrices Z t - Z3:

Z I = G I * G ' I ; Z 2 = G 2 * G ~ ; Z 3 = G 3*G' 3

where G't-G3 are the transpose matrices of Gt -Ga . The three new topological indices are defined as:

Aat = Amaxl/2; Aa2 = Amax2/2; Aa3 ~ Amax3/2

where Amaxl-Amax3 are the largest eigenvalnes of matrices Z~-Z3.

(i,j = 1 ,2 . . . n )

(i,j = 1 ,2 . . . n )

(i,j = 1,2. . . n)

4. Experimental

Table l Structure of colour rcagcms and molar absorptivitics of their colour reaction with ytterbium

No. Stmcture pH ~ x l 0 -4

R~ R 2 R~

1 - C O C H 3 2 4.0 2 - C O C H 3 2 3.9 3 a - B r 1 4.5 4 - B r 2 4.2 5 - B r 3 0.65 6 - S O 3 H 2 6,0 7 - S O a H 2 3.9 8 a - C O O H 2 3.7 9 - C I - N O 2 2 2,7

1 0 , - C I 1 0.58 I t - C I i 4.3 12 -CI 1 3.9 13 ~ - C O O H 2 2.0 14 - O ~ H 3 l 5.7 15 - O C H 3 2 3.7 16 - O H 2 3.0 17 a - O H 1 2.2 18 - C H 3 1 5.1 19 - C H 3 2 4.6 2O 2 3.9

i Member of test set.

The experimental conditions (pH values) em- ployed and colour reactions with ytterbium (i.e,, molar absorptivities) ate listed is Table 1.

The neural network softwar0 included two mod- ules: BP and quasi-Newton neural networks, which were written in FORTRAN 77 and installed on a micro VAX II computer. All computations were performed using multivariate statistic analysis pro- grams (MSAP), which consist of multivariate regres- sion, pattern recognition and calculations of topolog- ical indices, etc.

4.1. Structure of colour reagents

The structures of asymmetrical phosphono bisazo derivatives of chromotropic acid are as follows:

4.2. Calculation of descriptors

The topological indices A~1-A~3 and the molecu- lar connectivity index mx~' of twenty asymmetrical phosphono bisazo derivatives of chromotropic acid have been calculated. A total of 20 topological in- dices were generated for each colour reagent.

4.3. Objective feature selection

The next step in the model formation procedm¢ is the evaluation of the computed topological indices. To refine the topological indices to reduce the possi- bility for chance correlations, the multiple stepwise regression analysis was used to weed out the less useful topological indices in the multiple stepwise regression analysis. The user can select any combina-

H. Lt et ~L / Analygca Chimica Acre 321 (1996) 97-t03 101

finn of v~riables Io develop an equation while the program umvides the necessary statistics, when eval- uating regression models to select the topological indices, a model possessing a high multiple correla- tion value, a small standard deviation of regression and as few topological indices as possible is consid- ered high quality. A cheek of correlation value, standard deviations, and numbers of the topolo~eal indices shows AI/~ 2, A'~/~ 2, 2Xp, 4Xpc to ]30 signifi- cant a f f e c t s on log e.

The 4Xpc index is compu ted 4Xpc ~ ( 8 i 8 i 8~ 818 m) - 0.s. A valence delta ( 8 ~ ) is assigned on the basis of adjacency or, in the case of het- eroatoms or unsaturated carbons, use the pgescription 8 ~ - Z ~ - - h where Z ~ is the number of valence electrons and h is the number of attached hydrogen atoms. The * X ~ index has been found to carry information on the number of benzene ring sub- stituents, the substitution pattern, and the length of the substituents up to three bond lengths [18]. The 2Xp index is computed 2 X p = ~(SiSjSk)-O'$. The magnitude of 2Xp is, therefore, dependent not only on the values of 81, 8j and 8 k in each fi'agmvnt, but also on the number of fra~nents into which the molecule can be dissected []9].

$. Resulgs and d i scmsioa

5.1, Regress ion analys~s

First the reg~ssion model was developed fo~ all 20 molar absorptivities of their reactions with ytter- bium, and the following model containing four de- scriptors was obtained:

log E = ~:~ 8123 4- 1.13692Xp -- 2.0313' tX~

+ 6.8844A'~/z -- 7.9967A'( 2 (9 )

R ffi 0.9325, F ffi 24,9908, ILMS ffi 0.0908, n = 20 Where R is the correlation coefficient, F denotes

the F-~est value, RMS is the toot mean square error and n represents the number of samples.

In order to validate both the regression and the neural network results, the 20 compounds were ran- domly divided into two groups: a training set consist- ing of 15 observations and a prediction or test set containing the other 5 coiour reagents. Using the same feur descriptors, a regression model wes d©vcl- oped with the 15 colour reagents set.

log e = - 13.2258 + 1.50992Xp - 2.72544Xp, + 1.6657A*x/22 -- 0.5706Alx/2 (10)

R = 0 . 9 6 1 6 , F ~ 30.6884, RMS ~ 0.0608, n = 15

Table 2 Topological indices Axj-A~3 and molecular connectivity indices of ¢olo¢r reagents No A'~ 2 Ay~ ~ x~ ' x ,~ tog ~ log , ( ,zO ERR1 log e (neeral) ERR2

1 10.4206 12.7396 15.8423 5.9011 4 .6021 4.6999 2 10.4169 12.7475 15.8304 5.9351 4 .5911 4.5786 3 i 10.2420 12.6157 16.4435 6.0815 4 .6532 4.8892 4 10.2392 12.6154 16.4316 6.1711 4 .6233 4.6225 5 10.2439 12.6451 16.2762 6 ,3649 3 .8129 3.8506 6 10.5124 12.Tg~t 15.6717 5 .8234 4 .7782 4.7736 7 10..5099 12.8118 16.9502 6 .6035 4 .5911 4.5655 8 a 10.4137 12.7354 15.5699 5 .7292 4 .5682 4.7480 9 I0~5031 12.8304 16.1086 6 .2070 4 .3617 4.3538

10 a 10,2593 12.6557 15.6975 5 .9546 3 .7634 4,1146 11 10.2574 12.6243 15.8064 5 .7922 4 .6335 4.7366 12 10.2543 12.6242 15.7945 5 .8525 4 .5911 4,5488 13 a 10.4218 12.7833 1&5138 5.7531 4 .3010 4.5844 14 10.3303 12.6772 15.3924 5 .6150 4.7.559 4.6855 15 10.3269 12.6808 15.3805 5 .6456 4 .5682 4.5765 16 10.2502 12-6203 15.1886 5 .5116 4 .4771 4.5587 17 ~ 10.2472 12.6201 15.1767 5.54.~5 4 .3424 4,4485 18 10.2574 12.6243 15,6399 5 .7165 4 .7076 4,6912 19 10.2543 12.6242 15.6280 5 .7693 4.662.8 4.5243 20 10.1637 12.5637 15.0061 5,3611 415911 4.5816

- 0.0978 4 f~ ,30 -- 0.0,109 0.0125 4.5531 0.0380

- -0 .2360 4.6452 0:0080 0.0008 4.5022 0:02 ! 1

- 0.1[~,'77 3.'FJ82 0.0147 0.0046 4.7732 0.0050 0.0256 4.6115 --0.0"204

--0.1798 4.7O48 --0.1366 0,0079 43"772 - 0.0155

-0.3.512 3.7782 --0.0148 --0.1031 4.6672 --0.0337

0.0423 4~5535 0.0~76 - -0 .2834 4.2950 0.006O 0.0704 4.6966 0.0593

- -0 .0084 4.5505 0.0177 -- 0.0~16 4.6044 --0.1273 - - 0.1061 43355 0.0069

0.0164 4.6679 0.0397 0.1385 4-5223 0.1405: 0.0095 4.6553 -- 0.0642

• Member of test set; REEl = log • -- log e(r~g); P-JEF..2 = tog e(ncuraD.

102 H. Li et at./Analytica Chimica Acta 321 (1990) 97-103

The molar absorptivities calculated by Eq, 10 for training set and test set are listed in Table 2, column log 6(reg,). The RMS to test set is 0.2959.

5.2. Rvsults with the neural network

To obtain the best network performance, the opti- m um network architecture must be chosen. Studies o f the network structure include the selection of the number o f layers and the number of neurons in each layer. The architecture o f the neural network in this paper is as follows.

(a) There are three types o f layers, i.e., input, hidden and output layers. The number of input neu- rons is four (one for each descriptoD, and there is one output neuron. The nmaber of liCUlOllS used in the hidden layer was determined by trial and error.

(b) The input neurons are Al/x22, Alx/2) 2Xp, 4Xp~. Usually, the value o f each neuron is defined between 0 and 1, thus the input data should be scaled within the defined region. Note that i f the value of a neuron in the input layer is zero, the connections from this neuron are always zero, i.e., the information from that neuron cannot be propagated to the following layers. To avoid this situation, the values were set between 0.05 to 0.95 in our research.

(c) When one is training a neural network, the objective is to find the global min imum on an error surface, where tl, e surface is multidimensional. The dimensions on the error surface correspond to the number of adjustable parameters in a given neural network. Because this error surface is so complex, it is very easy to get caught in local minima. There- fore, when one is ua in ing a neural network, it is essential to try various sets o f random starting weights in searching for the global minimum. This is a very time consuming procedure and requires training the network several t imes using different sets o f starting weights and biases.

(d) As a usual rule of thumb, the total number o f weights and biases should be less than the number o f observations. Finding a compromise between the performance and the training time of the work, the opt imum number of neurons in the hidden layer for this application was found to be three. Therefore, a hidden layer with three neurons was used in all the studies, yielding an overall network architecture of 4:3:1.

0.22

OA9

O I B

~ 0.13

~0, I0

007

0.06

001 • : = : : , = - - o : . . . . ==1 I GO 200 .300 400 500 600

,~I~r~*r of *I*_~r)lng I~poehs

Fig. 1. Plot o f RMS ¢~or vs. Iraining eyctas for quasi-Newton neural network.

(e) Fig. 1 shows a plot of the root mean square error as a function of the number of training epochs for both the training and test set. The 213 epoch results are highlighted in Fig. 1 with an arrow, since the weights and biases associated with the 213 epochs network produced the min imum test set errors, they were determined to be the opt imum for the parame- ters used in this experiment. The neural network root mean square is significantly lower than that o f the regression result for both the training and the test set, indicating that for the majority of the observations the neural network predictions are superior. A com- parison o f the calculated vs. observed values for the neural network and the regression model shows the emphasis that the regression analysis puts on attain- ing reasonable values for all observations, while the neural network focuses on getting many excellent molar absorptivities at the cost o f a few bad predic- tions.

Due to the tremendous number o f adjustable pa- rameters, one potential problem associated with the use o f neural network techniques is that it may learn a far too specialized relationship, in other words, overtraining. This problem is averted by choosing a portion of the compounds to serve as test set. Be- cause this set of compounds has no direct influence on the actual learning process, it can be used to monitor the predictive capability of the network at

H. Li et aL /Analyaca Chimica Acm 321 (1996) 97-103 103

regular intervals during the training run. As long as the test set results ceased to improve, training was stopped, in spite of the continued improvement in the training set results.

The 20 colour reagents were divided into the same two sets as in the regression analysis above, The calculated results by the best model obtained using the network are also listed in Table 2, column log e(neural), The RMS for training set and test set are 0.0603 and 0.0617, respectively, which all ate less than the deviations obtained by using regression analysis.

!1 should be emphasized that although neural net- works produced superior results, they will not re- place regression analysis in these studies. Regression analysis was necessary to determine which set of descriptors was important in calculating the molar absorptivities for this data set. The same descriptors that were found important using regression analysis were used to train the neural networks. Therefore, neural networks will provide a supplement to regres- sion analysis in an attempt to improve the accuracy and quality of the results.

6. C o n d u s l o n

The topological indices A~l-Ax3, recently intro- duced by us, and molecular connectivity indices have been successfully used to model the structure prop- erty relationships between colour reagents and their colour reactions with ytterbium. The experiments reveal that the results obtained using a new quasi- Newton neural network are more accurate than these

achieved using regression analysis, The study demonstrates convincingly that Axi -Ax3 are useful topological indices. These results also demonstrate that neural networks can be used successfully in the prediction of molar absorptivities o f colour reaftiom.

References

[i] s.B. saw[n, T.V. Pctnova ~r~a P.N. Romanov, Tal~m~k 19 (1972) 1437.

[2] T. Taketatsu, M. Kancko and N. Kono, Talanta, 21 (1974) 87.

[3] N.U. Perisic-Janii¢, A.A. Muk and V.D. Caa~ Anal. Chem., 45 (1973) 798.

[4] C.-G. Su, C.-S. Flu. X.-P. Jia and J.-M. Pan. Anal. C~m. Acta, 124 (1981) 177.

[5] H. Wiener, J. Am, Chem. Seer., 69 (1947) 17. [6] H. Hosoya, Bull. Chem. Sot:., 44 (1971) 2332. [71 M. Randic, J. Am. Chem. See., 97 (1975) 6609. [81 A.T. Balabaa, Pure Appl. Chem., ,~ (1983) 199. [9] L. Xu, H.Y. Wu~g and O. Su, Comlmt. Chem. 16 (1992)

187. ll0] Y.Y. Yao, L, Xtt, Y.Q. Yang and X.S, Yuan, $. Cl~m. inf.

Comput. Sci., 33 (1993) 590. [Ill LB, Kier and LH. 14~l, ~ Cmmectivity in Chem-

istry and Drug Research, Academic Press, New York., 1976. [12] P.A. Jansen, Anal, Chem., 63 (1991) 357A. [13] C.G. Ikoyden, J, Inst. Math. Appl., 6:76-90 (1970) 222. [14] R. Fletcher, Comput. J., 13 (1970) 317. [15] D. Goldfmb, Math. C(mlpul., 24(1970) 23. [161 D.F. Sharmo, Math. Comput., 24 (1970) 647. [17] R. Fletcher. Practical Methods of Optimi=atioa, Vol. I, Wi-

ley, New York, 1980. [18] LB. Kicr and L.H. Hall, Molecular Connectivity in Strut-

lure-Activity Analysis, Research Studies Press, l.~tchworlh, 1986, p. 59.

[191 LB. Kicr and L.H. Hail, Mokanl~ Connectivity in Stmc- lure-Actlvity Analysis, Research Studies Pms~ l~'tchworth, 1986, p. 50.