Comparative Study of the Quick Convergent Inflow Algorithm (QCIA) and the Modified Quick Convergent Inflow Algorithm (MQCIA)

The performance of two line search algorithms, the Quick Convergent Inflow Algorithm and the Modified Quick Convergent Inflow Algorithm, used in locating the optimizers of response functions is studied. The methodology requires the use of the same starting experimental design. The indicator variables are the number of iterations and the optimal point reached at each iteration. The Modified Quick Convergent Inflow Algorithm seems to perform generally better than the Quick Convergent Inflow Algorithm in the sense that solutions obtained are much closer to the exact solutions than those obtained using the Quick Convergent Inflow Algorithm. As a consequence to the study, a new algorithm is developed for solving Linear Programming problems. The algorithm iteratively eliminates from an N-sized starting design a point that contributes less to the process as measured by the predictive variances at the design points. The design size is immediately recoverd by adding to the resulting N-1 sized design a design point from the candidate set that optimizes performance. The new algorithm offers approximate solutions to Linear Programming problems as demonstrated with some numerical illustrations.


Introduction
There have been growing concerns on the development and application of specific designed techniques for determining optimal choices.For many decades, Linear Programming problems have been solved using graphical method, simplex method, interior point algorithm, Newton's method, karmarker's method and many others.However, in some few years now, experimental design techniques have been employed as alternative methods in solving linear programming problems.These methods, usually based on sequential algorithms, include Maximum Norm Exchange Algorithm of Umoren (1999), Quadratic Exchange Algorithm of Umoren (2002), Quick Convergent Inflow Algorithm of Odiakosa and Iwundu (2013), Modified Quick Convergent Inflow Algorithm of Iwundu and Ebong (2014), etc. Umoren (1999) and Umoren (2002) applied the optimal design principles in solving constrained optimization problems.Etukudo and Umoren (2008) showed that it is easier and in fact better to use the modified super convergent line series algorithm, which is based on the principles of optimal design of experiments, in solving quadratic programming problem rather than using the modified simplex method.Umoren and Etukudo (2010) presented an algorithm for solving unconstrained optimization problems.The algorithm relied on the principles of optimal experimental design.The algorithm was further modified and extended to handle constrained optimization problems.Odiakosa and Iwundu (2013) developed the Quick Convergent Inflow Algorithm (QCIA), for solving linear programming problems.Iwundu and Hezekiah (2014) studied the effect of the Quick Convergent Inflow Algorithm on segmented regions.A stopping rule based on the concepts of variance exchange algorithm was proposed.Iwundu and Ebong (2014) presented the Modified Quick Convergent Inflow algorithm (MQCIA).The method sequentially adds a point of maximum variance to an initial design in a maximization problem and similarly adds a point of minimum variance to an initial design in a minimization problem.The MQCIA is a useful alternative particularly when the point reached by the line equation in the QCIA does not satisfy the linear inequality constraints and hence cannot be used as an admissible point of the experimental design.
We seek in this work to compare the working of the QCIA and the MQCIA in solving linear programming problems.In comparing the working of the two line search algorithms, we shall (i) attempt to locate the optimizer of a response function using a unique starting design on the Quick Convergent Inflow Algorithm and the Modified Quick Convergent Inflow Algorithm.
(ii) compare the performances of the two techniques as measured by the number of iterations required to reach the optimum and the value of the objective function.
(iii) develop an alternative modification to the Quick Convergent Inflow Algorithm.

Methodology
For given Linear Programming problem of the form: subject to A ≤  ;  ≥ 0 where  is the vector of variables sought for, A is a matrix of known coefficients,  and  are vectors of known coefficients, we outline in sections 2.1 and 2.2 the sequential steps of the Quick Convergent Inflow Algorithm and the Modified Quick Convergent Inflow Algorithm, respectively.

2.1
The Quick Convergent Inflow Algorithm (Odiakosa and Iwundu;2013) The Quick Convergent Inflow Algorithm follows the sequence of steps; S 1 : Form the design measure,    , at the kth iteration, by selecting N support points S 2 : Obtain the optimal starting point  * .For n-variates, say, x 1 , x 2 , ... , x n the average of N support points can be used as the optimal starting point.
where g is the vector of the coefficients of the objective function.where k = 0,1, …, q. x 1  satisfy the constraints, continue from step S 2 to step S 11 .Thus obtaining If No, go to step S 12 and continue the process.
If Yes, the optimizer of the objective function is (Iwundu and Ebong;2014) The sequential steps that make up the Modified Quick Convergent Inflow Algorithm are; (i) Obtain ̃ grid of points  (1) ,  ( ) , ⋯,  ( ̃) from the feasible region to make up the candidate set S = {  (1) ,  ( ) , ⋯,  ( ̃) from which design points will be selected into the design measure.
(ii) From the ̃ grid of points, select an N-point (N ≤ ̃), n-variate non-singular initial design,   .
The initial design measure and the corresponding design matrix are ( )  ()  ) is the vector of coefficients of the objective function.
(v) Evaluate the step-length of search.The step-length is taken as ( ) where ;  = ‖ ‖ (vi) Make a move to the point  (1) =   -( )  for minimization problem and
The value of objective function at this point is f (  (1) ).
At this point we add  (+1) additional design point to   and thus form the new design measure (+1) ) In a minimization problem  (+1) is such that Similarly, in a maximization problem  (+1) is such that (viii) At the (k+1) st iteration, make a move to )  for minimization problem and )  for maximization problem.
The objective function has the value f (  (+1) ) at this point.

Comparing The Quick Convergent Inflow Algorithm and the Modified Quick Convergent Inflow Algorithm
For the two algorithms the feasible region comprises of a continuum of points.The support points that make up the design measure must satisfy the m linear inequality constraints and must result in a non-singular information matrix.To obtain a non-singular information matrix, the number of distinct support points must not be less than the model parameters.The starting point of search is the average of the initial support points and it satisfies the linear inequality constraints.Infact, since the region of search is convex, the starting point of search is a feasible point of the problem and consequently satisfies the constraints.The direction of search is ∇ which is  , the function being linear in the variables.Thus for both algorithms, the search is in the direction of the gradient and the step length of the search continues to decrease in the region of optimality.
In comparing the working of the two algorithms, we commence search for the optimizer of an objective function in an LP problem from the same initial experimental design.After a first move has been made, the Quick Convergent Inflow Algorithm (QCIA) adds a point reached by the line equation to the initial experimental design and the process continues until convergence is established.On the other hand, the Modified Quick Convergent Inflow Algorithm (MQCIA) adds a point having minimum predictive variance to the initial experimental design in a minimization problem and alternatively adds a point having maximum predictive variance to the initial experimental design in a maximization problem.The algorithm stops when the addition of a point on an existing experimental design measure does not improve the design as measured by the value of the objective function of the optimizer at the current iteration.Where there is no justification to terminate the search at the current iteration, the process continues.

The New Method
The success of the Modified Quick Convergent Inflow Algorithm (MQCIA) of Iwundu and Ebong (2014) lies fundamentally on the fact that by improving an experimental design, the optimizer of a response function is approached.Many techniques are suggested on the subject of designing experiments, on how to improve an existing experimental design.They include single augmentation processes, multiple augmentation processes, addition/deletion methods, exchange methods etc.These techniques are available in literatures on optimal design of experiments and include Mitchell-Miller (1970), Van Schalkwyk (1971), Fedorov (1972), Mitchell (1974), Cook and Nachtsheim (1980), Johnson and Nachtsheim (1983), Atkinson and Donev (1992).
Elimination technique has been one of the methods employed in obtaining optimal choices.One of such techniques is the Backward elimination method used in model building.The process of Elimination, usually iterative, identifies entities of interest that do not significantly contribute to the system.The fundamental idea embodied in the new algorithm is that from an existing N-point design, we eliminate or delete a point in the design having a minimum variance of prediction in a maximization problem or we eliminate or delete a point in the design having a maximum predictive variance in a minimization problem.The design size is immediately recoverd by adding to the resulting N-1 sized design a design point from the candidate set that optimizes performance.Specifically, the point added in a maximization problem has maximum predictive variance and the point added in a minimization problem has minimum predictive variance.
The Elimination technique is an alternative to the Modified Quick Convergent Inflow Algorithm (MQCIA) and serve as an approximation to the exact method.Apart from the way the experimental design measure is formed and improved, other steps of the new algorithm are as in the Modified Quick Convergent Inflow Algorithm.
Hence the Elimination technique is defined by the following sequencial steps; (i) Obtain from the feasible region ̃ grid of points  (1) ,  ( ) , ⋯,  ( ̃) that satisfy the constraint equations, to make up the candidate set S = {  (1) ,  ( ) , ⋯,  ( ̃) .(ii) From the ̃ grid of points, select N design points (N ≤ ̃) to make up an n-variate non-singular initial design,   .
Without loss of generality we write ( )    ()  ) Where  is the design matrix for   and the corresponding information matrix is   =  ʹ  and normalized as N -1   .

(iii)
Obtain the starting point of search as where   is the average of the initial design points at the k th iteration.Here k = 0.
(iv) Obtain the direction vector, d.The direction of search is ∇ which is  where ] is the vector of coefficients of the objective function.
(v) Obtain the step-length of search ( ) , where ;  = ‖ ‖ (vi) Make a move to the next point of search
At this point the objective function has the value f (  (1) ).
Let   () be a point in the design measure having the minimum variance of prediction.Then is the predictive variance associated with   () .
Also, let   ( ) be a point in the candidate set having the maximum variance of prediction.Then is the predictive variance associated with   ( ) .Similarly, let   ( ) be a point in the design measure having the maximum variance of prediction.Then is the predictive variance associated with   ( ) .
Also, let   () be a point in the candidate set having the minimum variance of prediction.Then is the predictive variance associated with   () .In a maximization problem   () is such that Similarly, in a minimization problem   ( ) is such that The objective function has the value f (  (+1) ) at this point.
With the starting design ξ = we obtain the results as tabulated in table 2 1 2 3 0  Illustration 2 Using the Quick Convergent Inflow Algorithm with the initial design measure as ξ = we obtain the results as tabulated in table 5 Table 5. Iterative steps of the QCIA for problem 2

Determinant Of Info Matrix
Optimizer Value Of Obj.Function

Normalized direction of search
Step length The global mimimizer of the objective function is  * = ( ) and the value of the objective function at  * is -2.0000.The optimizer of the objective function was reached at the first iteration using the Quick Convergent Inflow Algorithm.
With  ,  and a move is made to The minimum predictive variance in the design measure is attained at the design point (0,1) and the maximun predictive variance in the candidate set is attained at the design point (4,0).With these we eliminate the point (0,1) from the design and form a new design measure The design matrix associated with the design measure  1 is The starting point of search is  1 = ( ) The direction of search and optimal step length are respectively,  1 = ( ) and 1 = 0.4176.
With  1 ,  1 and 1 a move is made to At this iteration, the minimum predictive variance in the design measure is attained at the design point (3,1.5)and the maximun predictive variance in the set is attained at the design point (1,2).With these we eliminate the point (3,1.5)from the design and form a new design measure  = ( ) The design matrix associated with the design measure  is = ( ) The starting point of search is  = ( ) The direction of search and optimal step length are respectively,  = ( ) and = 0.6496120535.
With  ,  and a move is made to At this iteration, the minimum predictive variance in the design measure is attained at the design point (3,0.5)and the maximun predictive variance in the candidate set is attained at the design point (4,0).With these we eliminate the point (3,0.5)from the design and form a new design measure  = ( ) The design matrix associated with the design measure  is = ( ).The two algorithms converged at k = 1 to the exact solution ( ).Unfortunately, with the starting design  ( ) = ( ) , the Quick Convergent Inflow Algorithm could not arrive at an optimal or near-optimal solution as the value of objective function remained stationary from one iteration to another.Furthermore, the solution reached is far away from the exact optimum, the norm of the two vectors being 0.9428.The introduction of the Elimination method stems from the success of the Modified Quick Convergent Inflow Algorithm.In addition to adding to an existing experimental design a point that optimizes performance as seen in the Modified Quick Convergent Inflow Algorithm, the elimination method expunges from the experimental design a point that contributes less to the process.The Elimination method, though an approximate method, serves as an alternative to the Modified Quick Convergent Inflow Algorithm.The method converges and has proven comparatively satisfactory.For the maximization problem in Illustration 1, the elimination method converged at k=3 to the optimizer ( ) with value of objective function 20.9929.The norm of the vector reached and the exact optimal vector ( ) is 0.01309.For the minimization problem in Illustration 2, the Elimination method converged at k=3 to the optimizers ( ) with the value of objective function being f() = -1.9815.The exact solution to this problem is ( ) and the corresponding value of objective function is -2.
The norm of the vectors of approximate and exact solutions is 0.1859.
It may be helpful when applying any of the three algorithms to commence search from r starting designs.By so

Conclution
The performance of two line search algorithms has been studied numerically.The effectiveness of the algorithms is influenced by the starting experimental design.It is therefore important to commence search from several starting designs.The new Elimination algorithm is a useful alternative to the Modified Quick Convergent Inflow Algorithm.It preserves the design size from one iteration to another.In particular, it eliminates from an N-sized starting design a point that contributes less to the process as measured by the predictive variances at the design points.The design size is immediately recovered by adding to the resulting N-1 sized design a design point from the candidate set that optimizes performance.Numerical illustrations show that the Elimination algorithm converges to a near-optimum solution.

S 3 :
Determine the information matrix, Obtain the determinant of the information matrix, say, det ( k M ).

S 5 :
Obtain the variance-covariance matrix, 1  k M , of the information matrix.S 6 : Relate the coefficients of the objective function with the information matrix by

S 7 :
Determine the direction of search k d , where k d is an n-component vector defined by

S 12 :
Setting k = k+1 and N = N+1 add the points * 1  k x in step S 10 above to the design measure in step S 1 and if * k ) At this point in a maximization problem, delete  () from the initial N-point design measure and form a new N-point design measure by adding   ( ) to the resulting N-1 sized design.Similarly in a minimization problem, delete   ( ) from the initial N-point design measure and form a new N-point design measure by adding  () to the resulting N-1 sized design.(ix)Using the new design points, compute the starting point of search at (k+1) st iteration as k+1) st iteration, make a move to the next point of search  (+1) =  +1 -()  for minimization problem and  (+1) =  +1 + ()  for maximization problem.

Table 1 .
Iterative steps of the QCIA for problem 1 * = ( ) and the value of the objective function at  * is 20.30784.The optimizer of the objective function was reached at the second iteration using the Quick Convergent Inflow Algorithm.

Table 2 .
Iterative steps of the MQCIA for problem 1 The global maximum of the objective function is  * = ( ) and the value of the objective function at  * is 20.9565.The optimizer of the objective function was reached at the third iteration using the Modified we obtain the results as tabulated in table 3 using the QCIA.

Table 3 .
Iterative steps of the QCIA using a new starting design on problem 1 is 20.942.The optimizer of the objective function was reached at the first iteration using the Quick * = ( ) and the value of the objective function at  *

Table 4 .
Iterative steps of the MQCIA using a new starting design on problem 1 The global maximum of the objective function is  * = ( ) and the value of the objective function at  * is 20.942.The optimizer of the objective function was reached at the first iteration using the Modified Quick Convergent Inflow Algorithm.

Table 6 .
Iterative steps of the MQCIA for problem 2The global mimimizer of the objective function is  * = () and the value of the objective function at  * is -2.0000.The optimizer of the objective function was reached at the first iteration using the Modified Quick

Table 7 .
Iterative steps of the QCIA using a new starting design on problem 2

Table 8 .
Iterative steps of the MQCIA using a new starting design on problem 2 As opposed to the Quick Convergent Inflow Algorithm, the Modified Quick Convergent Inflow Algorithm converged at k = 2 to the value of objective function, f() = -2.0000with() as the optimizers.The norm of the vector of optimizers reached by the Modified Quick Convergent Inflow Algorithm and vector of the exact solution is 0.0001.The Modified Quick Convergent Inflow Algorithm seems to perform generally better than the Quick Convergent Inflow Algorithm in the sense that solutions obtained are much closer than those obtained using the Quick Convergent Inflow Algorithm.