Table 2 summarizes all the test functions that we used in the analysis and compares both NM algorithms with the best of the five genetically obtained algorithms. For each of the three algorithms, and for each of the test functions, one can see a computed minimum value and how many evaluations it took to get to this minimum. Apart from that, we summarize the known minima of the test functions in the last column of the table. The computed minimum values in bold match one of the known minima up to at least the double precision used in our computations. The results for quadratic functions are very interesting. We didn’t test the solvers on a quadratic function with n = 10, which is the number of parameters used in training. Instead, we took quadratic functions with 4, 8, 16, and 24 parameters. The tree-formed NM algorithm failed to arrive at the exact minimum in all cases except n = 4, while the original NM was successful

Table 2:
A comparison between the original and tree-formed NM algorithms and the best of the five genetically obtained solvers by individual test functions. The table shows the dimensions of each test function (n), the minimum value that an algorithm arrived at, and the number of actual test function evaluations (NCF) before this minimum was reached. All the exactly computed minima (up to at least double precision) are written in bold. For reference, the last column shows the known minima of each test function.
Best GP Solver
Original NMTree-Formed NM(Solver 1)
MinimumMinimumMinimumActual
Test Function (n)(NCF)(NCF)(NCF)Minima
Rosenbrock (2) 1.2325 4.9303 4.4373 0.0
(313) (313) (867)
Freudenstein and Roth (2) 48.9842 48.9842 48.9842 0.0
(159) (218) (425) 48.9842
Powell badly scaled (2) 0.0 0.0 0.0 0.0
(804) (802) (1,957)
Brown badly scaled (2) 0.0 0.0 0.0 0.0
(405) (411) (1,349)
Beale (2) 0.0 0.0 0.0 0.0
(255) (259) (683)
Jennrich and Sampson (2) 124.362 124.362 124.362 124.362
(143) (2,342) (397)
McKinnon (2) −0.25 −0.25 −0.25 -0.25
(175) (176) (503)
Helical valley (3) 0.0 0.0 0.0 0.0
(3,566) (3,586) (10,679)
Bard (3) 8.2148 8.2148 8.2148 8.2148
(322) (769) (1,067)
Gaussian (3) 1.1279 1.1279 1.1279 1.1279
(368) (931) (870)
Meyer (3) 87.9458 87.9458 87.9458 87.9458
(1,892) (2,268) (4,511)
Gulf research (3) 6.5000 3.0092 4.3122 0.0
(3,948) (3,880) (16,324)
Box 3D (3) 7.5588 7.5588 0.0 0.0
(496) (886) (2,430)
Powell singular (4) 5.7442 2.1964 1.9509 0.0
(2,291) (2,565) (4,871)
Wood (4) 5.5910 2.0411 3.2183 0.0
(907) (908) (2,779)
Kowalik and Osborne (4) 3.0750 3.0750 3.0750 3.0750
(427) (9,332) (1,206) 1.0273
Brown and Dennis (4) 85,822.2 85,822.2 85,822.2 85,822.2
(426) (1,137) (1,288)
Quadratic (4) 0.0 0.0 0.0 0.0
(5,689) (5,623) (17,173)
Penalty I (4) 2.2499 2.8355 3.9053 2.2499
(1,365) (119) (289)
Penalty II (4) 9.3762 9.3762 9.3762 9.3762
(3,709) (5,940) (5,322)
Osborne 1 (5) 5.4648 5.4648 5.4648 5.4648
(1,273) (4,970) (2,790)
Brown almost linear (5) 1.5777 0.0 0.0 0.0
(1,092) (1,146) (2,788) 1.0
Biggs EXP6 (6) 5.6556 5.6556 5.3402 5.6556
(1,059) (4,450) (8,068) 0.0
Extended Rosenbrock (6) 3.1554 2.7240 3.9443 0.0
(4,635) (4,861) (7,494)
Brown almost linear (7) 9.6635 6.5081 7.8886 0.0
(2,311) (2,541) (4,104) 1.0
Quadratic (8) 0.0 1.9762 0.0 0.0
(18,964) (18,814) (39,785)
Extended Rosenbrock (8) 3.1914 7.6420 2.7523 0.0
(13,583) (15,961) (19,164)
Variably dimensioned (8) 0.0 9.2814 1.0292 0.0
(4,682) (4,856) (7,160)
Extended Powell (8) 4.4086 9.8782 9.7234 0.0
(11,251) (12,894) (20,353)
Watson (6) 2.2876 2.2876 2.2876 2.2876
(2,963) (5,120) (5,151)
Extended Rosenbrock (10) 9.72337 1.9748 9.0484 0.0
(7,821) (20,691) (36,268)
Penalty I (10) 7.5675 9.1907 9.4271 7.0876
(5,427) (1,419) (2,100)
Penalty II (10) 2.9778 2.9778 3.0000 2.9366
(6,342) (6,547) (1,543)
Trigonometric (10) 2.7950 2.7950 2.7950 0.0
(3,610) (4,277) (4,252)
Osborne 2 (11) 4.0137 4.0137 4.0137 4.0137
(4,821) (14,517) (7,381)
Extended Powell (12) 8.3858 1.0430 5.7700 0.0
(39,823) (39,093) (50,117)
Quadratic (16) 2.2211 3.2435 0.0 0.0
(39,965) (52,675) (112,564)
Quadratic (24) 0.58675 0.58527 8.0493 0.0
(39,976) (48,904) (158,849)
Best GP Solver
Original NMTree-Formed NM(Solver 1)
MinimumMinimumMinimumActual
Test Function (n)(NCF)(NCF)(NCF)Minima
Rosenbrock (2) 1.2325 4.9303 4.4373 0.0
(313) (313) (867)
Freudenstein and Roth (2) 48.9842 48.9842 48.9842 0.0
(159) (218) (425) 48.9842
Powell badly scaled (2) 0.0 0.0 0.0 0.0
(804) (802) (1,957)
Brown badly scaled (2) 0.0 0.0 0.0 0.0
(405) (411) (1,349)
Beale (2) 0.0 0.0 0.0 0.0
(255) (259) (683)
Jennrich and Sampson (2) 124.362 124.362 124.362 124.362
(143) (2,342) (397)
McKinnon (2) −0.25 −0.25 −0.25 -0.25
(175) (176) (503)
Helical valley (3) 0.0 0.0 0.0 0.0
(3,566) (3,586) (10,679)
Bard (3) 8.2148 8.2148 8.2148 8.2148
(322) (769) (1,067)
Gaussian (3) 1.1279 1.1279 1.1279 1.1279
(368) (931) (870)
Meyer (3) 87.9458 87.9458 87.9458 87.9458
(1,892) (2,268) (4,511)
Gulf research (3) 6.5000 3.0092 4.3122 0.0
(3,948) (3,880) (16,324)
Box 3D (3) 7.5588 7.5588 0.0 0.0
(496) (886) (2,430)
Powell singular (4) 5.7442 2.1964 1.9509 0.0
(2,291) (2,565) (4,871)
Wood (4) 5.5910 2.0411 3.2183 0.0
(907) (908) (2,779)
Kowalik and Osborne (4) 3.0750 3.0750 3.0750 3.0750
(427) (9,332) (1,206) 1.0273
Brown and Dennis (4) 85,822.2 85,822.2 85,822.2 85,822.2
(426) (1,137) (1,288)
Quadratic (4) 0.0 0.0 0.0 0.0
(5,689) (5,623) (17,173)
Penalty I (4) 2.2499 2.8355 3.9053 2.2499
(1,365) (119) (289)
Penalty II (4) 9.3762 9.3762 9.3762 9.3762
(3,709) (5,940) (5,322)
Osborne 1 (5) 5.4648 5.4648 5.4648 5.4648
(1,273) (4,970) (2,790)
Brown almost linear (5) 1.5777 0.0 0.0 0.0
(1,092) (1,146) (2,788) 1.0
Biggs EXP6 (6) 5.6556 5.6556 5.3402 5.6556
(1,059) (4,450) (8,068) 0.0
Extended Rosenbrock (6) 3.1554 2.7240 3.9443 0.0
(4,635) (4,861) (7,494)
Brown almost linear (7) 9.6635 6.5081 7.8886 0.0
(2,311) (2,541) (4,104) 1.0
Quadratic (8) 0.0 1.9762 0.0 0.0
(18,964) (18,814) (39,785)
Extended Rosenbrock (8) 3.1914 7.6420 2.7523 0.0
(13,583) (15,961) (19,164)
Variably dimensioned (8) 0.0 9.2814 1.0292 0.0
(4,682) (4,856) (7,160)
Extended Powell (8) 4.4086 9.8782 9.7234 0.0
(11,251) (12,894) (20,353)
Watson (6) 2.2876 2.2876 2.2876 2.2876
(2,963) (5,120) (5,151)
Extended Rosenbrock (10) 9.72337 1.9748 9.0484 0.0
(7,821) (20,691) (36,268)
Penalty I (10) 7.5675 9.1907 9.4271 7.0876
(5,427) (1,419) (2,100)
Penalty II (10) 2.9778 2.9778 3.0000 2.9366
(6,342) (6,547) (1,543)
Trigonometric (10) 2.7950 2.7950 2.7950 0.0
(3,610) (4,277) (4,252)
Osborne 2 (11) 4.0137 4.0137 4.0137 4.0137
(4,821) (14,517) (7,381)
Extended Powell (12) 8.3858 1.0430 5.7700 0.0
(39,823) (39,093) (50,117)
Quadratic (16) 2.2211 3.2435 0.0 0.0
(39,965) (52,675) (112,564)
Quadratic (24) 0.58675 0.58527 8.0493 0.0
(39,976) (48,904) (158,849)
with quadratic functions with n = 4 and n = 8. In the case of n = 24, the failure of both NM variants was very serious.

Close Modal