MOEA#
Toy tracker problem#
This section outlines the toy Problem that is used for the hands-on session through out the lectures with the goal to introduce optimization technique and useful frameworks in python.
Problem Statement#
Consider a toy wire based tracker. the toy wire based tracker consists in a 2D tracking system with of 4 layers of wires along z.
A total of 8 parameters can be tuned. The adjustable parameters are the radius of the wire, the pitch (along the y-axis), and the shift along y and z of a plane with respect to the previous one.
Straight tracks are generated at different angles and random origin. The tracker geometry and random origin. The tracker geometry and the tracker generation is already defined in the imported module detector.py
Objectives#
One can evaluate the performance of the tracker on multiple metrics. These are are refered to as objectives. In the following lectures, the objectives against which the performance of the tracker will be evaulated are,
Efficiency - Defined as the fraction of tracks that has atleast 2 wires hit in the event.
Volume - Volume occupied by the tracking system. This is a proxy for the cost of the tracker.
Resolution - Resolution extracted from the tracks.
import warnings; warnings.simplefilter('ignore')
!wget https://raw.githubusercontent.com/cfteach/modules/master/detector2.py
!pip install pymoo > /dev/null 2>&1
!pip install plotly > /dev/null 2>&1
!pip install ipyvolume > /dev/null 2>&1
!pip install altair > /dev/null 2>&1
--2025-06-08 17:04:39-- https://raw.githubusercontent.com/cfteach/modules/master/detector2.py
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.109.133, 185.199.111.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7976 (7.8K) [text/plain]
Saving to: ‘detector2.py’
detector2.py 100%[===================>] 7.79K --.-KB/s in 0s
2025-06-08 17:04:39 (93.4 MB/s) - ‘detector2.py’ saved [7976/7976]
%load_ext autoreload
%autoreload 2
import os
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
#import AI4NP_detector_opt.sol2.detector2 as detector2
import detector2
from pymoo.algorithms.moo.nsga2 import NSGA2
from pymoo.core.problem import Problem
from pymoo.optimize import minimize
from pymoo.visualization.scatter import Scatter
from pymoo.util.display.column import Column
from pymoo.util.display.output import Output
from pymoo.util.nds.non_dominated_sorting import NonDominatedSorting
Create detector geometry and simulate tracks#
The module detector creates a simple 2D geometry of a wire based tracker made by 4 planes.
The adjustable parameters are the radius of each wire, the pitch (along the y axis), and the shift along y and z of a plane with respect to the previous one.
A total of 8 parameters can be tuned.
The goal of this toy model, is to tune the detector design so to optimize the efficiency (fraction of tracks which are detected) as well as the cost for its realization. As a proxy for the cost, we use the material/volume (the surface in 2D) of the detector. For a track to be detetected, in the efficiency definition we require at least two wires hit by the track.
So we want to maximize the efficiency (defined in detector.py) and minimize the cost.
LIST OF PARAMETERS#
(baseline values)
R = .5 [cm]
pitch = 4.0 [cm]
y1 = 0.0, y2 = 0.0, y3 = 0.0, z1 = 2.0, z2 = 4.0, z3 = 6.0 [cm]
# CONSTANT PARAMETERS
#------ define mother region ------#
y_min=-10.1
y_max=10.1
N_tracks = 1000
print("::::: BASELINE PARAMETERS :::::")
R = 0.5
pitch = 4.0
y1 = 2.0
y2 = 1.0
y3 = 0.0
z1 = 2.0
z2 = 4.0
z3 = 6.0
print("R, pitch, y1, y2, y3, z1, z2, z3: ", R, pitch, y1, y2, y3, z1, z2, z3,"\n")
#------------- GEOMETRY ---------------#
print(":::: INITIAL GEOMETRY ::::")
tr = detector2.Tracker(R, pitch, y1, y2, y3, z1, z2, z3)
Z, Y = tr.create_geometry()
num_wires = detector2.calculate_wires(Y, y_min, y_max)
volume = detector2.wires_volume(Y, y_min, y_max,R)
detector2.geometry_display(Z, Y, R, y_min=y_min, y_max=y_max,block=False,pause=5) #5
print("# of wires: ", num_wires, ", volume: ", volume)
#------------- TRACK GENERATION -----------#
print(":::: TRACK GENERATION ::::")
t = detector2.Tracks(b_min=y_min, b_max=y_max, alpha_mean=0, alpha_std=0.3)
tracks = t.generate(N_tracks)
detector2.geometry_display(Z, Y, R, y_min=y_min, y_max=y_max,block=False, pause=-1)
detector2.tracks_display(tracks, Z,block=False,pause=-1)
#a track is detected if at least two wires have been hit
score = detector2.get_score(Z, Y, tracks, R)
frac_detected = score[0]
resolution = score[1]
print("fraction of tracks detected: ",frac_detected)
print("resolution: ",resolution)
::::: BASELINE PARAMETERS :::::
R, pitch, y1, y2, y3, z1, z2, z3: 0.5 4.0 2.0 1.0 0.0 2.0 4.0 6.0
:::: INITIAL GEOMETRY ::::
# of wires: 21 , volume: 65.94
:::: TRACK GENERATION ::::
fraction of tracks detected: 0.259
resolution: 0.2405574316639721
Define Objectives#
Defines a class for the objectives of the problem that can be used in the MOO.
# The objectives are computed by setting up the geometry and
# computing the values of the objectives. Mimicing a G4 type simulation
class objectives():
def __init__(self,tracks,y_min,y_max):
self.tracks = tracks
self.y_min = y_min
self.y_max = y_max
def wrapper_geometry(fun):
def inner(self):
R, pitch, y1, y2, y3, z1, z2, z3 = self.X
self.geometry(R, pitch, y1, y2, y3, z1, z2, z3)
return fun(self)
return inner
def update_tracks(self, new_tracks):
self.tracks = new_tracks
def update_design_point(self,X):
self.X = X
def geometry(self,R, pitch, y1, y2, y3, z1, z2, z3):
tr = detector2.Tracker(R, pitch, y1, y2, y3, z1, z2, z3)
self.R = R
self.Z, self.Y = tr.create_geometry()
@wrapper_geometry
def calc_score(self):
res = detector2.get_score(self.Z, self.Y, self.tracks, self.R)
assert res[0] >= 0 and res[1] >= 0,"Fraction or Resolution negative."
return res
def get_score(self,X):
R, pitch, y1, y2, y3, z1, z2, z3 = X
self.geometry(R, pitch, y1, y2, y3, z1, z2, z3)
res = detector2.get_score(self.Z, self.Y, self.tracks, self.R)
return res
def get_volume(self):
volume = detector2.wires_volume(self.Y, self.y_min, self.y_max,self.R)
return volume
res = objectives(tracks,y_min,y_max)
#res.geometry(R, pitch, y1, y2, y3, z1, z2, z3)
X = R, pitch, y1, y2, y3, z1, z2, z3
#fscore = res.get_score(X)
res.update_design_point(X)
fscore = res.calc_score()[0]
fvolume = res.get_volume()
print("...check: ", fvolume, fscore)
...check: 65.94 0.259
Multi-Objective Optimization#
We will be using pymoo (https://pymoo.org/getting_started.html).
In the constructor method we specify number of variables N, objectives M, and constraint functions, as well as the lower and upper boundaries of each variable. In our toy model, these boundaries are taken in such a way that all solutions are feasible and no constraint function is needed. You can try to change this and introduce some constraint.
The _evaluate method takes a one-dimensional NumPy array x number of entries equal to n_var. This behavior is enabled by setting elementwise_evaluation=True while calling the super() method.
Notice that every function is minimized. Our efficiency is defined as an tracking inefficiency = 1 - efficiency
We add the resolution as a third objective. The average residual of the track hit from the wire centre is used as a proxy for the resolution for this toy-model
from pymoo.core.problem import ElementwiseProblem
class MyProblem(ElementwiseProblem):
#--------- vectorized ---------#
def __init__(self, n_var=8, n_obj=2, n_constr=0,
xl=np.array([0.5,2.5,0.,0.,0.,2.,2.,2.]),
xu=np.array([1.0,5.0,4.,4.,4.,10.,10.,10.])
):
super().__init__(n_var=n_var, n_obj=n_obj, n_constr=n_constr, xl=xl, xu=xu)
def _evaluate(self, x, out, *args, **kwargs):
f1 = 1.- res.get_score(x)[0] # Ineffiency
f2 = res.get_volume() # volume
#f3 = res.get_score(x)[1] #resolution
out["F"] = [f1, f2]#, f3]
Creation of Problem and choice of optimization algorithm.#
We will use NSGA-II, as explained in the lectures. You can decide the population size and the number of offsprings, based on what we discussed.
Pymoo offers different algorithms that can be used which are highly customizable and can be easily extended. https://pymoo.org/algorithms/index.html
Before dealing with a problem, it’s useful to compare with a list of test problems reported in https://pymoo.org/problems/index.html, where different scenarios in terms of Variables, Objectives, Constraints are described.
n_var = 8 # number of design variables
n_obj = 2 # number of objectives
n_constr = 0 # number of constraints
xl = np.array([0.5,2.5,0.,0.,0.,2.,2.,2.]) # lower bounds of the problem, pitch, diameter, y1, y2, y3, z1, z2, z3
xu = np.array([1.0,5.0,4.,4.,4.,10.,10.,10.]) # upper bounds of the problem, pitch, diameter, y1, y2, y3, z1, z2, z3
pop_size = 40
n_offsprings=10
problem = MyProblem(n_var=n_var, n_obj=n_obj, n_constr=n_constr, xl=xl, xu=xu)
from pymoo.algorithms.moo.nsga2 import NSGA2
from pymoo.operators.crossover.sbx import SBX
from pymoo.operators.mutation.pm import PM
from pymoo.operators.sampling.rnd import FloatRandomSampling
from pymoo.termination import get_termination
from pymoo.termination.xtol import DesignSpaceTermination
from pymoo.termination.robust import RobustTermination
# check out other sampling methods here https://pymoo.org/operators/sampling.html
# check out other crossover methods here https://pymoo.org/operators/crossover.html
algorithm = NSGA2(pop_size=pop_size,n_offsprings=10)
crossover = SBX(prob=1.0, eta=1)
mutation = PM(eta=1)
termination = RobustTermination(DesignSpaceTermination(tol=0.01), period=20)
res = minimize(problem,
algorithm,
crossover = crossover,
mutation = mutation,
termination = termination,
verbose=True,
seed=1,
save_history=True # Saves all of the history of the algorithm
)
==========================================================
n_gen | n_eval | n_nds | eps | indicator
==========================================================
1 | 40 | 11 | - | -
2 | 50 | 13 | 0.0871021776 | ideal
3 | 60 | 16 | 0.1372832370 | ideal
4 | 70 | 17 | 0.0034332054 | f
5 | 80 | 19 | 0.0138353295 | f
6 | 90 | 19 | 0.0093112237 | f
7 | 100 | 21 | 0.0101346341 | f
8 | 110 | 24 | 0.0029376102 | f
9 | 120 | 26 | 0.0212164074 | ideal
10 | 130 | 30 | 0.0048070233 | f
11 | 140 | 34 | 0.0037747897 | f
12 | 150 | 35 | 0.0261707989 | ideal
13 | 160 | 35 | 0.0081967213 | ideal
14 | 170 | 36 | 0.0042495998 | f
15 | 180 | 35 | 0.0025737781 | ideal
16 | 190 | 35 | 0.0055096419 | nadir
17 | 200 | 37 | 0.0021475795 | f
18 | 210 | 38 | 0.0033018330 | f
19 | 220 | 38 | 0.0014084363 | f
20 | 230 | 36 | 0.0051750618 | f
21 | 240 | 39 | 0.0030777914 | f
22 | 250 | 40 | 0.0004512055 | f
23 | 260 | 40 | 0.0019378695 | f
24 | 270 | 40 | 0.0031356632 | f
25 | 280 | 40 | 0.0018864889 | f
26 | 290 | 40 | 0.0037118183 | f
27 | 300 | 40 | 0.0005345111 | f
28 | 310 | 37 | 0.0034453848 | f
29 | 320 | 35 | 0.0007127194 | f
30 | 330 | 39 | 0.0024632977 | f
31 | 340 | 39 | 0.0055401662 | nadir
32 | 350 | 40 | 0.0009238943 | f
33 | 360 | 40 | 0.0013676146 | f
34 | 370 | 40 | 0.0109589041 | ideal
35 | 380 | 40 | 0.0004454631 | f
36 | 390 | 40 | 0.0032116838 | f
37 | 400 | 40 | 0.000000E+00 | f
38 | 410 | 40 | 0.0012674082 | f
39 | 420 | 40 | 0.0013873061 | f
40 | 430 | 40 | 0.0014899052 | f
41 | 440 | 40 | 0.0054421769 | ideal
42 | 450 | 40 | 0.000000E+00 | f
43 | 460 | 40 | 0.0002040816 | f
44 | 470 | 39 | 0.0016481057 | f
45 | 480 | 39 | 0.0029361551 | f
46 | 490 | 39 | 0.0001395430 | f
47 | 500 | 40 | 0.0021085939 | f
48 | 510 | 40 | 0.0033714327 | f
49 | 520 | 40 | 0.0003692177 | f
50 | 530 | 40 | 0.0011959046 | f
51 | 540 | 40 | 0.0019442040 | f
52 | 550 | 40 | 0.0022730790 | f
53 | 560 | 36 | 0.0027137042 | ideal
54 | 570 | 38 | 0.1153944328 | nadir
55 | 580 | 39 | 0.0017101727 | f
56 | 590 | 38 | 0.0023680533 | f
57 | 600 | 39 | 0.0095759234 | nadir
58 | 610 | 40 | 0.0013871604 | f
59 | 620 | 38 | 0.0026538160 | f
60 | 630 | 40 | 0.0008785492 | f
61 | 640 | 39 | 0.1304328300 | nadir
62 | 650 | 38 | 0.0004128484 | f
63 | 660 | 37 | 0.0018002286 | f
64 | 670 | 39 | 0.0022656368 | f
65 | 680 | 40 | 0.0028459596 | f
66 | 690 | 40 | 0.0013359151 | f
67 | 700 | 40 | 0.0040871935 | ideal
68 | 710 | 40 | 0.0011913540 | f
69 | 720 | 40 | 0.0018853333 | f
70 | 730 | 40 | 0.0067658999 | ideal
71 | 740 | 36 | 0.0020183791 | f
72 | 750 | 35 | 0.0030924854 | f
73 | 760 | 36 | 0.0004979059 | f
74 | 770 | 36 | 0.0004979059 | f
75 | 780 | 39 | 0.0339869281 | nadir
76 | 790 | 39 | 0.0039370079 | nadir
77 | 800 | 39 | 0.000000E+00 | f
78 | 810 | 40 | 0.0003293652 | f
79 | 820 | 40 | 0.0015105658 | f
80 | 830 | 40 | 0.0015105658 | f
81 | 840 | 38 | 0.0036463685 | f
82 | 850 | 39 | 0.0009362785 | f
83 | 860 | 40 | 0.0018519342 | f
84 | 870 | 40 | 0.0023008236 | f
85 | 880 | 40 | 0.0026990944 | f
86 | 890 | 40 | 0.000000E+00 | f
87 | 900 | 40 | 0.0001421387 | f
88 | 910 | 40 | 0.0005686479 | f
89 | 920 | 40 | 0.0012968010 | f
90 | 930 | 40 | 0.0018809364 | f
91 | 940 | 40 | 0.0037484296 | f
92 | 950 | 40 | 0.0004041196 | f
93 | 960 | 40 | 0.0004041196 | f
94 | 970 | 40 | 0.0012171399 | f
95 | 980 | 40 | 0.0028474014 | f
96 | 990 | 40 | 0.0017459606 | f
97 | 1000 | 40 | 0.0017459606 | f
98 | 1010 | 40 | 0.0021841178 | f
99 | 1020 | 39 | 0.0036804709 | f
100 | 1030 | 40 | 0.0008938589 | f
101 | 1040 | 40 | 0.0012089796 | f
102 | 1050 | 40 | 0.0025379159 | f
103 | 1060 | 40 | 0.0002952756 | f
104 | 1070 | 40 | 0.0015769598 | f
105 | 1080 | 40 | 0.0021359743 | f
106 | 1090 | 40 | 0.0031071820 | f
107 | 1100 | 40 | 0.0008202198 | f
108 | 1110 | 40 | 0.0019695235 | f
109 | 1120 | 40 | 0.0066050198 | nadir
110 | 1130 | 40 | 0.0003232341 | f
111 | 1140 | 40 | 0.0003232341 | f
112 | 1150 | 40 | 0.0004877058 | f
113 | 1160 | 40 | 0.0005726696 | f
114 | 1170 | 40 | 0.0005726696 | f
115 | 1180 | 39 | 0.0008583286 | f
116 | 1190 | 40 | 0.0013826204 | f
117 | 1200 | 40 | 0.0013826204 | f
118 | 1210 | 40 | 0.0014486706 | f
119 | 1220 | 40 | 0.0015477459 | f
120 | 1230 | 40 | 0.0015477459 | f
121 | 1240 | 40 | 0.0631188119 | nadir
122 | 1250 | 40 | 0.0003571164 | f
123 | 1260 | 40 | 0.0006697341 | f
124 | 1270 | 40 | 0.0006697341 | f
125 | 1280 | 39 | 0.0520833333 | nadir
126 | 1290 | 40 | 0.0004107358 | f
127 | 1300 | 40 | 0.0166453265 | nadir
128 | 1310 | 40 | 0.0006090277 | f
129 | 1320 | 40 | 0.0006730482 | f
130 | 1330 | 40 | 0.0025673941 | nadir
131 | 1340 | 40 | 0.0011755222 | f
132 | 1350 | 40 | 0.0025657359 | f
133 | 1360 | 40 | 0.0002114598 | f
134 | 1370 | 40 | 0.0007613258 | f
135 | 1380 | 40 | 0.0011801038 | f
136 | 1390 | 40 | 0.0014050395 | f
137 | 1400 | 40 | 0.0021458622 | f
138 | 1410 | 40 | 0.0028045006 | f
139 | 1420 | 40 | 0.0004590571 | f
140 | 1430 | 40 | 0.0014654146 | f
141 | 1440 | 40 | 0.0014654146 | f
142 | 1450 | 40 | 0.0014654146 | f
143 | 1460 | 40 | 0.0176767677 | nadir
144 | 1470 | 40 | 0.0008078188 | f
145 | 1480 | 40 | 0.0010993989 | f
146 | 1490 | 40 | 0.0013834898 | f
147 | 1500 | 40 | 0.0010935075 | f
148 | 1510 | 40 | 0.0019886628 | f
149 | 1520 | 40 | 0.0023565324 | f
150 | 1530 | 40 | 0.0024178212 | f
151 | 1540 | 40 | 0.0037214191 | f
152 | 1550 | 40 | 0.0008936919 | f
153 | 1560 | 40 | 0.0008936919 | f
154 | 1570 | 40 | 0.0011462171 | f
155 | 1580 | 40 | 0.0366492147 | nadir
156 | 1590 | 40 | 0.0026109661 | ideal
157 | 1600 | 40 | 0.0004033669 | f
158 | 1610 | 40 | 0.0004693168 | f
159 | 1620 | 40 | 0.0006405200 | f
160 | 1630 | 40 | 0.0007928876 | f
161 | 1640 | 40 | 0.0010615733 | f
162 | 1650 | 40 | 0.0026215868 | f
163 | 1660 | 40 | 0.000000E+00 | f
164 | 1670 | 40 | 0.0010957789 | f
165 | 1680 | 40 | 0.0015372429 | f
166 | 1690 | 40 | 0.0019172949 | f
167 | 1700 | 40 | 0.0034568194 | f
168 | 1710 | 40 | 0.000000E+00 | f
169 | 1720 | 40 | 0.0005439098 | f
170 | 1730 | 40 | 0.0090439276 | nadir
171 | 1740 | 40 | 0.0000322997 | f
172 | 1750 | 40 | 0.0006573678 | f
173 | 1760 | 40 | 0.0007155438 | f
174 | 1770 | 40 | 0.0010062415 | f
175 | 1780 | 40 | 0.0025773196 | ideal
176 | 1790 | 40 | 0.0004202123 | f
177 | 1800 | 40 | 0.0007229533 | f
178 | 1810 | 40 | 0.0020523486 | f
179 | 1820 | 40 | 0.0029410007 | f
180 | 1830 | 40 | 0.0012932801 | f
181 | 1840 | 40 | 0.0012932801 | f
182 | 1850 | 40 | 0.0013577678 | f
183 | 1860 | 40 | 0.0013577678 | f
184 | 1870 | 40 | 0.0014866338 | f
185 | 1880 | 40 | 0.0143790850 | nadir
186 | 1890 | 40 | 0.0003656509 | f
187 | 1900 | 40 | 0.0005464097 | f
188 | 1910 | 40 | 0.0007260360 | f
189 | 1920 | 40 | 0.0011583793 | f
190 | 1930 | 40 | 0.0012237388 | f
191 | 1940 | 40 | 0.0012237388 | f
192 | 1950 | 40 | 0.0013217780 | f
193 | 1960 | 40 | 0.0013199456 | f
194 | 1970 | 40 | 0.0014179848 | f
195 | 1980 | 40 | 0.0018428214 | f
196 | 1990 | 40 | 0.0018155948 | f
197 | 2000 | 40 | 0.0020115157 | f
198 | 2010 | 40 | 0.0027233316 | f
199 | 2020 | 40 | 0.000000E+00 | f
200 | 2030 | 40 | 0.0001307190 | f
201 | 2040 | 40 | 0.0005724210 | f
202 | 2050 | 40 | 0.0014337063 | f
203 | 2060 | 40 | 0.0016394168 | f
204 | 2070 | 40 | 0.0016856970 | f
205 | 2080 | 40 | 0.0022507372 | f
206 | 2090 | 40 | 0.0026102143 | f
207 | 2100 | 40 | 0.0005228758 | f
208 | 2110 | 40 | 0.0005570404 | f
209 | 2120 | 40 | 0.0006550796 | f
210 | 2130 | 40 | 0.0009051468 | f
211 | 2140 | 40 | 0.0019920214 | f
212 | 2150 | 40 | 0.0029332910 | f
213 | 2160 | 39 | 0.0010169764 | f
214 | 2170 | 39 | 0.0010169764 | f
215 | 2180 | 40 | 0.0009915520 | f
216 | 2190 | 40 | 0.0009915520 | f
217 | 2200 | 40 | 0.0012600368 | f
218 | 2210 | 40 | 0.0016211921 | f
219 | 2220 | 40 | 0.0016211921 | f
220 | 2230 | 40 | 0.0017845908 | f
221 | 2240 | 40 | 0.0064935065 | nadir
222 | 2250 | 40 | 0.0001623377 | f
223 | 2260 | 40 | 0.0025861083 | f
224 | 2270 | 40 | 0.0008456861 | f
225 | 2280 | 40 | 0.0008781536 | f
226 | 2290 | 40 | 0.0014701746 | f
227 | 2300 | 40 | 0.0517241379 | nadir
228 | 2310 | 40 | 0.0004789981 | f
229 | 2320 | 40 | 0.0009696589 | f
230 | 2330 | 40 | 0.0009696589 | f
231 | 2340 | 40 | 0.0011543879 | f
232 | 2350 | 40 | 0.0015243866 | f
233 | 2360 | 40 | 0.0015243866 | f
234 | 2370 | 40 | 0.0016090988 | f
235 | 2380 | 40 | 0.0018425963 | f
236 | 2390 | 40 | 0.0023782345 | f
237 | 2400 | 40 | 0.0028708430 | f
238 | 2410 | 40 | 0.0010923123 | f
239 | 2420 | 40 | 0.0014269031 | f
240 | 2430 | 40 | 0.0022994087 | f
241 | 2440 | 40 | 0.0023609851 | f
242 | 2450 | 40 | 0.0036243301 | f
243 | 2460 | 40 | 0.0011241390 | f
244 | 2470 | 40 | 0.0011241390 | f
245 | 2480 | 40 | 0.0022262385 | f
246 | 2490 | 40 | 0.0027808808 | f
247 | 2500 | 40 | 0.000000E+00 | f
248 | 2510 | 40 | 0.0001231527 | f
249 | 2520 | 40 | 0.0009632735 | f
250 | 2530 | 40 | 0.0009632735 | f
251 | 2540 | 40 | 0.0013327391 | f
252 | 2550 | 40 | 0.0014251036 | f
253 | 2560 | 40 | 0.0017329854 | f
254 | 2570 | 40 | 0.0020408672 | f
255 | 2580 | 40 | 0.0020408672 | f
256 | 2590 | 40 | 0.0021640199 | f
257 | 2600 | 40 | 0.0021640199 | f
258 | 2610 | 40 | 0.0017317051 | f
259 | 2620 | 40 | 0.0014867197 | f
260 | 2630 | 40 | 0.0014867197 | f
261 | 2640 | 40 | 0.0014867197 | f
262 | 2650 | 40 | 0.0019296484 | f
263 | 2660 | 40 | 0.0024755597 | f
264 | 2670 | 40 | 0.0025679242 | f
265 | 2680 | 40 | 0.0004002463 | f
266 | 2690 | 40 | 0.0005541872 | f
267 | 2700 | 40 | 0.0005541872 | f
268 | 2710 | 40 | 0.0005541872 | f
269 | 2720 | 40 | 0.0005541872 | f
270 | 2730 | 40 | 0.0005541872 | f
271 | 2740 | 40 | 0.0006157635 | f
272 | 2750 | 40 | 0.0009445070 | f
273 | 2760 | 40 | 0.0009445070 | f
274 | 2770 | 40 | 0.0013453647 | f
275 | 2780 | 40 | 0.0013508097 | f
276 | 2790 | 40 | 0.0018414564 | f
277 | 2800 | 40 | 0.0024866716 | f
278 | 2810 | 40 | 0.0032108381 | f
279 | 2820 | 40 | 0.000000E+00 | f
280 | 2830 | 40 | 0.0005503361 | f
281 | 2840 | 40 | 0.0013963636 | f
282 | 2850 | 40 | 0.0020817318 | f
283 | 2860 | 40 | 0.0099502488 | nadir
284 | 2870 | 40 | 0.0002487562 | f
285 | 2880 | 40 | 0.0004485017 | f
286 | 2890 | 39 | 0.0005556772 | f
287 | 2900 | 39 | 0.0016033853 | f
288 | 2910 | 40 | 0.0019220906 | f
289 | 2920 | 40 | 0.0019220906 | f
290 | 2930 | 40 | 0.0028576611 | f
291 | 2940 | 40 | 0.0009616552 | f
292 | 2950 | 40 | 0.0009927497 | f
293 | 2960 | 40 | 0.0016689810 | f
294 | 2970 | 40 | 0.0018386778 | f
295 | 2980 | 40 | 0.0021638044 | f
296 | 2990 | 40 | 0.0021638044 | f
297 | 3000 | 40 | 0.0021638044 | f
298 | 3010 | 39 | 0.0606427719 | nadir
299 | 3020 | 40 | 0.0006870042 | f
300 | 3030 | 40 | 0.0006870042 | f
301 | 3040 | 40 | 0.0007802878 | f
302 | 3050 | 40 | 0.0007802878 | f
303 | 3060 | 40 | 0.0003927343 | f
304 | 3070 | 40 | 0.0008706544 | f
305 | 3080 | 40 | 0.0013452138 | f
306 | 3090 | 40 | 0.0015006864 | f
307 | 3100 | 40 | 0.0023079250 | f
308 | 3110 | 40 | 0.0023079250 | f
309 | 3120 | 40 | 0.0020790636 | f
310 | 3130 | 40 | 0.0029802325 | f
311 | 3140 | 40 | 0.0008581513 | f
312 | 3150 | 40 | 0.0009825294 | f
313 | 3160 | 40 | 0.0009825294 | f
314 | 3170 | 40 | 0.0013245692 | f
315 | 3180 | 40 | 0.0013556637 | f
316 | 3190 | 40 | 0.0017053789 | f
317 | 3200 | 40 | 0.0028001057 | f
318 | 3210 | 40 | 0.0007817164 | f
319 | 3220 | 40 | 0.0012994897 | f
320 | 3230 | 40 | 0.0015589441 | f
321 | 3240 | 40 | 0.0017372302 | f
322 | 3250 | 40 | 0.0023843328 | f
323 | 3260 | 40 | 0.0026655248 | f
324 | 3270 | 40 | 0.0294494238 | nadir
325 | 3280 | 40 | 0.0006953623 | f
326 | 3290 | 40 | 0.0013252612 | f
327 | 3300 | 40 | 0.0195822454 | nadir
328 | 3310 | 40 | 0.0003916449 | f
329 | 3320 | 40 | 0.0006023140 | f
330 | 3330 | 40 | 0.0020546850 | f
331 | 3340 | 40 | 0.0024518355 | f
332 | 3350 | 40 | 0.0024518355 | f
333 | 3360 | 40 | 0.0025046855 | f
334 | 3370 | 40 | 0.000000E+00 | f
335 | 3380 | 40 | 0.0001305483 | f
336 | 3390 | 40 | 0.0006370559 | f
337 | 3400 | 40 | 0.0006696929 | f
338 | 3410 | 40 | 0.0012837859 | f
339 | 3420 | 40 | 0.0012837859 | f
340 | 3430 | 40 | 0.0019472322 | f
341 | 3440 | 40 | 0.0015090471 | f
342 | 3450 | 40 | 0.0020679654 | f
343 | 3460 | 40 | 0.0020679654 | f
344 | 3470 | 40 | 0.0020679654 | f
345 | 3480 | 40 | 0.0020679654 | f
346 | 3490 | 40 | 0.0027648331 | f
347 | 3500 | 40 | 0.0003918788 | f
348 | 3510 | 40 | 0.0005877013 | f
349 | 3520 | 40 | 0.0021564904 | f
350 | 3530 | 40 | 0.0022240327 | f
351 | 3540 | 40 | 0.0023581580 | f
352 | 3550 | 40 | 0.0025222415 | f
353 | 3560 | 40 | 0.0005761838 | f
354 | 3570 | 40 | 0.0006963174 | f
355 | 3580 | 40 | 0.0007289545 | f
356 | 3590 | 40 | 0.0012496476 | f
357 | 3600 | 40 | 0.0016626099 | f
358 | 3610 | 40 | 0.0023533310 | f
359 | 3620 | 40 | 0.0029674441 | f
360 | 3630 | 40 | 0.0005587512 | f
361 | 3640 | 40 | 0.0007877108 | f
362 | 3650 | 40 | 0.0011186688 | f
363 | 3660 | 40 | 0.0017789181 | f
364 | 3670 | 40 | 0.0014675719 | f
365 | 3680 | 40 | 0.0014675719 | f
366 | 3690 | 40 | 0.0015326879 | f
367 | 3700 | 40 | 0.0015467182 | f
368 | 3710 | 40 | 0.0029939346 | f
369 | 3720 | 40 | 0.000000E+00 | f
370 | 3730 | 40 | 0.0003984434 | f
371 | 3740 | 40 | 0.0003984434 | f
372 | 3750 | 40 | 0.0007467945 | f
373 | 3760 | 40 | 0.0007467945 | f
374 | 3770 | 40 | 0.0012096102 | f
375 | 3780 | 39 | 0.0010704998 | f
376 | 3790 | 40 | 0.0013808807 | f
377 | 3800 | 40 | 0.0015440661 | f
378 | 3810 | 40 | 0.0052493438 | nadir
379 | 3820 | 40 | 0.0005287858 | f
380 | 3830 | 40 | 0.0006080088 | f
381 | 3840 | 40 | 0.0013263928 | f
382 | 3850 | 40 | 0.0019220794 | f
383 | 3860 | 40 | 0.0035137257 | f
384 | 3870 | 40 | 0.0006596362 | f
385 | 3880 | 40 | 0.0011201877 | f
386 | 3890 | 40 | 0.0017911113 | f
387 | 3900 | 40 | 0.0028699619 | f
388 | 3910 | 40 | 0.0001320010 | f
389 | 3920 | 40 | 0.0001320010 | f
390 | 3930 | 40 | 0.0007049307 | f
391 | 3940 | 40 | 0.0008130369 | f
392 | 3950 | 40 | 0.0011812768 | f
393 | 3960 | 40 | 0.0017062112 | f
394 | 3970 | 40 | 0.0028019653 | f
395 | 3980 | 40 | 0.000000E+00 | f
396 | 3990 | 40 | 0.0007039265 | f
397 | 4000 | 40 | 0.0017228424 | f
398 | 4010 | 40 | 0.0010576912 | f
399 | 4020 | 40 | 0.0017279249 | f
400 | 4030 | 40 | 0.0014325100 | f
401 | 4040 | 40 | 0.0132978723 | nadir
402 | 4050 | 40 | 0.0005388727 | f
403 | 4060 | 40 | 0.0005388727 | f
404 | 4070 | 40 | 0.0005388727 | f
405 | 4080 | 40 | 0.0005388727 | f
406 | 4090 | 40 | 0.0006051826 | f
407 | 4100 | 40 | 0.0012694907 | f
408 | 4110 | 40 | 0.0013380749 | f
409 | 4120 | 40 | 0.0014045643 | f
410 | 4130 | 40 | 0.0015368568 | f
411 | 4140 | 40 | 0.0017372602 | f
412 | 4150 | 40 | 0.0017716094 | f
413 | 4160 | 40 | 0.0020244258 | f
414 | 4170 | 40 | 0.0020244258 | f
415 | 4180 | 40 | 0.0025346502 | f
416 | 4190 | 40 | 0.0002749029 | f
417 | 4200 | 40 | 0.0006911890 | f
418 | 4210 | 40 | 0.0011233719 | f
419 | 4220 | 40 | 0.0023360260 | f
420 | 4230 | 40 | 0.0358974359 | nadir
421 | 4240 | 40 | 0.0003295624 | f
422 | 4250 | 40 | 0.0008765219 | f
423 | 4260 | 40 | 0.0010690143 | f
424 | 4270 | 40 | 0.0014274376 | f
425 | 4280 | 40 | 0.0012667092 | f
426 | 4290 | 40 | 0.0012988526 | f
427 | 4300 | 40 | 0.0014588601 | f
428 | 4310 | 40 | 0.0019621734 | f
429 | 4320 | 40 | 0.0022179224 | f
430 | 4330 | 40 | 0.0036260987 | f
431 | 4340 | 40 | 0.0008272374 | f
432 | 4350 | 40 | 0.0009181376 | f
433 | 4360 | 40 | 0.0013207711 | f
434 | 4370 | 40 | 0.0014118218 | f
435 | 4380 | 40 | 0.0019949265 | f
436 | 4390 | 40 | 0.0025706941 | nadir
437 | 4400 | 40 | 0.0003213368 | f
438 | 4410 | 40 | 0.0012408930 | f
439 | 4420 | 40 | 0.0012408930 | f
440 | 4430 | 40 | 0.0016987220 | f
441 | 4440 | 40 | 0.0018820154 | f
442 | 4450 | 40 | 0.0022762726 | f
443 | 4460 | 40 | 0.0024452801 | f
444 | 4470 | 40 | 0.0025315388 | f
445 | 4480 | 40 | 0.0002272853 | f
446 | 4490 | 40 | 0.0002272853 | f
447 | 4500 | 40 | 0.0130208333 | nadir
448 | 4510 | 40 | 0.0026109661 | nadir
449 | 4520 | 40 | 0.0001086953 | f
450 | 4530 | 40 | 0.0001861233 | f
451 | 4540 | 40 | 0.0026041667 | ideal
452 | 4550 | 40 | 0.0006660227 | f
453 | 4560 | 40 | 0.0006660227 | f
454 | 4570 | 40 | 0.0009351252 | f
455 | 4580 | 40 | 0.0014426020 | f
456 | 4590 | 40 | 0.0023086647 | f
457 | 4600 | 40 | 0.0027668319 | f
458 | 4610 | 40 | 0.000000E+00 | f
459 | 4620 | 40 | 0.0003291968 | f
460 | 4630 | 40 | 0.0009028519 | f
461 | 4640 | 40 | 0.0009028519 | f
462 | 4650 | 40 | 0.0020120693 | f
463 | 4660 | 40 | 0.0022724860 | f
464 | 4670 | 40 | 0.0022724860 | f
465 | 4680 | 40 | 0.0022724860 | f
466 | 4690 | 40 | 0.0022724860 | f
467 | 4700 | 40 | 0.0027638221 | f
468 | 4710 | 40 | 0.0007529497 | f
469 | 4720 | 40 | 0.0010340796 | f
470 | 4730 | 40 | 0.0007529497 | f
471 | 4740 | 40 | 0.0014813761 | f
472 | 4750 | 40 | 0.0014813761 | f
473 | 4760 | 40 | 0.0010841543 | f
474 | 4770 | 40 | 0.0019571075 | f
475 | 4780 | 40 | 0.0027855286 | f
476 | 4790 | 40 | 0.0003109896 | f
477 | 4800 | 40 | 0.0003109896 | f
478 | 4810 | 39 | 0.0004191283 | f
479 | 4820 | 40 | 0.0008789277 | f
480 | 4830 | 40 | 0.0016183083 | f
481 | 4840 | 40 | 0.0025484101 | f
482 | 4850 | 40 | 0.0005774054 | f
483 | 4860 | 40 | 0.0005774054 | f
484 | 4870 | 40 | 0.0005774054 | f
485 | 4880 | 40 | 0.0011422189 | f
486 | 4890 | 40 | 0.0009495605 | f
487 | 4900 | 40 | 0.0011967288 | f
488 | 4910 | 40 | 0.0017799638 | f
489 | 4920 | 40 | 0.0017799638 | f
490 | 4930 | 40 | 0.0017799638 | f
491 | 4940 | 40 | 0.0025929238 | f
492 | 4950 | 40 | 0.0005906467 | f
493 | 4960 | 40 | 0.0011357101 | f
494 | 4970 | 40 | 0.0010145262 | f
495 | 4980 | 40 | 0.0016240721 | f
496 | 4990 | 40 | 0.0118577075 | nadir
497 | 5000 | 40 | 0.0003998442 | f
498 | 5010 | 40 | 0.0008086652 | f
499 | 5020 | 40 | 0.0003552518 | f
500 | 5030 | 40 | 0.0008712438 | f
501 | 5040 | 40 | 0.0008712438 | f
502 | 5050 | 40 | 0.0009049681 | f
503 | 5060 | 40 | 0.0007749427 | f
504 | 5070 | 40 | 0.0014483733 | f
505 | 5080 | 40 | 0.0017134870 | f
506 | 5090 | 40 | 0.0018217499 | f
507 | 5100 | 40 | 0.0020526206 | f
508 | 5110 | 40 | 0.0020108220 | f
509 | 5120 | 40 | 0.0021112168 | f
510 | 5130 | 40 | 0.0022366022 | f
511 | 5140 | 40 | 0.0023139025 | f
512 | 5150 | 40 | 0.0027502877 | f
513 | 5160 | 40 | 0.000000E+00 | f
514 | 5170 | 40 | 0.000000E+00 | f
515 | 5180 | 40 | 0.0003917540 | f
516 | 5190 | 40 | 0.0004661616 | f
517 | 5200 | 40 | 0.0004661616 | f
518 | 5210 | 40 | 0.0004661616 | f
519 | 5220 | 40 | 0.0017424352 | f
520 | 5230 | 40 | 0.0017847819 | f
521 | 5240 | 40 | 0.0039421813 | nadir
522 | 5250 | 40 | 0.0000328515 | f
523 | 5260 | 40 | 0.0155239327 | nadir
524 | 5270 | 40 | 0.0697745570 | nadir
525 | 5280 | 40 | 0.0004844961 | f
526 | 5290 | 40 | 0.0013573895 | f
527 | 5300 | 40 | 0.0019779560 | f
528 | 5310 | 40 | 0.0021924472 | f
529 | 5320 | 40 | 0.0018699370 | f
530 | 5330 | 40 | 0.0021990883 | f
531 | 5340 | 40 | 0.0022632491 | f
532 | 5350 | 40 | 0.0026332054 | f
533 | 5360 | 40 | 0.0000777319 | f
534 | 5370 | 40 | 0.0002070659 | f
535 | 5380 | 40 | 0.0007410718 | f
536 | 5390 | 40 | 0.0009136110 | f
537 | 5400 | 40 | 0.0010452131 | f
538 | 5410 | 40 | 0.0064184852 | nadir
539 | 5420 | 40 | 0.0000320924 | f
540 | 5430 | 40 | 0.0005807968 | f
541 | 5440 | 40 | 0.0006128892 | f
542 | 5450 | 40 | 0.0007687609 | f
543 | 5460 | 40 | 0.0007687609 | f
544 | 5470 | 40 | 0.0014458968 | f
545 | 5480 | 40 | 0.0020556549 | f
546 | 5490 | 40 | 0.0020556549 | f
547 | 5500 | 40 | 0.0027492027 | f
548 | 5510 | 40 | 0.0008778370 | f
549 | 5520 | 40 | 0.0013425576 | f
550 | 5530 | 40 | 0.0021010319 | f
551 | 5540 | 40 | 0.0022510155 | f
552 | 5550 | 40 | 0.0032924347 | f
553 | 5560 | 40 | 0.0002952307 | f
554 | 5570 | 40 | 0.0002952307 | f
555 | 5580 | 40 | 0.0006622420 | f
556 | 5590 | 40 | 0.0009748128 | f
557 | 5600 | 40 | 0.0012007543 | f
558 | 5610 | 40 | 0.0016032002 | f
559 | 5620 | 40 | 0.0017957594 | f
560 | 5630 | 40 | 0.0017957594 | f
561 | 5640 | 40 | 0.0024191583 | f
562 | 5650 | 40 | 0.0034241219 | f
563 | 5660 | 40 | 0.0008776863 | f
564 | 5670 | 40 | 0.0009741447 | f
565 | 5680 | 40 | 0.0009741447 | f
566 | 5690 | 40 | 0.0009741447 | f
567 | 5700 | 40 | 0.0009103183 | f
568 | 5710 | 40 | 0.0005894021 | f
569 | 5720 | 40 | 0.0005894021 | f
570 | 5730 | 40 | 0.0015934542 | f
571 | 5740 | 40 | 0.0021956759 | f
572 | 5750 | 40 | 0.0023429453 | f
573 | 5760 | 40 | 0.0023750378 | f
574 | 5770 | 40 | 0.0038659794 | nadir
575 | 5780 | 40 | 0.0010894080 | f
576 | 5790 | 40 | 0.0010894080 | f
577 | 5800 | 40 | 0.0012182740 | f
578 | 5810 | 40 | 0.0013848635 | f
579 | 5820 | 40 | 0.0017392450 | f
580 | 5830 | 40 | 0.0017792785 | f
581 | 5840 | 40 | 0.0013247713 | f
582 | 5850 | 40 | 0.0019067823 | f
583 | 5860 | 40 | 0.0021635889 | f
584 | 5870 | 40 | 0.0021635889 | f
585 | 5880 | 40 | 0.0021958049 | f
586 | 5890 | 40 | 0.0022924544 | f
587 | 5900 | 40 | 0.0029045678 | f
588 | 5910 | 40 | 0.000000E+00 | f
589 | 5920 | 40 | 0.0005808693 | f
590 | 5930 | 40 | 0.0005808693 | f
591 | 5940 | 40 | 0.0014466078 | f
592 | 5950 | 40 | 0.0012638851 | f
593 | 5960 | 40 | 0.0014249676 | f
594 | 5970 | 40 | 0.0014866026 | f
595 | 5980 | 40 | 0.0025839793 | nadir
596 | 5990 | 40 | 0.0022741501 | f
597 | 6000 | 40 | 0.0024986202 | f
598 | 6010 | 40 | 0.0065019506 | nadir
599 | 6020 | 40 | 0.0004296817 | f
600 | 6030 | 40 | 0.0012424255 | f
601 | 6040 | 40 | 0.0012424255 | f
602 | 6050 | 40 | 0.0105124836 | nadir
603 | 6060 | 40 | 0.000000E+00 | f
604 | 6070 | 40 | 0.000000E+00 | f
605 | 6080 | 40 | 0.000000E+00 | f
606 | 6090 | 40 | 0.0008054596 | f
607 | 6100 | 40 | 0.0026212320 | nadir
608 | 6110 | 40 | 0.0000327654 | f
609 | 6120 | 40 | 0.0002293578 | f
610 | 6130 | 40 | 0.0005169403 | f
611 | 6140 | 40 | 0.0006301837 | f
612 | 6150 | 40 | 0.0009545802 | f
613 | 6160 | 40 | 0.0010241480 | f
614 | 6170 | 40 | 0.0014601408 | f
615 | 6180 | 39 | 0.0016770549 | f
616 | 6190 | 40 | 0.0027691554 | f
617 | 6200 | 40 | 0.0065876153 | nadir
618 | 6210 | 40 | 0.000000E+00 | f
619 | 6220 | 40 | 0.0004069557 | f
620 | 6230 | 39 | 0.0004041686 | f
621 | 6240 | 40 | 0.0007934843 | f
622 | 6250 | 40 | 0.0010198879 | f
623 | 6260 | 40 | 0.0209523910 | nadir
624 | 6270 | 40 | 0.000000E+00 | f
625 | 6280 | 40 | 0.000000E+00 | f
626 | 6290 | 40 | 0.0001647283 | f
627 | 6300 | 40 | 0.0001647283 | f
628 | 6310 | 40 | 0.0001647283 | f
629 | 6320 | 40 | 0.0003952948 | f
630 | 6330 | 40 | 0.0039682540 | nadir
631 | 6340 | 40 | 0.0000995243 | f
632 | 6350 | 40 | 0.0001458753 | f
633 | 6360 | 40 | 0.0001835214 | f
634 | 6370 | 40 | 0.0001835214 | f
635 | 6380 | 40 | 0.0011885969 | f
636 | 6390 | 40 | 0.0017746459 | f
637 | 6400 | 40 | 0.0019040867 | f
638 | 6410 | 40 | 0.0021076758 | f
639 | 6420 | 40 | 0.0024071786 | f
640 | 6430 | 40 | 0.0024071786 | f
641 | 6440 | 40 | 0.0026957258 | f
642 | 6450 | 40 | 0.0004786559 | f
643 | 6460 | 40 | 0.0006452265 | f
644 | 6470 | 40 | 0.0010420519 | f
645 | 6480 | 40 | 0.0011081895 | f
646 | 6490 | 40 | 0.0016704154 | f
647 | 6500 | 40 | 0.0020433164 | f
648 | 6510 | 40 | 0.0020433164 | f
649 | 6520 | 40 | 0.0023740315 | f
650 | 6530 | 40 | 0.0023740315 | f
651 | 6540 | 40 | 0.0023947231 | f
652 | 6550 | 40 | 0.0036849011 | f
653 | 6560 | 40 | 0.0009057749 | f
654 | 6570 | 40 | 0.0012364628 | f
655 | 6580 | 40 | 0.0013412844 | f
656 | 6590 | 40 | 0.0018499019 | f
657 | 6600 | 40 | 0.0029810706 | f
658 | 6610 | 40 | 0.000000E+00 | f
659 | 6620 | 40 | 0.0004981780 | f
660 | 6630 | 40 | 0.0004981780 | f
661 | 6640 | 40 | 0.0008223981 | f
662 | 6650 | 40 | 0.0011370631 | f
663 | 6660 | 40 | 0.0015075630 | f
664 | 6670 | 40 | 0.0015075630 | f
665 | 6680 | 40 | 0.0023484165 | f
666 | 6690 | 40 | 0.0023484165 | f
667 | 6700 | 40 | 0.0019128123 | f
668 | 6710 | 40 | 0.0019128123 | f
669 | 6720 | 40 | 0.0021587807 | f
670 | 6730 | 40 | 0.0021587807 | f
671 | 6740 | 40 | 0.0023079314 | f
672 | 6750 | 40 | 0.0027050857 | f
673 | 6760 | 40 | 0.0005859529 | f
674 | 6770 | 40 | 0.0008835720 | f
675 | 6780 | 40 | 0.0016564226 | f
676 | 6790 | 40 | 0.0016564226 | f
677 | 6800 | 40 | 0.0017225602 | f
678 | 6810 | 40 | 0.0024337934 | f
679 | 6820 | 40 | 0.0022291088 | f
680 | 6830 | 40 | 0.0021964076 | f
681 | 6840 | 40 | 0.0025342558 | f
682 | 6850 | 40 | 0.0001322751 | f
683 | 6860 | 40 | 0.0001322751 | f
684 | 6870 | 40 | 0.0004791436 | f
685 | 6880 | 40 | 0.0008180206 | f
686 | 6890 | 40 | 0.0011673172 | f
687 | 6900 | 40 | 0.0012003860 | f
688 | 6910 | 40 | 0.0012003860 | f
689 | 6920 | 40 | 0.0012003860 | f
690 | 6930 | 40 | 0.0012003860 | f
691 | 6940 | 40 | 0.0014970974 | f
692 | 6950 | 40 | 0.0014970974 | f
693 | 6960 | 40 | 0.0016968770 | f
694 | 6970 | 40 | 0.0021682072 | f
695 | 6980 | 40 | 0.0021682072 | f
696 | 6990 | 40 | 0.0024902083 | f
697 | 7000 | 40 | 0.0025515376 | f
698 | 7010 | 40 | 0.0001655629 | f
699 | 7020 | 40 | 0.0002709135 | f
700 | 7030 | 40 | 0.0006909251 | f
701 | 7040 | 40 | 0.0009728376 | f
702 | 7050 | 40 | 0.0012557975 | f
703 | 7060 | 40 | 0.0012187469 | f
704 | 7070 | 40 | 0.0014405764 | f
705 | 7080 | 40 | 0.0014405764 | f
706 | 7090 | 40 | 0.0014640029 | f
707 | 7100 | 40 | 0.0018697632 | f
708 | 7110 | 40 | 0.0025870124 | f
709 | 7120 | 40 | 0.0006005496 | f
710 | 7130 | 40 | 0.0006005496 | f
711 | 7140 | 40 | 0.0016238167 | f
712 | 7150 | 40 | 0.0016238167 | f
713 | 7160 | 40 | 0.0018941773 | f
714 | 7170 | 40 | 0.0025408499 | f
715 | 7180 | 40 | 0.0001216931 | f
716 | 7190 | 40 | 0.0012736812 | f
717 | 7200 | 40 | 0.0012736812 | f
718 | 7210 | 40 | 0.0012736812 | f
719 | 7220 | 40 | 0.0013399064 | f
720 | 7230 | 40 | 0.0017104288 | f
721 | 7240 | 40 | 0.0017104288 | f
722 | 7250 | 40 | 0.0017104288 | f
723 | 7260 | 40 | 0.0017104288 | f
724 | 7270 | 40 | 0.0093582888 | nadir
725 | 7280 | 40 | 0.0003905751 | f
726 | 7290 | 40 | 0.0003905751 | f
727 | 7300 | 40 | 0.0003905751 | f
728 | 7310 | 40 | 0.0003905751 | f
729 | 7320 | 40 | 0.0003905751 | f
730 | 7330 | 40 | 0.0009212038 | f
731 | 7340 | 40 | 0.0010676339 | f
732 | 7350 | 40 | 0.0012366513 | f
733 | 7360 | 40 | 0.0012366513 | f
734 | 7370 | 40 | 0.0012700738 | f
735 | 7380 | 40 | 0.0012700738 | f
736 | 7390 | 40 | 0.0015179930 | f
737 | 7400 | 40 | 0.0018823870 | f
738 | 7410 | 40 | 0.0020494993 | f
739 | 7420 | 40 | 0.0020494993 | f
740 | 7430 | 40 | 0.0021681672 | f
741 | 7440 | 40 | 0.0021681672 | f
742 | 7450 | 40 | 0.0027713748 | f
743 | 7460 | 40 | 0.0004344920 | f
744 | 7470 | 40 | 0.0004344920 | f
745 | 7480 | 40 | 0.0004679144 | f
746 | 7490 | 40 | 0.0005013369 | f
747 | 7500 | 40 | 0.0005013369 | f
748 | 7510 | 40 | 0.0005013369 | f
749 | 7520 | 40 | 0.0005347594 | f
750 | 7530 | 40 | 0.0005347594 | f
751 | 7540 | 40 | 0.0005347594 | f
752 | 7550 | 40 | 0.0006684492 | f
753 | 7560 | 40 | 0.0007018717 | f
754 | 7570 | 40 | 0.0009358297 | f
755 | 7580 | 40 | 0.0017713912 | f
756 | 7590 | 40 | 0.0025310500 | f
757 | 7600 | 40 | 0.0000417677 | f
758 | 7610 | 40 | 0.0053763441 | nadir
759 | 7620 | 40 | 0.000000E+00 | f
760 | 7630 | 40 | 0.000000E+00 | f
761 | 7640 | 40 | 0.0000672043 | f
762 | 7650 | 40 | 0.0000672043 | f
763 | 7660 | 40 | 0.0001680108 | f
764 | 7670 | 40 | 0.0005228280 | f
765 | 7680 | 40 | 0.0005228280 | f
766 | 7690 | 40 | 0.0005900323 | f
767 | 7700 | 40 | 0.0008008197 | f
768 | 7710 | 40 | 0.0008008197 | f
769 | 7720 | 40 | 0.0008680240 | f
#print(res.history[0].pop.get("F"))
Analysis of Results#
import plotly.express as px
fig = px.scatter(res.F, x=0, y=1, labels={'0': 'Efficiency', '1': 'Volume'},
title="In Efficiency vs Volume",
width=800, height=800)
fig.update_traces(marker=dict(size=8,
line=dict(width=1,
color='DarkSlateGrey')),
selector=dict(mode='markers'))
fig.show()
# Making a animation of evolution
import pandas as pd
import plotly.express as px
obj1 = []
obj2 = []
calls = []
for r in res.history:
objs = r.pop.get("F")
obj1.extend(objs[:, 0])
obj2.extend(objs[:, 1])
calls.extend([r.n_gen]*len(objs))
df = pd.DataFrame(data = {"InEfficiency": obj1, "Volume": obj2, "n_gen": calls})
obj_fig = px.scatter(df, x="InEfficiency", y="Volume",
animation_frame="n_gen", color="n_gen",
range_x=[0., 0.6], range_y=[0. , 400.],
hover_data = df.columns,
width = 800, height = 800)
obj_fig.update(layout_coloraxis_showscale=False)
obj_fig.layout.updatemenus[0].buttons[0].args[1]["frame"]["duration"] = 10
obj_fig.update_layout(transition = {'duration': 0.001})
obj_fig.show()
# Computing hyper volume
## Tracking hypervolume
from pymoo.indicators.hv import HV
import plotly.graph_objs as go
ref_point = np.array([0.8, 350.0])
hv_list = []
for gen in res.history:
hv = HV(ref_point)(gen.pop.get("F"))
hv_list.append(hv)
fig = go.Figure()
fig.add_trace(go.Scatter(x=np.arange(1, len(hv_list)+1), y=hv_list, mode='lines+markers'))
fig.update_layout(title='Hypervolume Tracking', xaxis_title='Generation', yaxis_title='Hypervolume')
fig.show()
print ("Hypervolume at the end of the optimization: ", hv_list[-1])
Hypervolume at the end of the optimization: 179.7466732835622
# A parallel plot of design adn objective space
import plotly.graph_objects as go
X = res.pop.get("X")
F = res.pop.get("F")
fig = go.Figure(data=go.Parcoords(
line=dict(color=F[:, 0],
colorscale='Viridis',
showscale=True,
reversescale=True),
dimensions = [dict(range = [xl[index], xu[index]], label = f'X{index}', values=X[:, index])
for index in range(len(xl))
]
))
fig.update_layout(title = "Parallel plot of design space and objective space",
template='plotly_dark')
fig.show()
fig = go.Figure(data=go.Parcoords(
line=dict(color=F[:, 1],
colorscale='Viridis',
showscale=True,
reversescale=True),
dimensions = [dict(range = [xl[index], xu[index]], label = f'X{index}', values=X[:, index])
for index in range(len(xl))
]
))
fig.update_layout(title = "Parallel plot of design space and objective space",
template='plotly_dark')
fig.show()
Exercise#
Can you plot the hypervolume metric to see if the convergence is reached
Can you implement a termination criterion
Plot the optimal configuration of the tracker corresponding to that point
Do analysis of convergence
Visualize the point with a radar or petal diagram, following https://pymoo.org/visualization/index.html