| This week I worked to reimplement the current methodology for the BAX project to understand its current state and to work to debug some issues. While doing this, the reruns appear to be more efficient at sampling the configuration space. For this run, 30 initial samples were used and 100 iterations of optimizing the neural network were run. Figures 1-3 show the results of this simulation, namely the proposed best cost for each round, relative error in the sampled point, and the distribution of sampled points over the configuration space. Figure 4 shows the model prediction of cost at the final iteration of the training.
Figure 7 is the same plot as figure 4, just with the y axis scaled to be the same as in figure 3, for visual comparison.
I believe I have narrowed down the difference in the two implementations to the following design choice: This run, for each round of training, retrains the neural network from scratch rather than fine-tuning it on each individual sampled point. I found that when implementing a similar behavior in my implementation convergence was slow and samples did not cluster about the true optimum. Figure 5 and 6 show the results for implementing fine-tuning at the current hyper parameters. This could suggest the learning rate in the fine-tuning step is too low. Regardless it appears that retraining the network appears to work well and is more simple to implement.
Additionally I have made some changes in the Finesse model which seem to have sped up run time. I reduced the lock accuracy from 1e-12 to 1e-9, the mirror map resolution was reduced from 1001 to 601, and odd order modes were turned off. As a result the full training run was able to run in under 5 minutes. |