\n",
"

\n",
" 100.00% [30000/30000 00:23<00:00 Average Loss = 122.57]\n",
"

\n",
" "
]
},
"metadata": {}
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"CPU times: user 24.8 s, sys: 217 ms, total: 25 s\n",
"Wall time: 26 s\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"> During the optimization process with ADVI, the parameters (mean and standard deviation) of these distributions are updated iteratively. However, these parameters themselves (the variational parameters) are what are saved during optimization, not the weights directly. Each update effectively shifts and shapes the assumed posterior distributions of the weights. In practice, these updates occur at each iteration of the optimizer, and the state of these parameters at convergence represents the learned posterior.\n",
"\n",
"\n",
"\n",
"> The mean and variance for each weight's distribution are adjusted to minimize the divergence between the true posterior and the variational approximation.\n",
"\n"
],
"metadata": {
"id": "fh-E4ydI7CGF"
}
},
{
"cell_type": "markdown",
"source": [
"Plotting the objective function (ELBO) we can see that the optimization iteratively improves the fit.\n",
"\n"
],
"metadata": {
"id": "t0U4qdaRrGX6"
}
},
{
"cell_type": "code",
"source": [
"plt.plot(approx.hist, alpha=0.3)\n",
"plt.ylabel(\"ELBO\")\n",
"plt.xlabel(\"iteration\");"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 508
},
"id": "rV_c7mm3qkMO",
"outputId": "e508242d-9737-402e-e0fb-5d848aec2cfa"
},
"execution_count": 94,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"