finish intro wandb

.gitignore

 .ipynb_checkpoints/
+wandb/

pre-exercises/intro_wandb.ipynb

@@ -35,56 +35,129 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
+   "outputs": [],
+   "source": [
+    "import wandb\n",
+    "wandb.login()"
+   ]
+  },
   {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "Failed to detect the name of this notebook, you can set it manually with the WANDB_NOTEBOOK_NAME environment variable to enable code saving.\n",
-      "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mihaide\u001b[0m. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n"
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If you want to log in in a terminal so you don't have to repeat this step every time you run an experiment, you can run the following command and enter your API key after setting up your python environment:\n",
+    "\n",
+    "`wandb login`"
    ]
   },
   {
-     "data": {
-      "text/plain": [
-       "True"
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Initialize a Run\n",
+    "\n",
+    "You can initialize a run in W&B, adding it to an existing project or creating a new one, as well as adding a config file to this run which tracks the hyperparameters of your experiment and will be stored in the database. You can do this by running the following cell:"
    ]
   },
-     "execution_count": 3,
+  {
+   "cell_type": "code",
+   "execution_count": null,
    "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
+   "outputs": [],
    "source": [
-    "import wandb\n",
-    "wandb.login()"
+    "run = wandb.init(project=\"intro_wandb\", config={\"learning_rate\": 0.01,\n",
+    "                                                \"date\": \"13.09.2023\"})"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "If you want to log in in a terminal so you don't have to repeat this step every time you run an experiment, you can run the following command and enter your API key after setting up your python environment:\n",
+    "You can add everything you want to your config. It often makes sense to have at least the parameters in there that make this run unique, so you can repeat experiments later on with the exact same setup.\n",
     "\n",
-    "`wandb login`"
+    "W&B initializes any run with a unique identifier, by default a random combination of an adjective and a noun. You can set the name yourself but be careful: If you use the same name twice, it will overwrite the older run."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Initialize a Run\n",
+    "## Logging"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We can now tie everything together and start monitoring our experiments. We will start with a simple example that mimicks the training of a machine learning model. \n",
     "\n",
-    "You can initialize a run in W&B, adding it to an existing project or creating a new one, as well as adding a config file to this run which tracks the hyperparameters of your experiment and will be stored in the database. You can do this by running the following cell:"
+    "To actually see what is happening, go to [wandb.ai](wandb.ai) and click on the project \"intro_wandb\". We will introduce a `sleep` command into the loop, so we can see the live updates (otherwise it would finish too quickly).\n",
+    "\n",
+    "We will also put the initialization of the run into the same cell. This will make sure that we start a new run each time we want to re-execute the \"training\"."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "import time\n",
+    "\n",
+    "epochs = 200\n",
+    "\n",
+    "run = wandb.init(project=\"intro_wandb\", config={\"learning_rate\": 0.01,\n",
+    "                                                \"date\": \"13.09.2023\",\n",
+    "                                                \"epochs\": epochs})\n",
+    "\n",
+    "for i in range(epochs):\n",
+    "    acc = 0.05 * i + np.random.randn()\n",
+    "    loss = 0.1*np.random.randn()\n",
+    "    print(f\"epoch {i}: accuracy = {acc}, loss = {loss}\")\n",
+    "    wandb.log({\"accuracy\": acc, \"loss\": loss})\n",
+    "    time.sleep(2)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### System Monitoring\n",
+    "\n",
+    "Additionally, W&B also tracks the system on which you run your training on. Go to your project workspace. You will see a dropdown menu called `System` below your charts with loss and accuracy. Here you can see variables like your Disk Utilization, Number of CPU Threads etc as well as the GPU variables if you train on GPU. This can be very helpful e.g. to see if you are utilizing the GPU enough or if you can improve on your training there."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## W&B in Use"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now take your model that you have trained in `intro_pytorch_model_optimization.ipynb` and implement tracking your training with W&B. You can log any variable that you want, it often makes sense to log training and testing loss and accuracy separately so you can monitor all charts at the same time. Try doing different trainings with different hyperparameters and store the changes in your config file."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Your code goes here"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "run = wandb.init(project=\"intro_wandb\", config={\"learning_rate\": 0.01})"
+    "You can also log your weights and biases with W&B (what a surprise there). Figure out how this works and implement it for your FashionMNIST model as well."
    ]
   }
  ],
@@ -104,7 +177,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.17"
+   "version": "3.9.13"
   },
   "orig_nbformat": 4
  },