A-7,30分钟吃掉wandb可视化自动调参.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r_rLI6AelSI7",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 30分钟吃掉wandb可视化自动调参\n",
    "\n",
    "wandb.sweep: **低代码，可视化，分布式** 自动调参工具。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r_rLI6AelSI7",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "使用wandb 的 sweep 进行超参调优，具有以下优点。\n",
    "\n",
    "(1)低代码：只需配置一个sweep.yaml配置文件，或者定义一个配置dict，几乎不用编写调参相关代码。\n",
    "\n",
    "(2)可视化：在wandb网页中可以实时监控调参过程中每次尝试，并可视化地分析调参任务的目标值分布，超参重要性等。\n",
    "\n",
    "(3)分布式：sweep采用类似master-workers的controller-agents架构，controller在wandb的服务器机器上运行，agents在用户机器上运行，controller和agents之间通过互联网进行通信。同时启动多个agents即可轻松实现分布式超参搜索。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](../data/sweep_arch.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r_rLI6AelSI7",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "使用 wandb 的sweeps 调参的缺点：\n",
    "\n",
    "需要联网：由于wandb的controller位于wandb的服务器机器上，wandb日志也需要联网上传，在没有互联网的环境下无法正常使用wandb 进行模型跟踪 以及 wandb sweep 可视化调参。\n",
    "\n",
    "B站视频演示：https://www.bilibili.com/video/BV1rM411Y7nZ\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 〇，使用Sweep的3步骤"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r_rLI6AelSI7"
   },
   "source": [
    "1. 配置 sweep_config\n",
    "\n",
    "```\n",
    "配置调优算法，调优目标，需要优化的超参数列表 等等。\n",
    "```\n",
    "\n",
    "2. 初始化 sweep controller: \n",
    "\n",
    "```python\n",
    "sweep_id = wandb.sweep(sweep_config,project)\n",
    "```\n",
    "\n",
    "3. 启动 sweep agents: \n",
    "\n",
    "```python\n",
    "wandb.agent(sweep_id, function=train)\n",
    "```\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "aVtp7wtulSI_",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Failed to detect the name of this notebook, you can set it manually with the WANDB_NOTEBOOK_NAME environment variable to enable code saving.\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mlyhue1991\u001b[0m. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os,PIL \n",
    "import numpy as np\n",
    "from torch.utils.data import DataLoader, Dataset\n",
    "import torch \n",
    "from torch import nn \n",
    "import torchvision \n",
    "from torchvision import transforms\n",
    "import datetime\n",
    "import wandb \n",
    "\n",
    "wandb.login()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from argparse import Namespace\n",
    "\n",
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
    "\n",
    "#初始化参数配置\n",
    "config = Namespace(\n",
    "    project_name = 'wandb_demo',\n",
    "    \n",
    "    batch_size = 512,\n",
    "    \n",
    "    hidden_layer_width = 64,\n",
    "    dropout_p = 0.1,\n",
    "    \n",
    "    lr = 1e-4,\n",
    "    optim_type = 'Adam',\n",
    "    \n",
    "    epochs = 15,\n",
    "    ckpt_path = 'checkpoint.pt'\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "P0eVUQcElSI_",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 一. 配置 Sweep config\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "详细配置文档可以参考：https://docs.wandb.ai/guides/sweeps/define-sweep-configuration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fPlztw_TlSJA",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 1，选择一个调优算法 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qJOdaEnnlSJA",
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "Sweep支持如下3种调优算法:\n",
    "\n",
    "(1)网格搜索： grid. 遍历所有可能得超参组合，只在超参空间不大的时候使用，否则会非常慢。\n",
    "\n",
    "(2)随机搜索：random. 每个超参数都选择一个随机值，非常有效，一般情况下建议使用。\n",
    "\n",
    "(3)贝叶斯搜索： bayes. 创建一个概率模型估计不同超参数组合的效果，采样有更高概率提升优化目标的超参数组合。对连续型的超参数特别有效，但扩展到非常高维度的超参数时效果不好。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "0KLI5Y-JlSJA",
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "sweep_config = {\n",
    "    'method': 'random'\n",
    "    }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Eihe5EZUlSJB",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 2，定义调优目标"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "设置优化指标，以及优化方向。\n",
    "\n",
    "sweep agents 通过 wandb.log 的形式向 sweep controller 传递优化目标的值。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "v172MtfSlSJB",
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "metric = {\n",
    "    'name': 'val_acc',\n",
    "    'goal': 'maximize'   \n",
    "    }\n",
    "sweep_config['metric'] = metric\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kcLQFtu6lSJC",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 3，定义超参空间"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GQZOjd-8lSJC",
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "超参空间可以分成 固定型，离散型和连续型。\n",
    "\n",
    "* 固定型：指定 value\n",
    "* 离散型：指定 values，列出全部候选取值。\n",
    "* 连续性：需要指定 分布类型 distribution, 和范围 min, max。用于 random 或者 bayes采样。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "code_folding": [
     3,
     9
    ],
    "id": "AP02lswglSJC",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "sweep_config['parameters'] = {}\n",
    "\n",
    "# 固定不变的超参\n",
    "sweep_config['parameters'].update({\n",
    "    'project_name':{'value':'wandb_demo'},\n",
    "    'epochs': {'value': 10},\n",
    "    'ckpt_path': {'value':'checkpoint.pt'}})\n",
    "\n",
    "# 离散型分布超参\n",
    "sweep_config['parameters'].update({\n",
    "    'optim_type': {\n",
    "        'values': ['Adam', 'SGD','AdamW']\n",
    "        },\n",
    "    'hidden_layer_width': {\n",
    "        'values': [16,32,48,64,80,96,112,128]\n",
    "        }\n",
    "    })\n",
    "\n",
    "# 连续型分布超参\n",
    "sweep_config['parameters'].update({\n",
    "    \n",
    "    'lr': {\n",
    "        'distribution': 'log_uniform_values',\n",
    "        'min': 1e-6,\n",
    "        'max': 0.1\n",
    "      },\n",
    "    \n",
    "    'batch_size': {\n",
    "        'distribution': 'q_uniform',\n",
    "        'q': 8,\n",
    "        'min': 32,\n",
    "        'max': 256,\n",
    "      },\n",
    "    \n",
    "    'dropout_p': {\n",
    "        'distribution': 'uniform',\n",
    "        'min': 0,\n",
    "        'max': 0.6,\n",
    "      }\n",
    "})\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9IQwzn4HlSJD",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 4，定义剪枝策略 (可选)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "可以定义剪枝策略，提前终止那些没有希望的任务。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "sweep_config['early_terminate'] = {\n",
    "    'type':'hyperband',\n",
    "    'min_iter':3,\n",
    "    'eta':2,\n",
    "    's':3\n",
    "} #在step=3, 6, 12 时考虑是否剪枝\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "0G6J-wLzlSJD",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'early_terminate': {'eta': 2, 'min_iter': 3, 's': 3, 'type': 'hyperband'},\n",
      " 'method': 'random',\n",
      " 'metric': {'goal': 'maximize', 'name': 'val_acc'},\n",
      " 'parameters': {'batch_size': {'distribution': 'q_uniform',\n",
      "                               'max': 256,\n",
      "                               'min': 32,\n",
      "                               'q': 8},\n",
      "                'ckpt_path': {'value': 'checkpoint.pt'},\n",
      "                'dropout_p': {'distribution': 'uniform', 'max': 0.6, 'min': 0},\n",
      "                'epochs': {'value': 10},\n",
      "                'hidden_layer_width': {'values': [16,\n",
      "                                                  32,\n",
      "                                                  48,\n",
      "                                                  64,\n",
      "                                                  80,\n",
      "                                                  96,\n",
      "                                                  112,\n",
      "                                                  128]},\n",
      "                'lr': {'distribution': 'log_uniform_values',\n",
      "                       'max': 0.1,\n",
      "                       'min': 1e-06},\n",
      "                'optim_type': {'values': ['Adam', 'SGD', 'AdamW']},\n",
      "                'project_name': {'value': 'wandb_demo'}}}\n"
     ]
    }
   ],
   "source": [
    "from pprint import pprint\n",
    "pprint(sweep_config)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3ejRJtGGlSJD",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 二. 初始化 sweep controller"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "nYXAU39_lSJE",
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Create sweep with ID: 53wjom6j\n",
      "Sweep URL: https://wandb.ai/lyhue1991/wandb_demo/sweeps/53wjom6j\n"
     ]
    }
   ],
   "source": [
    "sweep_id = wandb.sweep(sweep_config, project=config.project_name)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IeA8ycwblSJE",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 三， 启动 Sweep agent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "我们需要把模型训练相关的全部代码整理成一个 train函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "code_folding": [],
    "id": "Xdve8EkMlSJE",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def create_dataloaders(config):\n",
    "    transform = transforms.Compose([transforms.ToTensor()])\n",
    "    ds_train = torchvision.datasets.MNIST(root=\"./mnist/\",train=True,download=True,transform=transform)\n",
    "    ds_val = torchvision.datasets.MNIST(root=\"./mnist/\",train=False,download=True,transform=transform)\n",
    "\n",
    "    ds_train_sub = torch.utils.data.Subset(ds_train, indices=range(0, len(ds_train), 5))\n",
    "    dl_train =  torch.utils.data.DataLoader(ds_train_sub, batch_size=config.batch_size, shuffle=True,\n",
    "                                            num_workers=2,drop_last=True)\n",
    "    dl_val =  torch.utils.data.DataLoader(ds_val, batch_size=config.batch_size, shuffle=False, \n",
    "                                          num_workers=2,drop_last=True)\n",
    "    return dl_train,dl_val"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "code_folding": [],
    "id": "Xdve8EkMlSJE",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def create_net(config):\n",
    "    net = nn.Sequential()\n",
    "    net.add_module(\"conv1\",nn.Conv2d(in_channels=1,out_channels=config.hidden_layer_width,kernel_size = 3))\n",
    "    net.add_module(\"pool1\",nn.MaxPool2d(kernel_size = 2,stride = 2)) \n",
    "    net.add_module(\"conv2\",nn.Conv2d(in_channels=config.hidden_layer_width,\n",
    "                                     out_channels=config.hidden_layer_width,kernel_size = 5))\n",
    "    net.add_module(\"pool2\",nn.MaxPool2d(kernel_size = 2,stride = 2))\n",
    "    net.add_module(\"dropout\",nn.Dropout2d(p = config.dropout_p))\n",
    "    net.add_module(\"adaptive_pool\",nn.AdaptiveMaxPool2d((1,1)))\n",
    "    net.add_module(\"flatten\",nn.Flatten())\n",
    "    net.add_module(\"linear1\",nn.Linear(config.hidden_layer_width,config.hidden_layer_width))\n",
    "    net.add_module(\"relu\",nn.ReLU())\n",
    "    net.add_module(\"linear2\",nn.Linear(config.hidden_layer_width,10))\n",
    "    return net "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "def train_epoch(model,dl_train,optimizer):\n",
    "    model.train()\n",
    "    for step, batch in enumerate(dl_train):\n",
    "        features,labels = batch\n",
    "        features,labels = features.to(device),labels.to(device)\n",
    "\n",
    "        preds = model(features)\n",
    "        loss = nn.CrossEntropyLoss()(preds,labels)\n",
    "        loss.backward()\n",
    "\n",
    "        optimizer.step()\n",
    "        optimizer.zero_grad()\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "def eval_epoch(model,dl_val):\n",
    "    model.eval()\n",
    "    accurate = 0\n",
    "    num_elems = 0\n",
    "    for batch in dl_val:\n",
    "        features,labels = batch\n",
    "        features,labels = features.to(device),labels.to(device)\n",
    "        with torch.no_grad():\n",
    "            preds = model(features)\n",
    "        predictions = preds.argmax(dim=-1)\n",
    "        accurate_preds =  (predictions==labels)\n",
    "        num_elems += accurate_preds.shape[0]\n",
    "        accurate += accurate_preds.long().sum()\n",
    "\n",
    "    val_acc = accurate.item() / num_elems\n",
    "    return val_acc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "code_folding": [],
    "id": "Xdve8EkMlSJE",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def train(config = config):\n",
    "    dl_train, dl_val = create_dataloaders(config)\n",
    "    model = create_net(config); \n",
    "    optimizer = torch.optim.__dict__[config.optim_type](params=model.parameters(), lr=config.lr)\n",
    "    #======================================================================\n",
    "    nowtime = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')\n",
    "    wandb.init(project=config.project_name, config = config.__dict__, name = nowtime, save_code=True)\n",
    "    model.run_id = wandb.run.id\n",
    "    #======================================================================\n",
    "    model.best_metric = -1.0\n",
    "    for epoch in range(1,config.epochs+1):\n",
    "        model = train_epoch(model,dl_train,optimizer)\n",
    "        val_acc = eval_epoch(model,dl_val)\n",
    "        if val_acc>model.best_metric:\n",
    "            model.best_metric = val_acc\n",
    "            torch.save(model.state_dict(),config.ckpt_path)   \n",
    "        nowtime = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')\n",
    "        print(f\"epoch【{epoch}】@{nowtime} --> val_acc= {100 * val_acc:.2f}%\")\n",
    "        #======================================================================\n",
    "        wandb.log({'epoch':epoch, 'val_acc': val_acc, 'best_val_acc':model.best_metric})\n",
    "        #======================================================================        \n",
    "    #======================================================================\n",
    "    wandb.finish()\n",
    "    #======================================================================\n",
    "    return model   \n",
    "\n",
    "#model = train(config) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "soYh-29XlSJF",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "一切准备妥当，点火🔥🔥。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "xdru6994lSJF",
    "scrolled": true,
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mwandb\u001b[0m: Agent Starting Run: jh1kqwm6 with config:\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: \tbatch_size: 168\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: \tckpt_path: checkpoint.pt\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: \tdropout_p: 0.136477226608858\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: \tepochs: 10\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: \thidden_layer_width: 16\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: \tlr: 1.1027452455943685e-06\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: \toptim_type: AdamW\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: \tproject_name: wandb_demo\n",
      "Failed to detect the name of this notebook, you can set it manually with the WANDB_NOTEBOOK_NAME environment variable to enable code saving.\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[33mWARNING\u001b[0m Ignored wandb.init() arg project when running a sweep.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "wandb version 0.13.10 is available!  To upgrade, please run:\n",
       " $ pip install wandb --upgrade"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Tracking run with wandb version 0.13.9"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Run data is saved locally in <code>/Users/liangyun2/CodeFiles/machine_learning_AK47/3_Tune_Tools/wandb/run-20230211_201921-jh1kqwm6</code>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Syncing run <strong><a href=\"https://wandb.ai/lyhue1991/wandb_demo/runs/jh1kqwm6\" target=\"_blank\">2023-02-11 20:19:19</a></strong> to <a href=\"https://wandb.ai/lyhue1991/wandb_demo\" target=\"_blank\">Weights & Biases</a> (<a href=\"https://wandb.me/run\" target=\"_blank\">docs</a>)<br/>Sweep page: <a href=\"https://wandb.ai/lyhue1991/wandb_demo/sweeps/53wjom6j\" target=\"_blank\">https://wandb.ai/lyhue1991/wandb_demo/sweeps/53wjom6j</a>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       " View project at <a href=\"https://wandb.ai/lyhue1991/wandb_demo\" target=\"_blank\">https://wandb.ai/lyhue1991/wandb_demo</a>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       " View sweep at <a href=\"https://wandb.ai/lyhue1991/wandb_demo/sweeps/53wjom6j\" target=\"_blank\">https://wandb.ai/lyhue1991/wandb_demo/sweeps/53wjom6j</a>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       " View run at <a href=\"https://wandb.ai/lyhue1991/wandb_demo/runs/jh1kqwm6\" target=\"_blank\">https://wandb.ai/lyhue1991/wandb_demo/runs/jh1kqwm6</a>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch【1】@2023-02-11 20:19:39 --> val_acc= 37.29%\n",
      "epoch【2】@2023-02-11 20:19:47 --> val_acc= 44.34%\n",
      "epoch【3】@2023-02-11 20:19:56 --> val_acc= 51.94%\n",
      "epoch【4】@2023-02-11 20:20:05 --> val_acc= 59.55%\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "Waiting for W&B process to finish... <strong style=\"color:red\">(failed 1).</strong> Press Control-C to abort syncing."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7c17bf1477344b46b32945c447066faf",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Label(value='4.379 MB of 4.394 MB uploaded (0.000 MB deduped)\\r'), FloatProgress(value=0.996552…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<style>\n",
       "    table.wandb td:nth-child(1) { padding: 0 10px; text-align: left ; width: auto;} td:nth-child(2) {text-align: left ; width: 100%}\n",
       "    .wandb-row { display: flex; flex-direction: row; flex-wrap: wrap; justify-content: flex-start; width: 100% }\n",
       "    .wandb-col { display: flex; flex-direction: column; flex-basis: 100%; flex: 1; padding: 10px; }\n",
       "    </style>\n",
       "<div class=\"wandb-row\"><div class=\"wandb-col\"><h3>Run history:</h3><br/><table class=\"wandb\"><tr><td>best_val_acc</td><td>▁▃▆█</td></tr><tr><td>epoch</td><td>▁▃▆█</td></tr><tr><td>val_acc</td><td>▁▃▆█</td></tr></table><br/></div><div class=\"wandb-col\"><h3>Run summary:</h3><br/><table class=\"wandb\"><tr><td>best_val_acc</td><td>0.5955</td></tr><tr><td>epoch</td><td>4</td></tr><tr><td>val_acc</td><td>0.5955</td></tr></table><br/></div></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       " View run <strong style=\"color:#cdcd00\">2023-02-11 20:19:19</strong> at: <a href=\"https://wandb.ai/lyhue1991/wandb_demo/runs/jh1kqwm6\" target=\"_blank\">https://wandb.ai/lyhue1991/wandb_demo/runs/jh1kqwm6</a><br/>Synced 7 W&B file(s), 0 media file(s), 0 artifact file(s) and 1 other file(s)"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Find logs at: <code>./wandb/run-20230211_201921-jh1kqwm6/logs</code>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 该agent 随机搜索 尝试5次\n",
    "wandb.agent(sweep_id, train, count=5)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 四，调参可视化和跟踪"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vZPE_3H6lSJF",
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "### 1，平行坐标系图\n",
    "\n",
    "可以直观展示哪些超参数组合更加容易获取更好的结果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vZPE_3H6lSJF",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](https://assets.website-files.com/5ac6b7f2924c652fd013a891/5e190366778ad831455f9af2_s_194708415DEC35F74A7691FF6810D3B14703D1EFE1672ED29000BA98171242A5_1578695138341_image.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KDYjDbAklSJF",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 2，超参数重要性图\n",
    "\n",
    "可以显示超参数和优化目标最终取值的重要性，和相关性方向。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KDYjDbAklSJF",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](https://assets.website-files.com/5ac6b7f2924c652fd013a891/5e190367778ad820b35f9af5_s_194708415DEC35F74A7691FF6810D3B14703D1EFE1672ED29000BA98171242A5_1578695757573_image.png)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}