4-3,nn.functional和nn.Module.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "90eb7ba2",
   "metadata": {},
   "source": [
    "# 4-3,nn.functional 和 nn.Module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "a7a6e4e6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.__version__=2.0.1\n",
      "torchkeras.__version__=3.9.3\n"
     ]
    }
   ],
   "source": [
    "import torch \n",
    "import torchkeras\n",
    "print(\"torch.__version__=\"+torch.__version__) \n",
    "print(\"torchkeras.__version__=\"+torchkeras.__version__) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "35e59099",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "c286e87a",
   "metadata": {},
   "source": [
    "### 一，nn.functional 和 nn.Module"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4ffd405",
   "metadata": {},
   "source": [
    "前面我们介绍了Pytorch的张量的结构操作和数学运算中的一些常用API。\n",
    "\n",
    "利用这些张量的API我们可以构建出神经网络相关的组件(如激活函数，模型层，损失函数)。\n",
    "\n",
    "Pytorch和神经网络相关的功能组件大多都封装在 torch.nn模块下。\n",
    "\n",
    "这些功能组件的绝大部分既有函数形式实现，也有类形式实现。\n",
    "\n",
    "其中nn.functional(一般引入后改名为F)有各种功能组件的函数实现。例如：\n",
    "\n",
    "(激活函数)\n",
    "* F.relu \n",
    "* F.sigmoid\n",
    "* F.tanh\n",
    "* F.softmax\n",
    "\n",
    "(模型层)\n",
    "* F.linear\n",
    "* F.conv2d\n",
    "* F.max_pool2d\n",
    "* F.dropout2d\n",
    "* F.embedding\n",
    "\n",
    "(损失函数)\n",
    "* F.binary_cross_entropy\n",
    "* F.mse_loss\n",
    "* F.cross_entropy\n",
    "\n",
    "为了便于对参数进行管理，一般通过继承 nn.Module 转换成为类的实现形式，并直接封装在 nn 模块下。例如：\n",
    "\n",
    "(激活函数)\n",
    "* nn.ReLU\n",
    "* nn.Sigmoid\n",
    "* nn.Tanh\n",
    "* nn.Softmax\n",
    "\n",
    "(模型层)\n",
    "* nn.Linear\n",
    "* nn.Conv2d\n",
    "* nn.MaxPool2d\n",
    "* nn.Dropout2d\n",
    "* nn.Embedding\n",
    "\n",
    "(损失函数)\n",
    "* nn.BCELoss\n",
    "* nn.MSELoss\n",
    "* nn.CrossEntropyLoss\n",
    "\n",
    "实际上nn.Module除了可以管理其引用的各种参数，还可以管理其引用的子模块，功能十分强大。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fbd527b1",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a4ff03a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "00572afc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(0.)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.relu(torch.tensor(-1.0)) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "1eb413ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn.functional as F "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "7eafc178",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(0.)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "F.relu(torch.tensor(-1.0)) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e542059f",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9b1ce9ee",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "c5bdbec5",
   "metadata": {},
   "source": [
    "### 二，使用nn.Module来管理参数(配合nn.Parameter使用)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c83d9d4d",
   "metadata": {},
   "source": [
    "在Pytorch中，模型的参数是需要被优化器训练的，因此，通常要设置参数为 requires_grad = True 的张量。\n",
    "\n",
    "同时，在一个模型中，往往有许多的参数，要手动管理这些参数并不是一件容易的事情。\n",
    "\n",
    "Pytorch一般将参数用nn.Parameter来表示，并且用nn.Module来管理其结构下的所有参数。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "45884d27",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch \n",
    "from torch import nn \n",
    "import torch.nn.functional  as F\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "7526ff94",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.1829, 0.0693],\n",
       "        [0.0767, 1.2441]], requires_grad=True)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.randn(2,2,requires_grad = True)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7cc7d2a8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parameter containing:\n",
      "tensor([[-0.8092, -0.8830],\n",
      "        [ 1.6357, -0.1740]], requires_grad=True)\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "# nn.Parameter 具有 requires_grad = True 属性\n",
    "w = nn.Parameter(torch.randn(2,2))\n",
    "print(w)\n",
    "print(w.requires_grad)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "a65226f4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ParameterList(\n",
      "    (0): Parameter containing: [torch.float32 of size 8x1]\n",
      "    (1): Parameter containing: [torch.float32 of size 8x2]\n",
      ")\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "# nn.ParameterList 可以将多个nn.Parameter组成一个列表\n",
    "params_list = nn.ParameterList([nn.Parameter(torch.rand(8,i)) for i in range(1,3)])\n",
    "print(params_list)\n",
    "print(params_list[0].requires_grad)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d4c83e34",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ParameterDict(\n",
      "    (a): Parameter containing: [torch.FloatTensor of size 2x2]\n",
      "    (b): Parameter containing: [torch.FloatTensor of size 2]\n",
      ")\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "# nn.ParameterDict 可以将多个nn.Parameter组成一个字典\n",
    "params_dict = nn.ParameterDict({\"a\":nn.Parameter(torch.rand(2,2)),\n",
    "                               \"b\":nn.Parameter(torch.zeros(2))})\n",
    "print(params_dict)\n",
    "print(params_dict[\"a\"].requires_grad)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "13194251",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('w', Parameter containing:\n",
      "tensor([[-1.2390,  0.3316],\n",
      "        [-0.4232, -0.0090]], requires_grad=True)) \n",
      "\n",
      "('params_list.0', Parameter containing:\n",
      "tensor([[0.8785],\n",
      "        [0.6456],\n",
      "        [0.4697],\n",
      "        [0.8962],\n",
      "        [0.1122],\n",
      "        [0.4837],\n",
      "        [0.8089],\n",
      "        [0.0515]], requires_grad=True)) \n",
      "\n",
      "('params_list.1', Parameter containing:\n",
      "tensor([[0.7440, 0.5626],\n",
      "        [0.2430, 0.0113],\n",
      "        [0.5884, 0.0815],\n",
      "        [0.7125, 0.4120],\n",
      "        [0.7275, 0.1608],\n",
      "        [0.4658, 0.0085],\n",
      "        [0.8578, 0.7290],\n",
      "        [0.0327, 0.2239]], requires_grad=True)) \n",
      "\n",
      "('params_dict.a', Parameter containing:\n",
      "tensor([[0.6698, 0.5646],\n",
      "        [0.2482, 0.8258]], requires_grad=True)) \n",
      "\n",
      "('params_dict.b', Parameter containing:\n",
      "tensor([0., 0.], requires_grad=True)) \n",
      "\n",
      "number of Parameters = 5\n"
     ]
    }
   ],
   "source": [
    "# 可以用Module将它们管理起来\n",
    "# module.parameters()返回一个生成器，包括其结构下的所有parameters\n",
    "\n",
    "module = nn.Module()\n",
    "module.w = nn.Parameter(torch.randn(2,2))\n",
    "module.params_list = nn.ParameterList([nn.Parameter(torch.rand(8,i)) for i in range(1,3)])\n",
    "module.params_dict = nn.ParameterDict({\"a\":nn.Parameter(torch.rand(2,2)),\n",
    "                               \"b\":nn.Parameter(torch.zeros(2))})\n",
    "\n",
    "num_param = 0\n",
    "for param in module.named_parameters():\n",
    "    print(param,\"\\n\")\n",
    "    num_param = num_param + 1\n",
    "print(\"number of Parameters =\",num_param)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "49edd89d",
   "metadata": {},
   "outputs": [],
   "source": [
    "#实践当中，一般通过继承nn.Module来构建模块类，并将所有含有需要学习的参数的部分放在构造函数中。\n",
    "\n",
    "#以下范例为Pytorch中nn.Linear的源码的简化版本\n",
    "#可以看到它将需要学习的参数放在了__init__构造函数中，并在forward中调用F.linear函数来实现计算逻辑。\n",
    "\n",
    "class Linear(nn.Module):\n",
    "    __constants__ = ['in_features', 'out_features']\n",
    "\n",
    "    def __init__(self, in_features, out_features, bias=True):\n",
    "        super(Linear, self).__init__()\n",
    "        self.in_features = in_features\n",
    "        self.out_features = out_features\n",
    "        self.weight = nn.Parameter(torch.Tensor(out_features, in_features))\n",
    "        if bias:\n",
    "            self.bias = nn.Parameter(torch.Tensor(out_features))\n",
    "        else:\n",
    "            self.register_parameter('bias', None)\n",
    "\n",
    "    def forward(self, input):\n",
    "        return F.linear(input, self.weight, self.bias)\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "819cdf27",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "0e86616d",
   "metadata": {},
   "source": [
    "### 三，使用nn.Module来管理子模块"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9e11be4",
   "metadata": {},
   "source": [
    "一般情况下，我们都很少直接使用 nn.Parameter来定义参数构建模型，而是通过一些拼装一些常用的模型层来构造模型。\n",
    "\n",
    "这些模型层也是继承自nn.Module的对象,本身也包括参数，属于我们要定义的模块的子模块。\n",
    "\n",
    "nn.Module提供了一些方法可以管理这些子模块。\n",
    "\n",
    "* children() 方法: 返回生成器，包括模块下的所有子模块。\n",
    "\n",
    "* named_children()方法：返回一个生成器，包括模块下的所有子模块，以及它们的名字。\n",
    "\n",
    "* modules()方法：返回一个生成器，包括模块下的所有各个层级的模块，包括模块本身。\n",
    "\n",
    "* named_modules()方法：返回一个生成器，包括模块下的所有各个层级的模块以及它们的名字，包括模块本身。\n",
    "\n",
    "其中chidren()方法和named_children()方法较多使用。\n",
    "\n",
    "modules()方法和named_modules()方法较少使用，其功能可以通过多个named_children()的嵌套使用实现。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "46fcd26b",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Net(nn.Module):\n",
    "    \n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        \n",
    "        self.embedding = nn.Embedding(num_embeddings = 10000,embedding_dim = 3,padding_idx = 1)\n",
    "        self.conv = nn.Sequential()\n",
    "        self.conv.add_module(\"conv_1\",nn.Conv1d(in_channels = 3,out_channels = 16,kernel_size = 5))\n",
    "        self.conv.add_module(\"pool_1\",nn.MaxPool1d(kernel_size = 2))\n",
    "        self.conv.add_module(\"relu_1\",nn.ReLU())\n",
    "        self.conv.add_module(\"conv_2\",nn.Conv1d(in_channels = 16,out_channels = 128,kernel_size = 2))\n",
    "        self.conv.add_module(\"pool_2\",nn.MaxPool1d(kernel_size = 2))\n",
    "        self.conv.add_module(\"relu_2\",nn.ReLU())\n",
    "        \n",
    "        self.dense = nn.Sequential()\n",
    "        self.dense.add_module(\"flatten\",nn.Flatten())\n",
    "        self.dense.add_module(\"linear\",nn.Linear(6144,1))\n",
    "        \n",
    "    def forward(self,x):\n",
    "        x = self.embedding(x).transpose(1,2)\n",
    "        x = self.conv(x)\n",
    "        y = self.dense(x)\n",
    "        return y\n",
    "    \n",
    "net = Net()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "9c1845cb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Embedding(10000, 3, padding_idx=1) \n",
      "\n",
      "Sequential(\n",
      "  (conv_1): Conv1d(3, 16, kernel_size=(5,), stride=(1,))\n",
      "  (pool_1): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (relu_1): ReLU()\n",
      "  (conv_2): Conv1d(16, 128, kernel_size=(2,), stride=(1,))\n",
      "  (pool_2): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (relu_2): ReLU()\n",
      ") \n",
      "\n",
      "Sequential(\n",
      "  (flatten): Flatten(start_dim=1, end_dim=-1)\n",
      "  (linear): Linear(in_features=6144, out_features=1, bias=True)\n",
      ") \n",
      "\n",
      "child number 3\n"
     ]
    }
   ],
   "source": [
    "i = 0\n",
    "for child in net.children():\n",
    "    i+=1\n",
    "    print(child,\"\\n\")\n",
    "print(\"child number\",i)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "b1084d51",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "embedding : Embedding(10000, 3, padding_idx=1) \n",
      "\n",
      "conv : Sequential(\n",
      "  (conv_1): Conv1d(3, 16, kernel_size=(5,), stride=(1,))\n",
      "  (pool_1): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (relu_1): ReLU()\n",
      "  (conv_2): Conv1d(16, 128, kernel_size=(2,), stride=(1,))\n",
      "  (pool_2): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (relu_2): ReLU()\n",
      ") \n",
      "\n",
      "dense : Sequential(\n",
      "  (flatten): Flatten(start_dim=1, end_dim=-1)\n",
      "  (linear): Linear(in_features=6144, out_features=1, bias=True)\n",
      ") \n",
      "\n",
      "child number 3\n"
     ]
    }
   ],
   "source": [
    "i = 0\n",
    "for name,child in net.named_children():\n",
    "    i+=1\n",
    "    print(name,\":\",child,\"\\n\")\n",
    "print(\"child number\",i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "030281d9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Net(\n",
      "  (embedding): Embedding(10000, 3, padding_idx=1)\n",
      "  (conv): Sequential(\n",
      "    (conv_1): Conv1d(3, 16, kernel_size=(5,), stride=(1,))\n",
      "    (pool_1): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "    (relu_1): ReLU()\n",
      "    (conv_2): Conv1d(16, 128, kernel_size=(2,), stride=(1,))\n",
      "    (pool_2): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "    (relu_2): ReLU()\n",
      "  )\n",
      "  (dense): Sequential(\n",
      "    (flatten): Flatten(start_dim=1, end_dim=-1)\n",
      "    (linear): Linear(in_features=6144, out_features=1, bias=True)\n",
      "  )\n",
      ")\n",
      "Embedding(10000, 3, padding_idx=1)\n",
      "Sequential(\n",
      "  (conv_1): Conv1d(3, 16, kernel_size=(5,), stride=(1,))\n",
      "  (pool_1): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (relu_1): ReLU()\n",
      "  (conv_2): Conv1d(16, 128, kernel_size=(2,), stride=(1,))\n",
      "  (pool_2): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (relu_2): ReLU()\n",
      ")\n",
      "Conv1d(3, 16, kernel_size=(5,), stride=(1,))\n",
      "MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "ReLU()\n",
      "Conv1d(16, 128, kernel_size=(2,), stride=(1,))\n",
      "MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "ReLU()\n",
      "Sequential(\n",
      "  (flatten): Flatten(start_dim=1, end_dim=-1)\n",
      "  (linear): Linear(in_features=6144, out_features=1, bias=True)\n",
      ")\n",
      "Flatten(start_dim=1, end_dim=-1)\n",
      "Linear(in_features=6144, out_features=1, bias=True)\n",
      "module number: 12\n"
     ]
    }
   ],
   "source": [
    "i = 0\n",
    "for module in net.modules():\n",
    "    i+=1\n",
    "    print(module)\n",
    "print(\"module number:\",i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "335e7de5",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "93cfd180",
   "metadata": {},
   "source": [
    "下面我们通过named_children方法找到embedding层，并将其参数设置为不可训练(相当于冻结embedding层)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "653ba472",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'embedding': Embedding(10000, 3, padding_idx=1), 'conv': Sequential(\n",
      "  (conv_1): Conv1d(3, 16, kernel_size=(5,), stride=(1,))\n",
      "  (pool_1): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (relu_1): ReLU()\n",
      "  (conv_2): Conv1d(16, 128, kernel_size=(2,), stride=(1,))\n",
      "  (pool_2): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (relu_2): ReLU()\n",
      "), 'dense': Sequential(\n",
      "  (flatten): Flatten(start_dim=1, end_dim=-1)\n",
      "  (linear): Linear(in_features=6144, out_features=1, bias=True)\n",
      ")}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Embedding(10000, 3, padding_idx=1)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "children_dict = {name:module for name,module in net.named_children()}\n",
    "\n",
    "print(children_dict)\n",
    "embedding = children_dict[\"embedding\"]\n",
    "embedding.requires_grad_(False) #冻结其参数\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "3ccedfd7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "False\n",
      "30000\n"
     ]
    }
   ],
   "source": [
    "#可以看到其第一层的参数已经不可以被训练了。\n",
    "for param in embedding.parameters():\n",
    "    print(param.requires_grad)\n",
    "    print(param.numel())\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "0ca0a92f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------------------------------------\n",
      "Layer (type)                            Output Shape              Param #\n",
      "==========================================================================\n",
      "Embedding-1                             [-1, 200, 3]               30,000\n",
      "Conv1d-2                               [-1, 16, 196]                  256\n",
      "MaxPool1d-3                             [-1, 16, 98]                    0\n",
      "ReLU-4                                  [-1, 16, 98]                    0\n",
      "Conv1d-5                               [-1, 128, 97]                4,224\n",
      "MaxPool1d-6                            [-1, 128, 48]                    0\n",
      "ReLU-7                                 [-1, 128, 48]                    0\n",
      "Flatten-8                                 [-1, 6144]                    0\n",
      "Linear-9                                     [-1, 1]                6,145\n",
      "==========================================================================\n",
      "Total params: 40,625\n",
      "Trainable params: 10,625\n",
      "Non-trainable params: 30,000\n",
      "--------------------------------------------------------------------------\n",
      "Input size (MB): 0.000763\n",
      "Forward/backward pass size (MB): 0.287788\n",
      "Params size (MB): 0.154972\n",
      "Estimated Total Size (MB): 0.443523\n",
      "--------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "from torchkeras import summary\n",
    "summary(net,input_shape = (200,),input_dtype = torch.LongTensor);\n",
    "# 不可训练参数数量增加"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "48959123",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "9fe95a70",
   "metadata": {},
   "source": [
    "**如果本书对你有所帮助，想鼓励一下作者，记得给本项目加一颗星星star⭐️，并分享给你的朋友们喔😊!** \n",
    "\n",
    "如果对本书内容理解上有需要进一步和作者交流的地方，欢迎在公众号\"算法美食屋\"下留言。作者时间和精力有限，会酌情予以回复。\n",
    "\n",
    "也可以在公众号后台回复关键字：**加群**，加入读者交流群和大家讨论。\n",
    "\n",
    "![算法美食屋logo.png](https://tva1.sinaimg.cn/large/e6c9d24egy1h41m2zugguj20k00b9q46.jpg)"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}