diff? residual and norm_layer_for_blocks

[rixejzvdl649]

internvideo2

        residual = None
        for blk in self.blocks:
            if isinstance(x, tuple) and len(x) == 2:
                x, residual = x
            x = blk(x, residual=residual)
        if isinstance(x, tuple) and len(x) == 2:
            x, residual = x
            if residual is not None:
                x = x + residual

umt or videomaev2

        for idx, blk in enumerate(self.blocks):
            if self.use_checkpoint and idx < self.checkpoint_num:
                x = checkpoint.checkpoint(blk, x)
            else:
                x = blk(x)

[Andy1621]

Hi! Since the return logic is different when using DropoutAddRMSNorm, we change the forward fuction.