Lora大模型微调

LoRA原理很简单，代码实现也不复杂。简单地说，在模型实现上，要在特定的模块上加一个旁路，这个旁路就是两个矩阵相乘的形式。这些特定的模块理论上可以是任何模块，目前作者实现的是在Linear, Embeding, Conv, Attention(只改其中的q和v)这些模块上加。

关键代码：

class LoRALayer():
    def __init__(
        self, 
        r: int, 
        lora_alpha: int, 
        lora_dropout: float,
        merge_weights: bool,
    ):
        self.r = r
        self.lora_alpha = lora_alpha
        # Optional dropout
        if lora_dropout > 0.:
            self.lora_dropout = nn.Dropout(p=lora_dropout)
        else:
            self.lora_dropout = lambda x: x
        # Mark the weight as unmerged
        self.merged = False
        self.merge_weights = merge_weights


class Linear(nn.Linear, LoRALayer):
    # LoRA implemented in a dense layer
    def __init__(
        self, 
        in_features: int, 
        out_features: int, 
        r: int = 0, 
        lora_alpha: int = 1, 
        lora_dropout: float = 0.,
        fan_in_fan_out: bool = False, # Set this to True if the layer to replace stores weight like (fan_in, fan_out)
        merge_weights: bool = True,
        **kwargs
    ):
        nn.Linear.__init__(self, in_features, out_features, **kwargs)
        LoRALayer.__init__(self, r=r, lora_alpha=lora_alpha, lora_dropout=lora_dropout,
                           merge_weights=merge_weights)

        self.fan_in_fan_out = fan_in_fan_out
        # Actual trainable parameters
        if r > 0:
            self.lora_A = nn.Parameter(self.weight.new_zeros((r, in_features)))
            self.lora_B = nn.Parameter(self.weight.new_zeros((out_features, r)))
            self.scaling = self.lora_alpha / self.r
            # Freezing the pre-trained weight matrix
            self.weight.requires_grad = False
        self.reset_parameters()
        if fan_in_fan_out:
            self.weight.data = self.weight.data.transpose(0, 1)

    def reset_parameters(self):
        nn.Linear.reset_parameters(self)
        if hasattr(self, 'lora_A'):
            # initialize A the same way as the default for nn.Linear and B to zero
            nn.init.kaiming_uniform_(self.lora_A, a=math.sqrt(5))
            nn.init.zeros_(self.lora_B)

    def train(self, mode: bool = True):
        def T(w):
            return w.transpose(0, 1) if self.fan_in_fan_out else w
        nn.Linear.train(self, mode)
        if mode:
            if self.merge_weights and self.merged:
                # Make sure that the weights are not merged
                if self.r > 0:
                    self.weight.data -= T(self.lora_B @ self.lora_A) * self.scaling
                self.merged = False
        else:
            if self.merge_weights and not self.merged:
                # Merge the weights and mark it
                if self.r > 0:
                    self.weight.data += T(self.lora_B @ self.lora_A) * self.scaling
                self.merged = True       

    def forward(self, x: torch.Tensor):
        def T(w):
            return w.transpose(0, 1) if self.fan_in_fan_out else w
        if self.r > 0 and not self.merged:
            result = F.linear(x, T(self.weight), bias=self.bias)            
            result += (self.lora_dropout(x) @ self.lora_A.transpose(0, 1) @ self.lora_B.transpose(0, 1)) * self.scaling
            return result
        else:
            return F.linear(x, T(self.weight), bias=self.bias)

在Linear层的实现上，多继承了一个LoRALayer, LoRALayer中就是设置了一些参数，最主要的就是上面的讲道的矩阵的秩r了，其他就是一些辅助参数，如控制训练和推理时主路参数和旁路参数是否合并等等。在Linear层中，多定义了A和B两个可训练的参数矩阵，然后在forward中把主路和旁路输出相加，基本上就是完全按照原理来的。LoRALayer 类

LoRALayer 是一个基础类，用于定义LoRA层的通用属性和行为。

__init__
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

   

  方法初始化类实例：

  - `r`: LoRA的秩，表示要引入的低秩结构的大小。
  - `lora_alpha`: 用于调整LoRA参数影响的缩放因子。
  - `lora_dropout`: Dropout概率，用于在训练过程中随机丢弃部分LoRA参数，防止过拟合。
  - `merge_weights`: 一个布尔值，用于控制是否将LoRA参数与原始权重合并。
  - `lora_dropout`: 如果大于0，则创建一个`nn.Dropout`层；否则，定义一个不改变输入的lambda函数。

### Linear 类

- `Linear` 类继承自PyTorch的`nn.Linear`类，并集成了`LoRALayer`的功能，实现了LoRA层。

  [python中的类class: 继承、覆盖、重写、重载、扩展、多态、封装_python class函数-CSDN博客](https://blog.csdn.net/eylier/article/details/128961571?ops_request_misc=%7B%22request%5Fid%22%3A%22B8F3F5FA-5712-4026-8DB6-DD482BCCC380%22%2C%22scm%22%3A%2220140713.130102334..%22%7D&request_id=B8F3F5FA-5712-4026-8DB6-DD482BCCC380&biz_id=0&utm_medium=distribute.pc_search_result.none-task-blog-2~all~top_positive~default-2-128961571-null-null.142^v100^pc_search_result_base8&utm_term=python类的继承&spm=1018.2226.3001.4187)

- ```
  __init__


 

方法初始化类实例：

- `in_features` 和 `out_features`: 分别表示输入和输出的特征数量。
- `r`, `lora_alpha`, `lora_dropout`, `merge_weights`: 与`LoRALayer`中的参数相同。
- `fan_in_fan_out`: 一个布尔值，用于控制权重矩阵的存储方式。
- `**kwargs`: 允许传递其他关键字参数给`nn.Linear`。
- 如果`r`大于0，初始化LoRA参数`lora_A`和`lora_B`，并将原始权重矩阵的梯度设置为False（即冻结预训练权重）。
- `reset_parameters` 方法用于初始化权重。