> For the complete documentation index, see [llms.txt](https://blessbingo.gitbook.io/garnet/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://blessbingo.gitbook.io/garnet/ji-qi-xue-xi/xun-lian-ji-qiao/ti-sheng-mo-xing-lu-bang-xing-xun-lian-xiao-shuai-mo-xing-xiao-guo-de-fang-fa-zong-jie.md).

# 提升模型鲁棒性, 训练效率, 模型效果的方法总结

## 预处理方法

* 数据增强(data augmentation), 对于文字和图像等不同类型的数据有不同的增强方式

## 模型结构

* **batch normalization**
* **Dropout**
  * 可以理解成训练了无数个共享同一套参数的不同结构的模型(结点被随机抛弃), 在预测时将所有的模型**隐式地**集成起来, 得到最终的结果
* **DropConnect**, 类似于Dropout, 随机断开的是连接而不是结点.
* **Stochastic Depth technique**, 与上面的也类似, 随机抛弃某些层, 在预测时也是**隐式地**集成了多种结构的模型.
* **Swapout**, 同时使用Dropout和Stochastic Depth.

## 训练技巧

* **Adam**降低到一定水平后, 切换成**SGD**
* SGD收敛后使用一个较大的**learning rate**跳出当前的局部极小, 再改用小的**learning rate**寻找附近的新的局部极小
* NLP问题中, **word embedding**使用预训练好的参数, 在模型刚开始时关闭训练, 减少干扰, 在模型收敛到一定程度后, 打开训练, 使之适合当前应用.

## 模型集成

* 同模型多次训练, 将预测结果进行集成, 比如通过投票或者求均值
* **Snapshot Ensemble**, 高效完成集成.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://blessbingo.gitbook.io/garnet/ji-qi-xue-xi/xun-lian-ji-qiao/ti-sheng-mo-xing-lu-bang-xing-xun-lian-xiao-shuai-mo-xing-xiao-guo-de-fang-fa-zong-jie.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.