第1节课笔记(5班)

[sanbuphy]

请在下方提交笔记链接

[Liu137]

大模型第一节课笔记

[Liu137]

https://github.com/Liu137?tab=repositories

[shengmouget]

大模型第一节课笔记：http://t.csdnimg.cn/cqCJt

[ZhaoHongSen0726]

笔记（csdn）：https://blog.csdn.net/m0_75085274/article/details/135372565?spm=1001.2014.3001.5501

[DreamBinary]

note.md

[yilisong007]

好奇问下，在按照《使用 Xtuner 跑起第一个微调实验》的时候，在开发机上执行下面的语句的时候，会提示“bash: xtuner: command not found”， 是因为前面安装source init_xtuner_env.sh运行环境出错了吗？
xtuner train ~/share/xtuner_config/internlm_7b/internlm_7b_qlora_alpaca_e3.py --deepspeed deepspeed_zero2

[ATang0729]

我的解决方案，先后运行：

1. pip install xtuner，解决你提到的“bash: xtuner: command not found”问题
2. pip install deepspeed，解决使用xtuner指令时deepspeed未安装的问题
3. 解决下一步安装mpi4py编译失败问题（以下三种方法择其一，我是采用方法三解决的）

1. conda install gcc_linux-64
2. pip3 install --upgrade pip
3. 先apt update，后apt-get install libopenmpi-dev

4. pip install mpi4py，解决微调internLM-7B时mpi4py未安装的问题
   完成上述操作后，执行xtuner train ~/share/xtuner_config/internlm_7b/internlm_7b_qlora_alpaca_e3.py --deepspeed deepspeed_zero2。执行结果：
   

[Milkve]

@ATang0729 大佬，这是正常的吗？为什么训练又重新开始了？

01/05 22:43:57 - mmengine - INFO - Epoch(train) [1][2770/2822]  lr: 1.5652e-04  eta: 7:17:10  time: 4.5459  data_time: 0.0027  memory: 9807  loss: 1.5672
01/05 22:44:42 - mmengine - INFO - Epoch(train) [1][2780/2822]  lr: 1.5624e-04  eta: 7:16:22  time: 4.5303  data_time: 0.0031  memory: 9807  loss: 1.6297
01/05 22:45:27 - mmengine - INFO - Epoch(train) [1][2790/2822]  lr: 1.5595e-04  eta: 7:15:35  time: 4.5615  data_time: 0.0030  memory: 9807  loss: 1.5719
01/05 22:46:13 - mmengine - INFO - Epoch(train) [1][2800/2822]  lr: 1.5566e-04  eta: 7:14:48  time: 4.5363  data_time: 0.0044  memory: 9807  loss: 1.5273
01/05 22:46:58 - mmengine - INFO - Epoch(train) [1][2810/2822]  lr: 1.5538e-04  eta: 7:14:00  time: 4.5385  data_time: 0.0028  memory: 9807  loss: 1.5930
01/05 22:47:44 - mmengine - INFO - Epoch(train) [1][2820/2822]  lr: 1.5509e-04  eta: 7:13:13  time: 4.5369  data_time: 0.0028  memory: 9807  loss: 1.5672
01/05 22:47:53 - mmengine - INFO - Exp name: internlm_7b_qlora_alpaca_e3_20240105_191004
01/05 22:47:53 - mmengine - INFO - Saving checkpoint at 1 epochs
[2024-01-05 22:47:53,340] [INFO] [logging.py:96:log_dist] [Rank 0] [Torch] Checkpoint epoch_1.pth is about to be saved!
/root/.conda/envs/xtuner/lib/python3.10/site-packages/torch/nn/modules/module.py:1879: UserWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
  warnings.warn(
[2024-01-05 22:47:53,404] [INFO] [logging.py:96:log_dist] [Rank 0] Saving model checkpoint: /root/work_dirs/internlm_7b_qlora_alpaca_e3/epoch_1.pth/mp_rank_00_model_states.pt
[2024-01-05 22:47:53,404] [INFO] [torch_checkpoint_engine.py:21:save] [Torch] Saving /root/work_dirs/internlm_7b_qlora_alpaca_e3/epoch_1.pth/mp_rank_00_model_states.pt...
[2024-01-05 22:47:53,664] [INFO] [torch_checkpoint_engine.py:23:save] [Torch] Saved /root/work_dirs/internlm_7b_qlora_alpaca_e3/epoch_1.pth/mp_rank_00_model_states.pt.
[2024-01-05 22:47:53,666] [INFO] [torch_checkpoint_engine.py:21:save] [Torch] Saving /root/work_dirs/internlm_7b_qlora_alpaca_e3/epoch_1.pth/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt...
[2024-01-05 22:47:54,967] [INFO] [torch_checkpoint_engine.py:23:save] [Torch] Saved /root/work_dirs/internlm_7b_qlora_alpaca_e3/epoch_1.pth/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt.
[2024-01-05 22:47:54,986] [INFO] [engine.py:3431:_save_zero_checkpoint] zero checkpoint saved /root/work_dirs/internlm_7b_qlora_alpaca_e3/epoch_1.pth/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt
[2024-01-05 22:47:54,986] [INFO] [torch_checkpoint_engine.py:33:commit] [Torch] Checkpoint epoch_1.pth is ready now!
/root/.conda/envs/xtuner/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
  warnings.warn(
01/05 22:48:40 - mmengine - INFO - Epoch(train) [2][  10/2822]  lr: 1.5474e-04  eta: 7:12:17  time: 4.5625  data_time: 0.0034  memory: 9807  loss: 1.5781
01/05 22:49:26 - mmengine - INFO - Epoch(train) [2][  20/2822]  lr: 1.5445e-04  eta: 7:11:30  time: 4.5558  data_time: 0.0030  memory: 9807  loss: 1.5539
01/05 22:50:11 - mmengine - INFO - Epoch(train) [2][  30/2822]  lr: 1.5416e-04  eta: 7:10:42  time: 4.5433  data_time: 0.0027  memory: 9807  loss: 1.5406
01/05 22:50:56 - mmengine - INFO - Epoch(train) [2][  40/2822]  lr: 1.5387e-04  eta: 7:09:54  time: 4.4951  data_time: 0.0032  memory: 9807  loss: 1.4773
01/05 22:51:41 - mmengine - INFO - Epoch(train) [2][  50/2822]  lr: 1.5358e-04  eta: 7:09:05  time: 4.4595  data_time: 0.0028  memory: 9807  loss: 1.5547
01/05 22:52:26 - mmengine - INFO - Epoch(train) [2][  60/2822]  lr: 1.5328e-04  eta: 7:08:17  time: 4.4913  data_time: 0.0047  memory: 9807  loss: 1.5703
01/05 22:53:11 - mmengine - INFO - Epoch(train) [2][  70/2822]  lr: 1.5299e-04  eta: 7:07:29  time: 4.5143  data_time: 0.0027  memory: 9807  loss: 1.5695

[ATang0729]

@Milkve 这个说明第一轮（epoch 1）训练好了，Checkpoint epoch_1.pth is about to be saved!也表明第一轮训练的模型参数已经保存好了，后面“重新”训练是开始第二轮（epoch 2）训练，是为了训练更好的结果。eta: 7:07:29表示的是所有轮次的训练结束所需的时间。你要是不想训练这么久，可以把模型配置文件（比如~/share/xtuner_config/internlm_7b/internlm_7b_qlora_alpaca_e3.py）里的max_epochs参数改为1

[Milkve]

明白了，谢谢大佬

[preordinary]

笔记：http://t.csdnimg.cn/P6rCS

[JayceNing]

第1节课笔记：https://zhuanlan.zhihu.com/p/675940087

[ATang0729]

笔记：https://github.com/ATang0729/InternLMCamp_Notes/blob/main/notes/class1-intro/note_class1.md

[Milkve]

大模型第一节课笔记：[学习笔记] 书生·浦语大模型全链路开源体系

[aJupyter]

打不开欸宝贝

[ironartisan]

https://ironartisan.github.io/2024/01/04/%E5%A4%A7%E6%A8%A1%E5%9E%8B%E5%85%A8%E9%93%BE%E8%B7%AF%E5%BC%80%E6%BA%90%E5%BC%80%E6%94%BE%E4%BD%93%E7%B3%BB/

[Farewell-CK]

https://github.com/8baby8/Farewell_Learn_puyu

[angellyao]

第一次课程笔记链接：
https://zhuanlan.zhihu.com/p/676164180

[Egber1t]

书生浦语Day1
https://editor.csdn.net/md/?articleId=135383638

[aJupyter]

这个链接是写博客的链接，建议给个完成的链接

[wangerlie]

https://github.com/wangerlie/LearnBlog/blob/main/OpenMMLabLLMcourse1.md

[aJupyter]

打不开欸

[sanbuphy]

宝子们都很棒~ 记得要把作业写到表格 我才能评分

[Beforcer]

https://zhuanlan.zhihu.com/p/676026221

[Lengxiaom0]

https://mp.csdn.net/mp_blog/creation/editor/135417626

[aJupyter]

这个链接是写博客的链接，建议给个完成的链接

[GitEasonXu]

第一节课笔记：https://github.com/GitEasonXu/InternLM/blob/main/homework/course_1/coursenote.md