HandOut

fruit image classification model

object detection model

palyGround

youtube - GitHub - tputflite

optimize book

optimize - youtube

about mAP blog

Analysis Hyparameter

根据初学者而言，训练卷积神经模型可以很简单也可以很复杂，简单是只要把参数代进去就好，而困难的是该如何调整这些参数到合适的位置，我将会分享我的学习内容并教您该如何优化自己的模型。

1. Model: SSD with Mobilenet v2 FPN-lite
2. training playform: colab
3. Type: object detection model

以上是我使用的模型并训练模型的平台。

Batch size

什么是Batch size 呢？

假设你有一个模型需要训练1000张图片，那一次性训练完1000张效率太慢了。所以我们可以分成一组10张，同时训练100组，这样效率就会快很多，而这个组里面到底应该设置多少张才能提高训练速度，并且提高准确率呢。

1. 首先我们需要知道batch size 会影响什么，它会影响你的加速器的响应速度，以及准确率，那么影响的原因是什么呢？

由于加速器的内存不高，所以处理的图像不多：

例如一个小组内有32个人，但是这个汽车只能搭载16个人，所以剩下16个人不能上车，这就大大减少了效率，同时浪费了资源，原因是汽车是单程的。所以我们只需要设置一个小组只有16个人就好。太少人的话效率也太少，所以也可以设置尽量接近16个人每一组。

结论而言就是批量大小会直接影响验证集的性能。

2. 我们无法通过计算来解决模型到底需要多大的batch size来得到最快的训练速度和最好的准确率。

我们已经知道了这个batch size不能太大也不能太小，我们总得有个临界点吧，所以我们可以训练小一些的数据来得到这个结果。

下面是我的常量和变量

1. 总步数=1000步
2. 变量 batch size

在训练之前可以通过palyGround 这个网站来知道步数超过临界点会发生什么。

我的dataset一共300张图片，训练240张，30张来验证，30张来测试。

第一次测试 batch_size =16

Num_step = 1000

batch size = 16

Epochs = 1

average per step = 0.325 s

spend 7 min37 second

第二次测试 batch_size =17

batch size = 17

spend time 9min25second

第三次测试 batch_size=15

Batch_size = 15

Spend time = 7min 18 second

第四次测试 batch_size =18

batch_size = 18

Num_step = 1000

Spend time = 8min 1 second

第五次测试 batch_size=14

batch_size = 14

Num_step = 1000

Spend_time = 6min 46 s

第六次测试 batch_size =19

batch size = 19

Spend time =9min 20 second

Num_step = 1000

第七次测试 batch_size = 13

batch size = 13

spend time = 7min 10 sec

Num_step=1000

第八次测试 batch_size =20

batch_size = 20

Spend time = 9min30sec

第九次测试 batch_size = 12

batch_size = 12

spend time = 6min 14sec

第十次测试 batch_size=21

Batch_size = 21

Spend time = 9min 12 sec

第十一次测试 batch_size = 22

batch _size = 22

spend time = 9min 26sec

第十二次测试 batch_size = 23

Spend time = 10.17

Total loss = 0.322

Batch_size = 24

spend time = 10.27

Step = 1000

Batch_size = 25

spend time = 10.45


Batch_size = 26

spend time = 11.36

Batch_size = 27

spend time = 11.45

Conclusion

batch_size	Total_loss	spend time	mAP results(have been Converted TFlite)	learning_rate
12	0.41	6.14	25.73	0.075
13	0.38	7.1	48.28	0.075
14	0.355	6.46	42.57	0.075
15	0.34	7.18	49.62	0.075
16	0.36	7.37	48.97	0.075
17	0.348	9.25	51.34	0.075
18	0.323	8.01	51.16	0.075
19	0.35	9.2	40.76	0.075
20	0.34	9.26	33.26	0.075
21	0.325	9.12	42.33	0.075
22	0.3	9.26	52.9	0.075
23	0.322	10.17	46.67	0.075

Learning_rate

Learning_rate =0.08 , batch_size = 17

Learning_rate_base = 0.08

Warmup_learning_rate = 0.0266

batch_size = 17

Step_number = 10000

spend time = 59min 17second

Total_loss = 0.1537

Learning_rate = 0.08 batch_size = 22

‘Loss/localization_loss’: 0.021741068,

‘Loss/regularization_loss’: 0.09711991,

‘Loss/total_loss’: 0.148702,

‘learning_rate’: 0.07352352


Learning_rate = 0.07 batch_size = 22

Total_step = 10000

Total_loss = 0.152

Learning_rate = 0.06 ,batch_size = 22

1 hour 19min 26 sec

num_step = 10000

Total_loss = 0.168

Learning_rate = 0.07 batch_size = 17

1 hour 11 second

10000

Total_loss = 0.162

Learning_rate = 0.06 batch_size = 17

spend time = 57 min 16 second

Num_step = 10000

Total_loss = 0.178

Learning_rate = 0.05 batch_size = 22

Num_step = 10000

spend time = 52 min 42 sec

Total_losss = 0.1648

rebegin for learning_rate step = 7000 batch_size

Lr = 0.08

‘Loss/localization_loss’: 0.02736458,

‘Loss/regularization_loss’: 0.11061755,

‘Loss/total_loss’: 0.1774221,

‘learning_rate’: 0.07707667}

spend time. =56min 46sec

For quntization

Lr = 1.0

batch_size = 22

step = 7000

spend time = 54min40sec

‘Loss/localization_loss’: 0.036090124,

‘Loss/regularization_loss’: 0.096989244,

‘Loss/total_loss’: 0.19243404,

‘learning_rate’: 0.9634584}


for quntization

Lr = 1.2

Batch_size = 22

spend time = 57min.27 sec

Total_loss = 0.243

Lr = 0.14