YOLOv5获得大中小目标的AP和AR指标（自制数据集）-Toy模板网

这篇具有很好参考价值的文章主要介绍了YOLOv5获得大中小目标的AP和AR指标（自制数据集）。希望对大家有所帮助。如果存在错误或未考虑完全的地方，请大家不吝赐教，您也可以点击"举报违法"按钮提交疑问。

前言

本文简要介绍YOLOv5如何调用pycocotools得到大中小目标的AP和AR指标，评价自制数据集。

代码版本-----YOLOv5_6.0版本。
数据集----Seaships7000数据集，共包含6类7000张船舶图片，其中测试集1400张。
模型-----自制模型。

一、运行示例

话不多说，运行示例：

(pytorch1.8) zmy@525:~/文档/A-YOLO$ python val.py
val: data=data/ship.yaml, weights=runs/train/exp28/weights/best.pt, batch_size=32, imgsz=640, conf_thres=0.001, iou_thres=0.6, task=test, device=, single_cls=False, augment=False, verbose=False, save_txt=False, save_hybrid=False, save_conf=False, save_json=True, project=runs/val, name=exp, exist_ok=False, half=False
YOLOv5 🚀 2021-10-12 torch 1.8.0+cu111 CUDA:0 (NVIDIA GeForce RTX 3090, 24268.3125MB)

Fusing layers... 
Model Summary: 540 layers, 4933647 parameters, 0 gradients
test: Scanning '/home/zmy/文档/A-YOLO/data/labels/test_fog.cache' images and lab
               Class     Images     Labels          P          R     mAP@.5 mAP@
                 all       1400       1837      0.844      0.543       0.66      0.449
         ore carrier       1400        417       0.94      0.448      0.624      0.392
        fishing boat       1400        428      0.785      0.613      0.678      0.456
      passenger ship       1400         94      0.628      0.628      0.681      0.451
  general cargo ship       1400        312      0.865      0.599      0.724      0.509
  bulk cargo carrier       1400        392      0.845      0.569      0.682      0.474
      container ship       1400        194          1      0.401      0.569      0.415
Speed: 0.1ms pre-process, 1.6ms inference, 1.0ms NMS per image at shape (32, 3, 640, 640)

Evaluating pycocotools mAP... saving runs/val/exp1/best_predictions.json...
loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
Loading and preparing results...
DONE (t=0.13s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=1.27s).
Accumulating evaluation results...
DONE (t=0.36s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.445
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.650
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.497
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.050
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.287
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.458
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.477
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.535
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.535
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.050
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.357
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.549
Results saved to runs/val/exp1

二、参考

主要参考了以下三个案例，并根据Seaships数据集特征修改了部分代码。

参考1：安装pycocotools
参考2：yolov5 调用cocotools 评价自己的模型和数据集
参考3：YOLO至COCO的格式转换器

三、方法

1.安装pycocotools库

pip install pycocotools

2.YOLOv5代码修改

只需修改val.py文件

   1.'--save-json' 添加 default=True parser.add_argument('--save-json', default=True, action='store_true', help='save a COCO-JSON results    file')
   
   2.'--task' 修改 default='test' parser.add_argument('--task', default='test', help='train, val, test, speed or study')
   
   3.注释下句    # opt.save_json |= opt.data.endswith('coco.yaml')
   
   4.为了生成的json文件是多行，方便自查格式
     改json.dump(jdict, f) 
     为json.dump(jdict, f, ensure_ascii=False, indent=1)

修改后终端输入python val.py，如下所示：
会提示我们Evaluating pycocotools mAP… saving runs/val/exp1/best_predictions.json…
并报错[Errno 2] No such file or directory: ‘…/coco/annotations/instances_val2017.json’
这说明此模型需要测试的json文件已经保存在runs/val/exp1/best_predictions.json
但标准的json文件在此路径../coco/annotations/instances_val2017.json没有找到。
接下来制作Seaships数据集的json文件

test: Scanning '/home/zmy/文档/A-YOLO/data/labels/test_fog.cache' images and lab
               Class     Images     Labels          P          R     mAP@.5 mAP@
                 all       1400       1837      0.844      0.543       0.66      0.449
         ore carrier       1400        417       0.94      0.448      0.624      0.392
        fishing boat       1400        428      0.785      0.613      0.678      0.456
      passenger ship       1400         94      0.628      0.628      0.681      0.451
  general cargo ship       1400        312      0.865      0.599      0.724      0.509
  bulk cargo carrier       1400        392      0.845      0.569      0.682      0.474
      container ship       1400        194          1      0.401      0.569      0.415
Speed: 0.1ms pre-process, 1.7ms inference, 1.3ms NMS per image at shape (32, 3, 640, 640)

Evaluating pycocotools mAP... saving runs/val/exp1/best_predictions.json...
loading annotations into memory...
pycocotools unable to run: [Errno 2] No such file or directory: '../coco/annotations/instances_val2017.json'

3.制作.json文件

根据参考3的README文档将YOLO标签的txt格式转换为json格式
只需修改main.py文件

1、根据Seaships数据集修改类别列表
classes = [
    "ore carrier",
    "fishing boat",
    "passenger ship",
    "general cargo ship",
    "bulk cargo carrier",
    "container ship",]
    
2、把image_id定义为文件名并去除尾缀
 for file_path in file_paths:
        # Check how many items have progressed
        print("\rProcessing " + str(image_id) + " ...", end='')
        # ---------------------image_id定义为文件名--------------------------
        image_id = int(file_path.stem)
        # ------------------------------------------------------------------
        # Build image annotation, known the image's width and height
        w, h = imagesize.get(str(file_path))
        image_annotation = create_image_annotation(
            file_path=file_path, width=w, height=h, image_id=image_id)
        images_annotations.append(image_annotation)
        label_file_name = f"{file_path.stem}.txt"

3、把标签从1开始改为标签从0开始
        for line1 in label_read_line:
            label_line = line1
            category_id = (
                # int(label_line.split()[0]) + 1) # you start with annotation id with '1'
                int(label_line.split()[0]) + 0) # you start with annotation id with '0'

最后将生成的train.json文件，标签改为从0开始，并改名为instances_val2017.json，然后放到根目录的coco/annotations/文件夹中，没有则需要自己创建。

4.运行程序

终端输入python val.py，即大功告成！

附录

需要测试的best_predictions.json示例：

[
 {
  "image_id": 4724,
  "category_id": 3,
  "bbox": [
   838.916,
   158.716,
   1081.084,
   332.775
  ],
  "score": 0.94571
 },
 {
  "image_id": 4724,
  "category_id": 1,
  "bbox": [
   623.036,
   369.717,
   210.212,
   30.21
  ],
  "score": 0.00897
 },
 {
  "image_id": 4724,
  "category_id": 4,
  "bbox": [
   838.773,
   159.783,
   1081.227,
   334.143
  ],
  "score": 0.00734
 }, 
 ...此处省略其余标注

制作的标准的instances_val2017.json示例：

{
    "images": [
        {
            "file_name": "000013.jpg",
            "height": 1080,
            "width": 1920,
            "id": 13}
         ...此处省略其余1399个图片文件
            ]，
    "categories": [
        {
            "supercategory": "Defect",
            "id": 0,
            "name": "ore carrier"
        },
        {
            "supercategory": "Defect",
            "id": 1,
            "name": "fishing boat"
        },
         ...此处省略其余4种类别
    ],
    "annotations": [
        {
            "id": 1,
            "image_id": 13,
            "bbox": [
                1640.0,
                544.0,
                280.0,
                30.0
            ],
            "area": 8400,
            "iscrowd": 0,
            "category_id": 0,
            "segmentation": []
        },
        ...此处省略其余标注