《HALCON机器视觉与算法原理编程实践》第14章机器视觉中的深度学习

乘舟泛海赏雨 2020-12-15

展开全文

文章目录

深度学习是模仿人类大脑认识世界的方式，使用神经网络算法对视觉图像的各层级的特征进行提取。它突破了传统的分类与检测算法的计算性能的局限性，尤其在分类、物体识别、分割方面表现良好。

Halcon从17.12版本开始支持深度学习。

本章将介绍如何在Halcon中应用深度学习算法进行训练、评估和检测。

14.1 深度学习的基本概念

深度学习的概念源于人工神经网络的研究。含多隐层的多层感知器就是一种深度学习结构。深度学习通过组合底层特征形成更加抽象的高层表示属性类别或特征，以发现数据的分布式特征表示。深度学习的概念由Hinton等人于2006年提出。基于深度置信网络（Deep Belief Network, DBN）提出非监督贪心逐层训练算法，为解决深层结构相关的优化难题带来希望，随后提出多层自动编码器深层结构。此外LeCun等人提出的卷积神经网络也是第一个真正多层结构学习算法，它利用空间相对关系减少参数数目以提高训练性能。

深度学习和传统机器学习相比有以下三个优点：

1、高效率：例如用传统算法去评估一个棋局的优劣，可能需要专业的棋手花大量的时间去研究影响棋局的每一个因素，而且还不一定准确。而利用深度学习技术只要设计好网络框架，就不需要考虑繁琐的特征提取的过程。这也是 DeepMind公司的AlphaGo 能够强大到轻松击败专业的人类棋手的原因，它节省了大量的特征提取的时间，使得本来不可行的事情变为可行。

2、可塑性：在利用传统算法去解决一个问题时，调整模型的代价可能是把代码重新写一遍，这使得改进的成本巨大。深度学习只需要调整参数，就能改变模型。这使得它具有很强的灵活性和成长性，一个程序可以持续改进，然后达到接近完美的程度。

3、普适性：神经网络是通过学习来解决问题，可以根据问题自动建立模型，所以能够适用于各种问题，而不是局限于某个固定的问题。

14.1.1 Halcon中深度学习的应用

在Halcon中，深度学习主要用于以下3个方向：
（1）分类
（2）物体检测
（3）语义分割

14.1.2 系统需求

训练网络需要NVIDIA的GPU，GPU的计算能力至少需要3.0，且支持CUDA 10.0,建议使用SSD硬盘，以加快训练速度。

这里我使用的显卡为：丽台P4000
在这里插入图片描述

安装CUDA 10.0:
https://developer./cuda-10.0-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exelocal
在这里插入图片描述

参考：
如何在Windows x64系统下面玩Halcon 17.12 深度学习？https://www./thread-956-1-1.html

cuda安装教程+cudnn安装教程
https://blog.csdn.net/sinat_23619409/article/details/84202651

14.1.3 搭建深度学习环境

在这里插入图片描述

在例程库找到：
classify_fruit_deep_learning.hdev
路径： MVTec\HALCON-17.12-Progress\examples\hdevelop\Deep-Learning\Classification

在这里插入图片描述

* This example shows how to train a deep learning fruit classifier, along with
* a short overview of the necessary steps.
* 
* Initialization.
dev_update_off ()
dev_close_window ()
WindowWidth := 800
WindowHeight := 600
dev_open_window_fit_size (0, 0, WindowWidth, WindowHeight, -1, -1, WindowHandle)
set_display_font (WindowHandle, 16, 'mono', 'true', 'false')
* 
* Some procedures use a random number generator. Set the seed for reproducibility.
set_system ('seed_rand', 42)
* 
* Introduction text.
try
    dev_disp_introduction_text ()
catch (Exception)
    if (Exception[0] == 5200)
        dev_disp_missing_images_text ()
        stop ()
    else
        throw (Exception)
    endif
endtry
stop ()
dev_close_window ()
dev_resize_window_fit_size (0, 0, WindowWidth, WindowHeight, -1, -1)
dev_clear_window ()
* 
* ** TRAINING **
* 
* Read one of the pretrained networks.
read_dl_classifier ('pretrained_dl_classifier_compact.hdl', DLClassifierHandle)
* Path to directory with images.
RawDataFolder := 'food/' + ['apple_braeburn','apple_golden_delicious','apple_topaz','peach','pear']
* Get the raw data set with labels.
read_dl_classifier_data_set (RawDataFolder, 'last_folder', RawImageFiles, Labels, LabelIndices, Classes)
* Path of output directory for preprocessed data set.
PreprocessedFolder := 'fruit_preprocessed'
* Set to true to overwrite existing images.
OverwritePreprocessingFolder := false
* By default, we will remove the folder with the preprocessed data.
* In a real application you might want to keep this data (if the
* preprocessing does not change to save time).
RemovePreprocessingAfterExample := true
* 
* If the preprocessed has been generated already,
* we skip this part.
file_exists (PreprocessedFolder, FileExists)
if (not FileExists or OverwritePreprocessingFolder)
    * Preprocessing of the raw data.
    if (FileExists)
        remove_dir_recursively (PreprocessedFolder)
    endif
    * Create output directories.
    make_dir (PreprocessedFolder)
    for I := 0 to |Classes| - 1 by 1
        make_dir (PreprocessedFolder + '/' + Classes[I])
    endfor
    * Define output file names.
    parse_filename (RawImageFiles, BaseNames, Extensions, Directories)
    ObjectFilesOut := PreprocessedFolder + '/' + Labels + '/' + BaseNames + '.hobj'
    * Check if output file names
    * overlap in the preprocessed folder.
    * This is just a sanity check.
    check_output_file_names_for_duplicates (RawImageFiles, ObjectFilesOut)
    * Preprocess images and save them as hobj files.
    for I := 0 to |RawImageFiles| - 1 by 1
        read_image (Image, RawImageFiles[I])
        * Preprocess the image with a custom procedure
        * in order to remove the background.
        preprocess_dl_fruit_example (Image, ImagePreprocessed, DLClassifierHandle)
        * Write preprocessed image to hobj file.
        write_object (ImagePreprocessed, ObjectFilesOut[I])
        dev_disp_preprocessing_progress (I, RawImageFiles, PreprocessedFolder, WindowHandle)
    endfor
    dev_clear_window ()
    dev_disp_text ('Preprocessing done.', 'window', 'top', 'left', 'black', [], [])
endif
* 
* 2) Split data into training, validation, and test set.
* 
* Read the data, i.e., the paths of the images and their respective ground truth labels.
read_dl_classifier_data_set (PreprocessedFolder, 'last_folder', ImageFiles, Labels, LabelsIndices, Classes)
* 
* Split the data into three subsets,
* for training 70%, validation 15%, and testing 15%.
TrainingPercent := 70
ValidationPercent := 15
split_dl_classifier_data_set (ImageFiles, Labels, TrainingPercent, ValidationPercent, TrainingImages, TrainingLabels, ValidationImages, ValidationLabels, TestImages, TestLabels)
* 
* Set training hyper-parameters.
* In order to retrain the neural network, we have to specify
* the class names of our classification problem.
set_dl_classifier_param (DLClassifierHandle, 'classes', Classes)
* Set the batch size.
BatchSize := 64
set_dl_classifier_param (DLClassifierHandle, 'batch_size', BatchSize)
* Try to initialize the runtime environment.
try
    set_dl_classifier_param (DLClassifierHandle, 'runtime_init', 'immediately')
catch (Exception)
    dev_disp_error_text (Exception)
    if (RemovePreprocessingAfterExample and Exception[0] != 4104)
        remove_dir_recursively (PreprocessedFolder)
        dev_disp_text ('Preprocessed data in folder "' + PreprocessedFolder + '" have been deleted.', 'window', 'bottom', 'left', 'black', [], [])
    endif
    stop ()
endtry
* For this data set, an initial learning rate of 0.001
* has proven to yield good results.
InitialLearningRate := 0.001
set_dl_classifier_param (DLClassifierHandle, 'learning_rate', InitialLearningRate)
* In this example, we reduce the learning rate
* by a factor of 1/10 every 4th epoch.
LearningRateStepEveryNthEpoch := 30
LearningRateStepRatio := 0.1
* We iterate 100 times over the full training set.
NumEpochs := 100
* 
* Train the classifier.
* 
dev_clear_window ()
dev_disp_text ('Training has started...', 'window', 'top', 'left', 'black', [], [])
* 
PlotIterationInterval := 20
FileName := 'classifier_fruit.hdl'
train_fruit_classifier (DLClassifierHandle, FileName, NumEpochs, TrainingImages, TrainingLabels, ValidationImages, ValidationLabels, LearningRateStepEveryNthEpoch, LearningRateStepRatio, PlotIterationInterval, WindowHandle)
dev_disp_text ('Press Run (F5) to continue', 'window', 'bottom', 'right', 'black', [], [])
stop ()
clear_dl_classifier (DLClassifierHandle)
read_dl_classifier (FileName, DLClassifierHandle)
* 
* Compute the confusion matrix for the validation data set.
get_error_for_confusion_matrix (ValidationImages, DLClassifierHandle, Top1ClassValidation)
gen_confusion_matrix (ValidationLabels, Top1ClassValidation, [], [], WindowHandle, ConfusionMatrix)
dev_disp_text ('Validation data', 'window', 'top', 'left', 'gray', 'box', 'false')
dev_disp_text ('Press Run (F5) to continue', 'window', 'bottom', 'right', 'black', [], [])
stop ()
clear_matrix (ConfusionMatrix)
dev_clear_window ()
* 
* ** INFERENCE **
* 
* This part shows a typical inference scenario.
* Read the classifier.
clear_dl_classifier (DLClassifierHandle)
read_dl_classifier (FileName, DLClassifierHandle)
* If it is not possible to accumulate more than one image
* at a time the batch size should be set to 1.
set_dl_classifier_param (DLClassifierHandle, 'batch_size', 1)
* This initializes the runtime environment immediately.
set_dl_classifier_param (DLClassifierHandle, 'runtime_init', 'immediately')
* 
dev_resize_window_fit_size (0, 0, WindowWidth, WindowHeight, -1, -1)
dev_disp_inference_text ()
stop ()
* Read / acquire images in a loop and classify them.
for Index := 0 to 20 by 1
    ImageFile := RawImageFiles[floor(rand(1) * |RawImageFiles|)]
    read_image (Image, ImageFile)
    dev_resize_window_fit_image (Image, 0, 0, -1, -1)
    preprocess_dl_fruit_example (Image, ImagePreprocessed, DLClassifierHandle)
    apply_dl_classifier (ImagePreprocessed, DLClassifierHandle, DLClassifierResultHandle)
    get_dl_classifier_result (DLClassifierResultHandle, 'all', 'predicted_classes', PredictedClass)
    clear_dl_classifier_result (DLClassifierResultHandle)
    * 
    dev_display (Image)
    Text := 'Predicted class: ' + PredictedClass
    dev_disp_text (Text, 'window', 'top', 'left', 'white', 'box', 'false')
    dev_disp_text ('Press Run (F5) to continue', 'window', 'bottom', 'right', 'black', [], [])
    stop ()
endfor
stop ()
clear_dl_classifier (DLClassifierHandle)
if (RemovePreprocessingAfterExample)
    remove_dir_recursively (PreprocessedFolder)
    dev_disp_text ('End of program.\nPreprocessed data have been deleted.', 'window', 'bottom', 'right', 'black', [], [])
else
    dev_disp_text ('      End of program      ', 'window', 'bottom', 'right', 'black', [], [])
endif

训练过程：
在这里插入图片描述