具体实施方式

下面将结合本申请实施例中附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。通常在此处附图中描述和示出的本申请实施例的组件可以以各种不同的配置来布置和设计。因此，以下对在附图中提供的本申请的实施例的详细描述并非旨在限制要求保护的本申请的范围，而是仅仅表示本申请的选定实施例。基于本申请的实施例，本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本申请保护的范围。

应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。同时，在本申请的描述中，术语“第一”、“第二”等仅用于区分描述，而不能理解为指示或暗示相对重要性。

第一方面，请参照图1，本申请一些实施例提供了一种咽喉定位模型训练方法，用于获取咽喉定位模型，方法包括以下步骤：

S1、基于公共数据集获取人脸数据集；

S2、基于喉结识别模型对人脸数据集进行分类以获取有喉结数据集和无喉结数据集，喉结识别模型用于分析图像中是否存在喉结；

S3、基于有喉结数据集和无喉结数据集分别训练两个咽喉定位模型，并获取两个咽喉定位模型的输出数据集；

S4、获取修正处理后的输出数据集以作为新的人脸数据集；

S5、重复执行步骤S2-步骤S4以获取训练后的咽喉定位模型。

具体地，公共数据集为公共的图像数据集，在本申请实施例中，优选为可从互联网直接获取的数据集；本申请实施例的咽喉定位模型训练方法基于公共数据集训练咽喉定位模型具有足够的训练数据来进行训练。

更具体地，人脸数据集为合适的适用于咽喉定位模型进行训练的图像所组成的数据集，人脸数据集中的元素优选为包含完整人脸和脖子的图像。

更具体地，喉结识别模型为预先训练好的或初步完成训练的识别模型，其能用于区分人脸数据集中的元素（图像）中是否存在喉结，以根据分类结果将元素归入至有喉结数据集或无喉结数据集中；由于不同人的喉结的明显程度不同，尤其是女性或肥胖人群的图像中可以认为是没有喉结，而咽喉的位置位于喉结中心，考虑到神经网络的特征识别特性，若仅训练一个咽喉定位模型来针对上述这些不同类型的图像进行识别，存在训练量大、识别精度不高的问题，因此，本申请实施例的咽喉定位模型训练方法基于喉结识别模型对人脸数据集进行分类，以获取有喉结数据集和无喉结数据集，然后分别基于两种数据集训练咽喉定位模型，使得最终训练完成获取的两个咽喉定位模型分别能针对具有喉结的人脸图像和不具有喉结的人脸图像进行咽喉定位，该训练方式可有效降低咽喉定位模型训练所需的数据量，并提高咽喉定位模型的定位准确性。

需要说明的是，本申请实施例的咽喉定位模型训练方法训练获取的两个咽喉定位模型应当结合喉结识别模型进行使用，即在实际使用过程中，咽喉定位模型需先利用喉结识别模型对采集到的图像进行分类再选择合适的咽喉定位模型进行咽喉定位。

更具体地，咽喉定位模型为神经网络模型，在本申请实施例中，属于根据输入图像而在图像上生成咽喉定位信息的学习模型，该咽喉定位信息可以是特征点或者热力图等，该咽喉定位模型可以采用常用的学习模型的训练方法进行训练，如在有喉结数据集和无喉结数据集中的元素中的咽喉位置增加咽喉特征点作为标签来训练咽喉定位模型；其中，有喉结数据集中的图像以喉结处最突起的地方作为咽喉特征点，无喉结数据集中的图像则以咽喉位置的中心点作为咽喉特征点。

更具体地，步骤S3为基于有喉结数据集和无喉结数据集对两个咽喉定位模型进行的一次训练过程或者是一次咽喉定位过程，为了提咽喉定位模型的定位精度，还需进行多次训练或咽喉定位以进行参数微调，故需要执行步骤S4确定下一阶段训练所需的人脸数据集，并基于步骤S5执行下一阶段的训练。

更具体地，在本申请实施中，步骤S3的训练过程中，咽喉定位模型对对应数据集中图像进行咽喉定位而在对应图像上生成咽喉定位信息，这些具有咽喉定位信息的图像为咽喉定位模型的输出数据，这些输出数据组成了输出数据集；本申请实施例的咽喉定位模型训练方法旨在利用这些输出数据集中的咽喉定位信息作为新的标签来重复训练咽喉定位模型，但尚未训练好的咽喉定位模型的定位精度略有欠缺，若步骤S5直接利用这些输出数据集对咽喉定位模型进行重复训练以微调咽喉定位模型的网络参数，则不利于咽喉定位模型的收敛，故本申请实施例的咽喉定位模型训练方法以修正处理后的输出数据集作为新的人脸数据集，其中，新的人脸数据集的标签为步骤S3中咽喉定位模型输出的咽喉定位信息。

更具体地，输出数据集的修正处理为去除咽喉定位信息偏差较大的图像或修正咽喉定位信息偏差较大的图像中的咽喉定位信息，优选为采用去除的方式进行；上述修正处理过程可以人工执行或基于软件比对原标签和咽喉定位信息之间关系执行。

更具体地，步骤S2基于步骤S4产生的新的人脸数据集进行识别前，还可以基于步骤S1引入新的图像以扩充对应的人脸数据集，从而确保人脸数据集的数据量充足；在本申请实施例中，公共数据集获取的人脸数据集中的图像均已基于标签工具在咽喉位置上标注标签；该处理方式中，新的人脸数据集中的图像无需人工进行标签标注，且对咽喉定位模型而言，相当于新的已进行标签标注的图像，故基于步骤S4产生的新的人脸数据集以对咽喉定位模型进行重复训练的处理的方式能极大地减少图像标注时间，进而大大地提高了咽喉定位模型的训练效率。

本申请实施例的咽喉定位模型训练方法基于公共数据集获取了足够多的训练数据来对咽喉定位模型进行训练，同时，训练过程先基于喉结识别模型对人脸数据集进行分类，以训练两个分别用于针对有喉结的人脸图像和无喉结的人脸图像进行咽喉定位的咽喉定位模型，有效提高了咽喉定位的准确率；另外，本申请实施例的咽喉定位模型训练方法基于修正处理后的、源自咽喉定位模型的输出数据集制作的新的人脸数据集来继续对咽喉定位模型进行训练，有效减少了人工标注标签的时间，提高了咽喉定位模型的训练效率。

更具体地，一般情况下，本申请实施例的咽喉定位模型训练方法一般执行一次步骤S5，即以新的人脸数据集对咽喉定位模型进而一轮二次训练，便能完成咽喉定位模型的训练。

在一些优选的实施方式中，公共数据集为FADID数据集。

具体地，FADID（Face Alignment Dataset used In Driving）即为车内人脸关键点数据集，该数据集为在FLD（日光荧光灯）环境且真实驾驶情况下建立的一个基准数据集，包括姿势的变化（最大俯仰45度，横滚和横摆30度）、照明的各种变化（前、左、右和后）以及使用近红外相机的头发或太阳镜引起的部分遮挡，该数据集中的视频或图像均包含下至肩膀、上至完全人脸的部分，其内具有众多有效数据（即具有外露咽喉和完整人脸的图像），故尤其适用于本申请实施例的咽喉定位模型训练方法训练咽喉定位模型使用。

需要说明的是，在别的实施方式中，公共数据集还可以是300W、300VW等包含人脸关键点的数据集。

在一些优选的实施方式中，人脸数据集中元素基于FADID的人脸关键点和咽喉特征点作为训练标签。

具体地，人脸数据集中元素即为基于咽喉位置进行标签标注（即标注有咽喉特征点）的图像，FADID中的图像本身包含19个人脸关键点，本申请实施例的咽喉定位模型训练方法结合人脸关键点和咽喉特征点共20个特征点作为训练标签来对咽喉定位模型进行训练，以使咽喉定位模型能利用特征点之间的关系来进行咽喉定位训练，使得咽喉定位模型能考虑人脸五官和咽喉之间的几何关系来进行咽喉定位，进而提高咽喉定位模型的定位精度并减少训练轮次。

在一些优选的实施方式中，喉结识别模型为二分类模型，且具有一个用于输出存在喉结的概率的全连接层，人脸数据集中元素基于比较存在喉结的概率和预设阈值的大小进行分类。

具体地，喉结识别模型只需简单判断图像中人物是否存在喉结，采用轻量化的神经网络即可，故采用二分类模型，在本申请实施例中优选为采用MobileNetV3模型，MobileNetV3模型为常用的图像分类模型，可基于图像进行训练。

更具体地，在本申请实施例中，喉结识别模型的最后一层输出层设计为：只有一个全连接层，该全连接层用于输出存在喉结的概率，即去除了输出不存在喉结的概率的全连接层，简化了模型的结构。

更具体地，步骤S2基于比较存在喉结的概率和预设阈值的大小来对图像进行分类，其中，存在喉结的概率大于或等于预设阈值的图像归入有喉结数据集；预设阈值为人工预先输入的判断值，在本申请实施例中，优选为0.7。

在一些优选的实施方式中，咽喉定位模型为HRNet V2模型。

具体地，HRNet是一种高分辨率神经网络，其特点是在神经网络传播过程中保持特征图的分辨率，根据场景在最终输出的时候采用不同的特征图融合方式来进行输出，其中，现有的HRNet V2模型自身的最后的输出层与前面的backbone层的四个分支输出的不同分辨率的特征图（低分辨率的特征图上采样到高分辨率）拼接连接，故其输出结果结合了四个分支的空间信息，所以在特征点检测的应用中有更好的效果，故应用在本申请实施例的咽喉定位模型训练方法中能准确地进行咽喉特征点的分析、定位。

在一些优选的实施方式中，HRNet V2模型的输入层分辨率为384*512。

具体地，现有的HRNet V2模型直接应用在本申请实施例的咽喉定位模型训练方法中进行咽喉定位存在一定缺陷，普遍表现为定位位置不准确，其主要原因有两点：一是HRNet V2模型输出分辨率较低，单位像素对于人脸上的特征点来说，在相应的现实生活中的物理距离较大，导致HRNet V2模型难以准确定位特征点，二是现有的HRNet V2模型输入的图像需要处理成分辨率为256*256的正方形图像，而包含咽喉的人脸ROI（感兴趣区域）一般为3:4宽高比区域，将输入图像处理成正方形图像时可能会导致咽喉位置的局部图像丢失；因此，本申请实施例的咽喉定位模型训练方法修改HRNet V2模型的输入层，使其输入层能输入分辨率为384*512的图像，使得HRNet V2模型能对分辨率为384*512的图像进行咽喉定位处理，解决了原分辨率过低及ROI形状不匹配引起的特征点定位不准确的缺陷，进而提高了咽喉定位模型对咽喉的定位精度。

需要说明的是，人脸数据集的元素或实际需要识别的图像在输入咽喉定位模型前应当处理为384*512的图像。

更具体地，原本HRNet V2模型一般需要对输入的图像进行两次卷积操作使输入的图像下采样至原来尺寸的1/4，会引起图像信息的损失，一定程度上也会影响咽喉定位模型的咽喉定位精度或咽喉定位模型的训练效率，故本申请实施例的咽喉定位模型训练方法去除了HRNet V2模型中的一个conv2d层和batchnorm层以减少一次卷积操作，使得输入的图像仅进行一次下采样处理便传输到HRNet V2模型backbone层中，该网络层结构能有效减少图像信息的损失，使得输入的图像保留了更多面部特征的信息，使得训练后的咽喉定位模型的定位精度更高；需要说明的是，减少了一次卷积操作一定程度上也弥补了分辨率提高带来的时间损失，故修改前后的HRNet V2模型的识别定位效率差异不大。

在一些优选的实施方式中，咽喉定位模型基于热力图进行输出。

具体地，由于咽喉属于一个人体器官，其占据图像中一定范围，故采用热力图进行输出能准确表达出咽喉所在区域。

更具体地，相对于原本输入的图像的尺寸为256*256的HRNet V2模型，本申请实施例的咽喉定位模型训练方法训练的咽喉定位模型的backbone层中的高分辨率通道的特征图由原来的64*64调整为192*256，尺寸越大的特征图越能提供准确的热力图输出。

在一些优选的实施方式中，由于HRNet V2模型去除了一个conv2d层和batchnorm层以减少了一次卷积操作，其余下的一个conv2d层优选为将原来的卷积核大小由3*3调整为6*6，以增大一倍卷积核大小来保证具有足够大的感受野，以使得其连接的backbone层里的stage4里低分辨率的卷积层的感受野能覆盖整个输入的图片，同时，靠近输入端的conv2d层采用更大的卷积核也能降低卷积层的计算量，提高卷积效率。

在一些优选的实施方式中，步骤S3在第一轮训练过程中（即基于步骤S1获取的人脸数据集进行训练的过程中），采用1e-2的学习率进行60次迭代训练，然后再使用1e-3的学习率进行30次迭代训练；该采用不同学习率分阶段进行训练的处理方式能使咽喉定位模型快速收敛以完成训练。

第二方面，请参照图2，本申请一些实施例还提供了一种咽喉定位模型训练装置，用于获取咽喉定位模型，装置包括：

第一获取模块201，用于基于公共数据集获取人脸数据集；

分类模块202，用于基于喉结识别模型对人脸数据集进行分类以获取有喉结数据集和无喉结数据集，喉结识别模型用于分析图像中是否存在喉结；

训练模块203，用于基于有喉结数据集和无喉结数据集分别训练两个咽喉定位模型，并获取两个咽喉定位模型的输出数据集；

第二获取模块204，用于获取修正处理后的输出数据集以作为新的人脸数据集；

循环模块205，用于重复触发控制分类模块202、训练模块203及第二获取模块204运行以获取训练后的咽喉定位模型。

本申请实施例的咽喉定位模型训练装置基于公共数据集获取了足够多的训练数据来对咽喉定位模型进行训练，同时，训练过程先基于喉结识别模型对人脸数据集进行分类，以训练两个分别用于针对有喉结的人脸图像和无喉结的人脸图像进行咽喉定位的咽喉定位模型，有效提高了咽喉定位的准确率；另外，本申请实施例的咽喉定位模型训练装置基于修正处理后的、源自咽喉定位模型的输出数据集制作的新的人脸数据集来继续对咽喉定位模型进行训练，有效减少了人工标注标签的时间，提高了咽喉定位模型的训练效率。

在一些优选的实施方式中，本申请实施例的咽喉定位模型训练装置用于执行上述第一方面提供的咽喉定位模型训练方法。

第三方面，请参照图3，本申请一些实施例还提供了一种电子设备的结构示意图，本申请提供一种电子设备，包括：处理器301和存储器302，处理器301和存储器302通过通信总线303和/或其他形式的连接机构（未标出）互连并相互通讯，存储器302存储有处理器301可执行的计算机可读取指令，当电子设备运行时，处理器301执行该计算机可读取指令，以执行时执行上述实施例的任一可选的实现方式中的方法。

第四方面，本申请实施例提供一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时，执行上述实施例的任一可选的实现方式中的方法。其中，计算机可读存储介质可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器（Static Random Access Memory, 简称SRAM），电可擦除可编程只读存储器（Electrically Erasable Programmable Read-Only Memory, 简称EEPROM），可擦除可编程只读存储器（Erasable Programmable Read Only Memory, 简称EPROM），可编程只读存储器（Programmable Red-Only Memory, 简称PROM），只读存储器（Read-OnlyMemory, 简称ROM），磁存储器，快闪存储器，磁盘或光盘。

综上，本申请实施例提供了一种咽喉定位模型训练方法、装置、电子设备及存储介质，其中，本申请实施例的咽喉定位模型训练方法基于公共数据集获取了足够多的训练数据来对咽喉定位模型进行训练，同时，训练过程先基于喉结识别模型对人脸数据集进行分类，以训练两个分别用于针对有喉结的人脸图像和无喉结的人脸图像进行咽喉定位的咽喉定位模型，有效提高了咽喉定位的准确率；另外，本申请实施例的咽喉定位模型训练方法基于修正处理后的、源自咽喉定位模型的输出数据集制作的新的人脸数据集来继续对咽喉定位模型进行训练，有效减少了人工标注标签的时间，提高了咽喉定位模型的训练效率。

在本申请所提供的实施例中，应该理解到，所揭露装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。

另外，作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。

再者，在本申请各个实施例中的各功能模块可以集成在一起形成一个独立的部分，也可以是各个模块单独存在，也可以两个或两个以上模块集成形成一个独立的部分。

在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。

以上所述仅为本申请的实施例而已，并不用于限制本申请的保护范围，对于本领域的技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

The specific embodiment

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application, and it is obvious that the described embodiments are only part of the embodiments of the present application, not all embodiments. The components of the embodiment of the present application, which are normally described and shown in the drawings herein, may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely indicates the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person skilled in the art without creative work fall within the scope of protection of the present application.

It should be noted that similar designations and letters indicate similar terms in the drawings below, so that once an item is defined in a drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, the terms "first", "second", etc., are only used to distinguish the description and cannot be construed as indicating or implying relative importance.

First, please refer to Figure 1, some embodiments of the present application provide a throat positioning model training method for obtaining a throat positioning model, the method comprises the following steps:

S1. Obtain face datasets based on public datasets;

S2, the face dataset was classified based on the Adam's apple recognition model to obtain the dataset with and without Adam's apple, and the Adam's apple recognition model was used to analyze whether there was an Adam's apple in the image.

S3, two throat localization models were trained based on the Adam's apple dataset and the Adam's apple dataset, and the output datasets of the two throat localization models were obtained.

S4. obtain the corrected output dataset as a new face dataset;

S5, Repeat Steps S2-S4 to obtain the trained throat localization model.

Specifically, the public dataset is a public image dataset, and in the embodiment of the present application, a dataset that can be directly obtained from the Internet is preferred, and the throat positioning model training method of the embodiment of the present application is based on the public dataset to train the throat positioning model with sufficient training data for training.

More specifically, the face dataset is a dataset composed of images suitable for throat positioning model training, and the elements in the face dataset are preferably images that contain the complete face and neck.

More specifically, the Adam's apple recognition model is a pre-trained or preliminarily trained recognition model, which can be used to distinguish whether there is an Adam's apple in the elements (images) in the face dataset, so as to classify the elements into the dataset with or without Adam's apple according to the classification results; Therefore, the throat positioning model training method of the embodiment of the present application classifies the face dataset based on the Adam's apple recognition model to obtain a dataset with Adam's apple and a dataset without Adam's apple, and then trains the throat positioning model based on the two datasets respectively, so that the two throat positioning models obtained after the final training can respectively locate the throat for the face image with the Adam's apple and the face image without the Adam's apple, and the training mode can effectively reduce the amount of data required for the training of the throat localization model and improve the positioning accuracy of the throat positioning model。

It should be noted that the two throat positioning models trained and obtained by the throat positioning model training method of the embodiment of the present application should be used in combination with the Adam's apple recognition model, that is, in the actual use process, the throat positioning model needs to first use the Adam's apple recognition model to classify the collected images, and then select an appropriate throat positioning model for throat positioning.

More specifically, the throat localization model is a neural network model, in the embodiment of the present application, it belongs to a learning model that generates throat positioning information on the image according to the input image, the throat positioning information may be a feature point or a heat map, etc., the throat localization model can be trained by the training method of a commonly used learning model, such as adding a throat landmark point as a label to the throat position in the elements in the Adam's apple dataset and the non-Adam's apple dataset to train the throat positioning model; wherein the image in the Adam's apple dataset takes the most protruding place at the Adam's apple as the throat feature point, and the image in the Adam's apple dataset takes the center point of the throat position as the throat feature point。

More specifically, step S3 is a training process or a throat positioning process for two throat positioning models based on a dataset with Adam's apple and a dataset without Adam's apple, and in order to improve the positioning accuracy of the throat positioning model, multiple training or throat positioning needs to be carried out for parameter fine-tuning, so step S4 needs to be executed to determine the face dataset required for the next stage of training, and the next stage of training is executed based on step S5.

More specifically, in the implementation of the present application, in the training process of step S3, the throat positioning model performs throat positioning on the images in the corresponding dataset and generates throat positioning information on the corresponding images, and these images with throat positioning information are the output data of the throat positioning model, and these output data constitute an output data set; the throat positioning model training method of the embodiment of the present application is designed to repeatedly train the throat positioning model by using the throat positioning information in these output datasets as a new label, but the positioning accuracy of the throat positioning model that has not yet been trained is slightly lackingIf step S5 directly uses these output datasets to repeatedly train the throat positioning model to fine-tune the network parameters of the throat positioning model, it is not conducive to the convergence of the throat positioning model, so the throat positioning model training method of the embodiment of the present application takes the modified output dataset as a new face dataset, wherein the label of the new face dataset is the throat positioning information output by the throat positioning model in step S3.

More specifically, the correction processing of the output dataset is to remove the throat positioning information in the image with a large deviation of the throat positioning information or the throat positioning information in the image with a large deviation from the correction of the throat positioning information, preferably by adopting the removal method, and the correction process can be performed manually or based on the software comparison of the relationship between the original label and the throat positioning information.

More specifically, before the step S2 recognizes based on the new face dataset generated by step S4, a new image can be introduced based on step S1 to enrich the corresponding face dataset, so as to ensure that the data volume of the face dataset is sufficient; in the embodiment of the present application, the images in the face dataset obtained by the public dataset have been labeled at the throat position based on the labeling tool; in this processing mode, the images in the new face dataset do not need to be manually labeled, and to the throat positioning model, it is equivalent to a new image that has been labeledTherefore, the new face dataset generated based on step S4 can greatly reduce the image annotation time by repeating the training of the throat localization model, thereby greatly improving the training efficiency of the throat positioning model.

The throat positioning model training method of the embodiment of the present application obtains enough training data based on the public dataset to train the throat positioning model, and at the same time, the training process first classifies the face dataset based on the Adam's apple recognition model to train two throat positioning models that are respectively used for throat positioning for face images with Adam's apple and face images without Adam's apple, effectively improves the accuracy of throat positioning; The new face dataset derived from the output dataset of the throat positioning model was used to continue the training of the throat positioning model, which effectively reduced the time of manual labeling and improved the training efficiency of the throat positioning model.

More specifically, in general, the throat positioning model training method of the embodiment of the present application generally performs a step S5, that is, the throat localization model is trained in a round of secondary training with a new face dataset, so that the training of the throat positioning model can be completed.

Among some preferred embodiments, the public dataset is the FAKE dataset.

Specifically, the FADID (Face Alignment Dataset used In Driving) is a key data set of faces in the car, which is a benchmark dataset established in the FLD (daylight fluorescent) environment and real driving situations, including changes in posture (maximum pitch 45 degrees, roll and roll 30 degrees), various changes in lighting (front, left, right and back) and the partial occlusion caused by the use of a near-infrared camera hair or sunglasses, the video or image in the dataset contains a part that goes down to the shoulder and up to the complete face, and has a large number of valid data (i.e., an image with an exposed throat and a complete face), so it is particularly suitable for the throat positioning model training method of the embodiment of the present application to train the throat positioning model.

It should be noted that in other embodiments, the public dataset may also be a dataset containing key points of faces, such as 300W and 300VW.

In some preferred embodiments, the elements in the face dataset are based on the face key points and throat feature points of FADID as training labels.

Specifically, the elements in the face dataset are images labeled (i.e., labeled with throat feature points) based on throat position, the image itself in FAKE contains 19 face key points, and the throat localization model training method in the embodiment of the present application combines a total of 20 feature points of face key points and throat feature points as training labels to train the throat positioning model, so that the throat positioning model can use the relationship between feature points to carry out throat positioning training, so that the throat positioning model can consider the geometric relationship between the facial features of the face and the throat to carry out throat positioning, so as to improve the positioning accuracy of the throat positioning model and reduce the number of training rounds。

In some preferred embodiments, the Adam's apple recognition model is a binary classification model with a fully connected layer for outputting the probability of the presence of the Adam's apple, and the elements in the face dataset are classified based on comparing the probability of the presence of the Adam's apple with the size of the preset threshold.

Specifically, the Adam's apple recognition model only needs to simply judge whether the person in the image has an Adam's apple, and a lightweight neural network is adopted, so a binary classification model is adopted, and in the embodiment of the present application, the MobileNetV3 model is preferred, and the MobileNetV3 model is a commonly used image classification model, which can be trained based on the image.

More specifically, in the embodiment of the present application, the last output layer of the Adam's apple recognition model is designed as follows: there is only one fully connected layer, and the fully connected layer is used to output the probability of the presence of the Adam's apple, that is, the fully connected layer that outputs the probability of the absence of the Adam's apple is removed, simplifying the structure of the model.

More specifically, step S2 classifies images based on comparing the probability of the presence of the Adam's apple and the size of the preset threshold, wherein the images with the probability of the presence of the Adam's apple greater than or equal to the preset threshold are classified into the dataset with the Adam's apple, and the preset threshold is a judgment value pre-entered by artificially, preferably 0.7 in the embodiment of the present application.

Among some preferred embodiments, the throat localization model is the HRNet V2 model.

Specifically, HRNet is a high-resolution neural network, which is characterized by maintaining the resolution of the feature map during the propagation process of the neural network, and using different feature map fusion methods to output according to the scene when the final output is carried out, among which the existing HRNet The last output layer of the V2 model itself is spliced and connected with the feature map of different resolutions (the low-resolution feature map is upsampled to high resolution) output by the four branches of the previous backbone layer, so the output result combines the spatial information of the four branches, so it has a better effect in the application of feature point detection, so it can accurately carry out the analysis and positioning of throat feature points in the throat positioning model training method of the present application embodiment.

In some preferred embodiments, the input layer resolution of the HRNet V2 model is 384*512.

Specifically, the existing HRNet V2 model is directly applied to the throat positioning method in the throat positioning model training method of the present application embodiment, which has certain defects, and is generally manifested as inaccurate positioning position, which is mainly due to two reasons: first, the output resolution of the HRNet V2 model is low, and the physical distance per unit pixel is large for the feature points on the human face in the corresponding real life, resulting in the difficulty of the HRNet V2 model to accurately locate the feature points, and the second is the existing HRNet The input image of the V2 model needs to be processed into a square image with a resolution of 256*256, and the ROI (region of interest) of the face containing the throat is generally a 3:4 aspect ratio region, and the local image of the throat position may be lost when the input image is processed into a square image; therefore, the throat positioning model training method of the embodiment of the present application modifies the input layer of the HRNet V2 model so that the input layer can input an image with a resolution of 384*512, so that HRNet The V2 model can locate the throat of the image with a resolution of 384*512, which solves the defects of inaccurate feature point positioning caused by the low original resolution and the mismatch of ROI shape, and then improves the positioning accuracy of the throat positioning model.

It should be noted that the elements of the face dataset or the images that actually need to be recognized should be processed as 384*512 images before being input into the throat positioning model.

More specifically, the original HRNet V2 model generally needs to perform two convolution operations on the input image to downsample the input image to 1/4 of the original size, which will cause the loss of image information, and will also affect the throat positioning accuracy of the throat positioning model or the training efficiency of the throat positioning model to a certain extent, so the throat positioning model training method in the embodiment of the present application removes HRNet A conv2d layer and a batchnorm layer in the V2 model reduce the convolution operation, so that the input image is downsampled only once before being transferred to the HRNet In the backbone layer of the V2 model, the network layer structure can effectively reduce the loss of image information, so that the input image retains more information about facial features, and makes the positioning accuracy of the trained throat positioning model higher.

In some preferred embodiments, the throat positioning model is output based on a heat map.

Specifically, since the throat is a human organ that occupies a certain area in the image, the output of the heat map can accurately represent the area where the throat is located.

More specifically, compared with the HRNet V2 model with a size of 256*256 of the original input image, the feature map of the high-resolution channel in the backbone layer of the throat localization model trained by the throat localization model training method of the embodiment of the present application is adjusted from the original 64*64 to 192*256, and the larger the size of the feature map, the more accurate the heat map output.

In some preferred embodiments, since the HRNet V2 model removes a conv2d layer and a batchnorm layer to reduce a convolution operation, the remaining conv2d layer is preferably adjusted from 3*3 to 6* 6. Doubling the size of the convolutional kernel to ensure that it has a large enough receptive field, so that the receptive field of the low-resolution convolutional layer in Stage4 in the backbone layer connected to it can cover the entire input image, and at the same time, the larger convolutional kernel of the Conv2D layer near the input can also reduce the computational cost of the convolutional layer and improve the convolution efficiency.

In some preferred embodiments, step S3 uses the learning rate of 1e-2 for 60 iterative training in the first round of training (i.e., in the process of training based on the face dataset obtained in step S1), and then uses the learning rate of 1e-3 for 30 iterative training.

Second, please refer to Figure 2, some embodiments of the present application also provide a throat positioning model training device for obtaining a throat positioning model, the device comprises:

the first acquisition module 201 is used for obtaining a face dataset based on a public dataset;

the classification module 202 is used to classify the face dataset based on the Adam's apple recognition model to obtain a dataset with and without Adam's apple, and the Adam's apple recognition model is used to analyze whether there is an Adam's apple in the image;

the training module 203 is used for training two throat localization models based on the Adam's apple dataset and the Adam's apple dataset respectively, and obtaining the output datasets of the two throat positioning models;

the second acquisition module 204 is used for obtaining the output dataset after correction processing as a new face dataset;

the cyclic module 205 is used for repeatedly triggering the operation of the control classification module 202, the training module 203 and the second acquisition module 204 to obtain the throat positioning model after training.

The throat localization model training device of the embodiment of the present application obtains enough training data based on the public dataset to train the throat positioning model, and simultaneously, the training process first classifies the face dataset based on the Adam's apple recognition model to train two throat positioning models for throat positioning for face images with Adam's apple and face images without Adam's apple, respectively, effectively improves the accuracy of throat positioning; The new face dataset derived from the output dataset of the throat positioning model was used to continue the training of the throat positioning model, which effectively reduced the time of manual labeling and improved the training efficiency of the throat positioning model.

In some preferred embodiments, the throat positioning model training device of the embodiment of the present application is used for performing the throat positioning model training method provided in the first aspect.

Thirdly, please refer to Figure 3, some embodiments of the present application also provide a schematic diagram of the structure of an electronic device, the present application provides an electronic device, comprising: a processor 301 and a memory 302, a processor 301 and a memory 302 through a communication bus 303 and/ or other forms of connecting mechanisms (not indicated) are interconnected and communicate with each other, the memory 302 stores executable computer-readable instructions of the processor 301, and when the electronic device is running, the processor 301 executes the computer-readable instructions to execute the method in any of the optional embodiments of the above embodiment when executed.

Fourth, the embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, executes a method in any of the optional embodiments of the above embodiment. Among them, computer-readable storage media can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory, EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic memory, flash memory, disk or optical disc.

In summary, the embodiment of the present application provides a throat positioning model training method, device, electronic equipment and storage medium, wherein the throat positioning model training method of the embodiment of the present application obtains enough training data based on the public dataset to train the throat positioning model, and simultaneously, the training process first classifies the face dataset based on the Adam's apple recognition model to train two throat positioning models for throat positioning for face images with Adam's apple and face images without Adam's apple, so that the accuracy of throat positioning is effectively improved; The new face dataset derived from the output dataset of the throat positioning model was used to continue the training of the throat positioning model, which effectively reduced the time of manual labeling and improved the training efficiency of the throat positioning model.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be realized by other means. The device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be another division mode when the actual implementation is realized, and for example, a plurality of units or components can be combined or can be integrated into another system, or some features can be ignored, or not executed. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be indirect coupling or communication connection through some communication interface, device or unit, which may be in electrical, mechanical or other form.

In addition, the cells described as detached parts may or may not be physically separated, and the parts shown as units may or may not be physical units, i.e., they may be located in one place, or they can be distributed across multiple network units. Some or all of these units can be selected according to actual needs to realize the purpose of the present embodiment.

Furthermore, each functional module in each embodiment of the present application may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

For the purposes of this article, relational terms such as first and second, etc., are used solely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or sequence between those entities or operations.

The foregoing is only an embodiment of the present application and is not intended to limit the scope of protection of the present application, which is subject to various changes and variations for those skilled in the art. Any amendments, equivalent replacements, improvements, etc., made within the spirit and principles of this application shall be included in the scope of protection of this application.