高级检索

基于上下文信息和大语言模型的开放词汇室内三维目标检测

Contextual Information and Large Language Model for Open-Vocabulary Indoor 3D Object Detection

  • 摘要: 现有室内三维目标检测算法能检测的目标类别通常有限,这限制其在智能机器人领域的应用。开放词汇目标检测能在不定义目标类别的前提下检测给定场景的所有感兴趣目标,从而解决室内三维目标检测的不足。与此同时,大语言模型的先验知识能显著提升视觉任务的性能。然而现有的开放词汇室内三维目标检测研究存在仅关注目标信息,而忽视了上下文信息的问题。室内三维目标检测的输入数据主要是点云,存在稀疏和噪声问题。仅依赖目标点云信息会对三维目标检测结果产生负面影响。上下文信息包含场景描述,能对目标信息进行补充,从而提升目标检测中类别判定的准确率。为此,本文提出了基于上下文信息和大语言模型的开放词汇室内三维目标检测算法,该算法通过结合上下文信息和大语言模型的思维链推理获取检测结果。此外,该算法在SUN RGB-D和ScanNetV2数据集上进行了验证,实验结果表明了其有效性。

     

    Abstract: Existing indoor three-dimensional (3D) object detection is able to detect a limited number of object categories, thus limiting the application on intelligent robotics. Open-vocabulary object detection is able to detect all objects of interest in a given scene without defining object categories, thus solving the shortcomings of indoor 3D object detection. At the same time, the large language model with prior knowledge can significantly improve the performance of visual tasks. However, existing researches on open-vocabulary indoor 3D object detection only focuses on object information and ignores contextual information. The input data for indoor 3D object detection is mainly point cloud, which suffers from sparsity and noise problems. Relying only on the object point cloud can negatively affect the 3D detection results. Contextual information contains scene information, which can complement the object information to promote the recognition on object category. For this reason, this paper proposes an open-vocabulary 3D object detection algorithm based on contextual information assistance. The algorithm integrates contextual information and object information through a large language model, and then performs chain-of-thought reasoning. The proposed algorithm is validated on SUN RGB-D and ScanNetV2 datasets, and the experimental results show the effectiveness of the proposed algorithm.

     

/

返回文章
返回