Vehicle trajectory prediction based on spatio-temporal Transformer feature fusion

ZHAO Wenhong; WANG Wei; WAN Zilu

doi:10.11959/j.issn.1000-436x.2024192

您当前的位置：

首页 >

文章列表页 >

Vehicle trajectory prediction based on spatio-temporal Transformer feature fusion

Correspondences | 更新时间：2024-12-24

- Vehicle trajectory prediction based on spatio-temporal Transformer feature fusion
- Journal on Communications Vol. 45, Issue 11, Pages: 267-276(2024)
- 作者机构：
  
  1.嘉兴南湖学院公共基础教学部，浙江嘉兴 314001
  2.电磁空间安全全国重点实验室，浙江嘉兴 314033
  3.浙江工业大学信息工程学院，浙江杭州 310013
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(62231027;U19B2015;U21B2001);Research Fund Project of Jiaxing Nanhu University(62211ZL)
- DOI：10.11959/j.issn.1000-436x.2024192
  CLC： TP183
- Received：28 June 2024，
  
  Revised：2024-10-17，
  
  Published：25 November 2024
- 稿件说明：
移动端阅览
赵文红,王巍,万子璐.基于时空Transformer特征融合的车辆轨迹预测[J].通信学报,2024,45(11):267-276.

ZHAO Wenhong,WANG Wei,WAN Zilu.Vehicle trajectory prediction based on spatio-temporal Transformer feature fusion[J].Journal on Communications,2024,45(11):267-276.
赵文红,王巍,万子璐.基于时空Transformer特征融合的车辆轨迹预测[J].通信学报,2024,45(11):267-276. DOI： 10.11959/j.issn.1000-436x.2024192.

ZHAO Wenhong,WANG Wei,WAN Zilu.Vehicle trajectory prediction based on spatio-temporal Transformer feature fusion[J].Journal on Communications,2024,45(11):267-276. DOI： 10.11959/j.issn.1000-436x.2024192.

摘要

在复杂的交通环境下，自动驾驶汽车需要充分地分析周围交通物体的运动方向、运动速度等信息，并准确预测未来的轨迹。针对这个问题，提出了一种基于时空Transformer的网络模型。该模型首先利用空间自注意力机制，通过捕捉同一时刻下车辆间的空间相互作用，实现对多车空间关系交互性的精确建模；随后通过时间自注意力机制提取连续帧的时间依赖关系，以此生成一组能够反映车辆动态行为的时空特征；最后这些特征被送入解码器，以预测所有车辆在未来5 s内的运动轨迹。在公开的NGSIM数据集上进行了训练和验证，与其他的先进方案相比，该模型在未来5 s的轨迹预测中具有更高的准确性和精度，长期预测准确率比先进方案提高14.6%。

Abstract

In complex traffic environments

autonomous vehicles must thoroughly analyze the motion direction

speed

and other information of surrounding traffic objects to accurately predict future trajectories. A network model based on spatio-temporal Transformer was proposed to address this issue. The framework initially employs a spatial self-attention mechanism to capture the spatial interactions between vehicles at the same moment

achieving precise modeling of the spatial relationship interactivity among multiple vehicles. Subsequently

a temporal self-attention mechanism was utilized to extract the temporal dependencies between consecutive frames

thereby generating a set of spatiotemporal features that reflect the dynamic behavior of vehicles. These features were then fed into a decoder to predict the motion trajectories of vehicles over the next 5 s. The proposed model was trained and validated on the publicly available NGSIM dataset. Compared to other state-of-the-art schemes

our scheme demonstrates greater accuracy and precision in trajectory prediction over the subsequent 5 s. The long-term forecasting accuracy is increased by 14.6% compared to the advanced schemes.

关键词

Keywords

references

LUO W J , YANG B , URTASUN R . Fast and furious: real time end-to-end 3D detection, tracking and motion forecasting with a single convolutional net [C ] // Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2018 : 3569 - 3577 .

温惠英 , 张伟罡 , 赵胜 . 基于生成对抗网络的车辆换道轨迹预测模型 [J ] . 华南理工大学学报(自然科学版) , 2020 , 48 ( 5 ): 32 - 40 .

WEN H Y , ZHANG W G , ZHAO S . Vehicle lane-change trajectory prediction model based on generative adversarial networks [J ] . Journal of South China University of Technology (Natural Science Edition) , 2020 , 48 ( 5 ): 32 - 40 .

KHOSROSHAHI A , OHN-BAR E , TRIVEDI M M . Surround vehicles trajectory analysis with recurrent neural networks [C ] // Proceedings of the 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC) . Piscataway : IEEE Press , 2016 : 2267 - 2272 .

PHILLIPS D J , WHEELER T A , KOCHENDERFER M J . Generalizable intention prediction of human drivers at intersections [C ] // Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV) . Piscataway : IEEE Press , 2017 : 1665 - 1670 .

KIM B , KANG C M , KIM J , et al . Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network [C ] // Proceedings of the 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC) . Piscataway : IEEE Press , 2017 : 399 - 404 .

LEE N , CHOI W , VERNAZA P , et al . DESIRE: distant future prediction in dynamic scenes with interacting agents [C ] // Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2017 : 2165 - 2174 .

蔡英凤 , 朱南楠 , 邰康盛 , 等 . 基于注意力机制的车辆行为预测 [J ] . 江苏大学学报(自然科学版) , 2020 , 41 ( 2 ): 125 - 130 .

CAI Y F , ZHU N N , TAI K S , et al . Vehicle behavior prediction based on attention mechanism [J ] . Journal of Jiangsu University (Natural Science Edition) , 2020 , 41 ( 2 ): 125 - 130 .

ALAHI A , GOEL K , RAMANATHAN V , et al . Social LSTM: human trajectory prediction in crowded spaces [C ] // Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2016 : 961 - 971 .

GUPTA A , JOHNSON J , LI F F , et al . Social GAN: socially acceptable trajectories with generative adversarial networks [C ] // Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2018 : 2255 - 2264 .

ZHANG P , OUYANG W L , ZHANG P F , et al . SR-LSTM: state refinement for LSTM towards pedestrian trajectory prediction [C ] // Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2019 : 12077 - 12086 .

DEO N , TRIVEDI M M . Multi-modal trajectory prediction of surrounding vehicles with maneuver based LSTMs [C ] // Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV) . Piscataway : IEEE Press , 2018 : 1179 - 1184 .

DEO N , RANGESH A , TRIVEDI M M . How would surround vehicles move? A unified framework for maneuver classification and motion prediction [J ] . IEEE Transactions on Intelligent Vehicles , 2018 , 3 ( 2 ): 129 - 140 .

DEO N , TRIVEDI M M . Convolutional social pooling for vehicle trajectory prediction [C ] // Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Piscataway : IEEE Press , 2018 : 1549 - 15498 .

IVANOVIC B , PAVONE M . The trajectron: probabilistic multi-agent trajectory modeling with dynamic spatiotemporal graphs [C ] // Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2019 : 2375 - 2384 .

ZHAO Z Y , FANG H W , JIN Z , et al . GISNet: graph-based information sharing network for vehicle trajectory prediction [C ] // Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN) . Piscataway : IEEE Press , 2020 : 1 - 7 .

LI X , YING X W , CHUAH M C . GRIP++: enhanced graph-based interaction-aware trajectory prediction for autonomous driving [J ] . arXiv Preprint , arXiv: 1907.07792 , 2019 .

XU D W , SHANG X T , LIU Y , et al . Group vehicle trajectory prediction with global spatio-temporal graph [J ] . IEEE Transactions on Intelligent Vehicles , 2023 , 8 ( 2 ): 1219 - 1229 .

XU D W , SHANG X T , PENG H , et al . MVHGN: multi-view adaptive hierarchical spatial graph convolution network based trajectory prediction for heterogeneous traffic-agents [J ] . IEEE Transactions on Intelligent Transportation Systems , 2023 , 24 ( 6 ): 6217 - 6226 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // Proceeding of the 31st International Conference on Neural Information Processing Systems . California : MIT Press , 2017 : 6000 - 6010 .

GIULIARI F , HASAN I , CRISTANI M , et al . Transformer networks for trajectory forecasting [C ] // Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR) . Piscataway : IEEE Press , 2021 : 10335 - 10342 .

YU C J , MA X , REN J W , et al . Spatio-temporal graph transformer networks for pedestrian trajectory prediction [C ] // Proceedings of the 16th European Conference on Computer Vision (ECCV 2020) . Berlin : Springer , 2020 : 507 - 523 .

LIU Y C , ZHANG J H , FANG L J , et al . Multimodal motion prediction with stacked transformers [C ] // Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2021 : 7573 - 7582 .

NGIAM J , CAINE B , VASUDEVAN V , et al . Scene Transformer: a unified architecture for predicting multiple agent trajectories [J ] . arXiv Preprint , arXiv: 2106.08417 , 2021 .

景荣荣 , 吴兰 , 张坤鹏 . 基于Transformer的自动驾驶交互感知轨迹预测 [J ] . 科学技术与工程 , 2023 , 23 ( 26 ): 11414 - 11423 .

JING R R , WU L , ZHANG K P . Transformer-based interaction-aware trajectory prediction for autonomous driving [J ] . Science Technology and Engineering , 2023 , 23 ( 26 ): 11414 - 11423 .

王庆荣 , 谭小泽 , 朱昌锋 , 等 . 基于门控循环单元和Transformer的车辆轨迹预测方法 [J ] . 汽车技术 , 2024 ( 7 ): 1 - 8 .

WANG Q R , TAN X Z , ZHU C F , et al . Vehicle trajectory prediction method based on GRU and transformer [J ] . Automobile Technology , 2024 ( 7 ): 1 - 8 .

KUEFLER A , MORTON J , WHEELER T , et al . Imitating driver behavior with generative adversarial networks [C ] // Proceedings of the 28th IEEE Intelligent Vehicles Symposium (IV) . Piscataway : IEEE Press , 2017 : 204 - 211 .

Views

1227

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Pedestrian trajectory prediction method based on group perception

Review of LLM safety research on autonomous driving under vehicle-road-cloud collaboration

Modulation recognition method based on multiscale convolutional fusion coding networks

Adaptive method runtime safety assurance for autonomous driving in dynamic scenarios

Research on university email analysis based on SVM-RFE and Transformer-TBAM

Related Author

CUI Yaping

DUAN Ang

WU Dapeng

ZHOU Yudie

WANG Ruyan

HE Peng

CHENG Ke

FENG Xia

Related Institution

Chongqing Key Laboratory of Ubiquitous Sensing and Networking

Advanced Network and Intelligent Connection Technology Key Laboratory of Chongqing Education Commission of China

School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications

School of Computer Science and Technology, Xidian University

School of Computing, National University of Singapore

AI问答

⁰