他们用ChatGPT方式搞自动驾驶，能告诉你“我在干嘛”

华尔街见闻2023-03-24

这是首个基于Transformer的驾驶行为描述框架，可以感知和预测驾驶行为。也就是说，输入车辆视频后，这个算法可以判断车辆行为并告诉你：车在做什么，为什么要这么做。还在玩ChatGPT？已经有自动驾驶算法能告诉你“我在干嘛”。基于视觉和神经网络的自动驾驶算法，虽然能通过传感器数据，以及学习人类的驾驶行为，自主决策并控制车辆。但是，算法基于什么做出的决策？特别是出现故障，也就是决策错误的时候，算法...

Source Link

Disclaimer: Investing carries risk. This is not financial advice. The above content should not be regarded as an offer, recommendation, or solicitation on acquiring or disposing of any financial products, any associated discussions, comments, or posts by author or other users should not be considered as such either. It is solely for general information purpose only, which does not consider your own investment objectives, financial situations or needs. TTM assumes no responsibility or warranty for the accuracy and completeness of the information, investors should do their own research and may seek professional advice before investing.

Comments

一个正常的名称
2023-03-25
一个正常的名称
Sksjsksks
一个正常的名称
2023-03-25
一个正常的名称
Zksksldk
一个正常的名称
2023-03-25
一个正常的名称
Djdjdjdj
一个正常的名称
2023-03-25
一个正常的名称
Gaosmm

Leave a comment

{"i18n":{"language":"en_US"},"isChannel":false,"data":{"share":"https://ttm.financial/m/news/2322314445?lang=en_US&edition=fundamental","thumbnail":"https://wpimg-wscn.awtmt.com/1f7e0c63-5923-4dff-8d60-ab2f7ca5d931.gif?imageView2/1/w/340/h/340","is_english":false,"pubTime":"2023-03-24 22:33","share_image_url":"https://static.laohu8.com/e9f99090a1c2ed51c021029395664489","id":"2322314445","market":"us","top_or_hot":-1,"title":"他们用ChatGPT方式搞自动驾驶，能告诉你“我在干嘛”","media":"华尔街见闻","content":"<div>\n<p>这是首个基于Transformer的驾驶行为描述框架，可以感知和预测驾驶行为。也就是说，输入车辆视频后，这个算法可以判断车辆行为并告诉你：车在做什么，为什么要这么做。还在玩ChatGPT？已经有自动驾驶算法能告诉你“我在干嘛”。基于视觉和神经网络的自动驾驶算法，虽然能通过传感器数据，以及学习人类的驾驶行为，自主决策并控制车辆。但是，算法基于什么做出的决策？特别是出现故障，也就是决策错误的时候，算法...</p>\n\n<a href=\"https://wallstreetcn.com/articles/3684853\">Source Link</a>\n\n</div>\n","source":"wallstreetcn_api","html":"<!DOCTYPE html>\n<html>\n<head>\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\" />\n<meta name=\"viewport\" content=\"width=device-width,initial-scale=1.0,minimum-scale=1.0,maximum-scale=1.0,user-scalable=no\"/>\n<meta name=\"format-detection\" content=\"telephone=no,email=no,address=no\" />\n<title>他们用ChatGPT方式搞自动驾驶，能告诉你“我在干嘛”</title>\n<style type=\"text/css\">\na,abbr,acronym,address,applet,article,aside,audio,b,big,blockquote,body,canvas,caption,center,cite,code,dd,del,details,dfn,div,dl,dt,\nem,embed,fieldset,figcaption,figure,footer,form,h1,h2,h3,h4,h5,h6,header,hgroup,html,i,iframe,img,ins,kbd,label,legend,li,mark,menu,nav,\nobject,ol,output,p,pre,q,ruby,s,samp,section,small,span,strike,strong,sub,summary,sup,table,tbody,td,tfoot,th,thead,time,tr,tt,u,ul,var,video{ font:inherit;margin:0;padding:0;vertical-align:baseline;border:0 }\nbody{ font-size:16px; line-height:1.5; color:#999; background:transparent; }\n.wrapper{ overflow:hidden;word-break:break-all;padding:10px; }\nh1,h2{ font-weight:normal; line-height:1.35; margin-bottom:.6em; }\nh3,h4,h5,h6{ line-height:1.35; margin-bottom:1em; }\nh1{ font-size:24px; }\nh2{ font-size:20px; }\nh3{ font-size:18px; }\nh4{ font-size:16px; }\nh5{ font-size:14px; }\nh6{ font-size:12px; }\np,ul,ol,blockquote,dl,table{ margin:1.2em 0; }\nul,ol{ margin-left:2em; }\nul{ list-style:disc; }\nol{ list-style:decimal; }\nli,li p{ margin:10px 0;}\nimg{ max-width:100%;display:block;margin:0 auto 1em; }\nblockquote{ color:#B5B2B1; border-left:3px solid #aaa; padding:1em; }\nstrong,b{font-weight:bold;}\nem,i{font-style:italic;}\ntable{ width:100%;border-collapse:collapse;border-spacing:1px;margin:1em 0;font-size:.9em; }\nth,td{ padding:5px;text-align:left;border:1px solid #aaa; }\nth{ font-weight:bold;background:#5d5d5d; }\n.symbol-link{font-weight:bold;}\n/* header{ border-bottom:1px solid #494756; } */\n.title{ margin:0 0 8px;line-height:1.3;color:#ddd; }\n.meta {color:#5e5c6d;font-size:13px;margin:0 0 .5em; }\na{text-decoration:none; color:#2a4b87;}\n.meta .head { display: inline-block; overflow: hidden}\n.head .h-thumb { width: 30px; height: 30px; margin: 0; padding: 0; border-radius: 50%; float: left;}\n.head .h-content { margin: 0; padding: 0 0 0 9px; float: left;}\n.head .h-name {font-size: 13px; color: #eee; margin: 0;}\n.head .h-time {font-size: 11px; color: #7E829C; margin: 0;line-height: 11px;}\n.small {font-size: 12.5px; display: inline-block; transform: scale(0.9); -webkit-transform: scale(0.9); transform-origin: left; -webkit-transform-origin: left;}\n.smaller {font-size: 12.5px; display: inline-block; transform: scale(0.8); -webkit-transform: scale(0.8); transform-origin: left; -webkit-transform-origin: left;}\n.bt-text {font-size: 12px;margin: 1.5em 0 0 0}\n.bt-text p {margin: 0}\n</style>\n</head>\n<body>\n<div class=\"wrapper\">\n<header>\n<h2 class=\"title\">\n他们用ChatGPT方式搞自动驾驶，能告诉你“我在干嘛”\n</h2>\n\n<h4 class=\"meta\">\n\n\n2023-03-24 22:33 北京时间&nbsp;&nbsp;&nbsp;<a href=https://wallstreetcn.com/articles/3684853><strong>华尔街见闻</strong></a>\n\n\n</h4>\n\n</header>\n<article>\n<div>\n<p>这是首个基于Transformer的驾驶行为描述框架，可以感知和预测驾驶行为。也就是说，输入车辆视频后，这个算法可以判断车辆行为并告诉你：车在做什么，为什么要这么做。还在玩ChatGPT？已经有自动驾驶算法能告诉你“我在干嘛”。基于视觉和神经网络的自动驾驶算法，虽然能通过传感器数据，以及学习人类的驾驶行为，自主决策并控制车辆。但是，算法基于什么做出的决策？特别是出现故障，也就是决策错误的时候，算法...</p>\n\n<a href=\"https://wallstreetcn.com/articles/3684853\">Source Link</a>\n\n</div>\n\n\n</article>\n</div>\n</body>\n</html>\n","isBrief":false,"type":0,"news_type":1,"symbol":null,"symbol_name":null,"start_time":0,"source_url":"https://wallstreetcn.com/articles/3684853","article_id":"2322314445","we_media_id":null,"thumbnails":["https://wpimg-wscn.awtmt.com/1f7e0c63-5923-4dff-8d60-ab2f7ca5d931.gif?imageView2/1/w/340/h/340"],"rights":{"source":"wallstreetcn_api","url":"https://wallstreetcn.com/articles/3684853","rn_cache_url":null,"directOrigin":true},"url":"https://stock-news.laohu8.com/highlight/detail?id=2322314445","pubTimestamp":1679668428,"sourceInfo":{"source_id":"wallstreetcn_api","name":"华尔街见闻"},"weMediaInfo":null,"summary":"这是首个基于Transformer的驾驶行为描述框架，可以感知和预测驾驶行为。也就是说，输入车辆视频后，这个算法可以判断车辆行为并告诉你：车在做什么，为什么要这么做。","collect":0,"end_time":0,"defaultTopTitle":"wallstreetcn.com","property":[],"viewcount":null,"language":"zh","relate_stocks":{},"translate_title":"They use ChatGPT to do automatic driving and can tell you \"What am I doing?\"","themeId":"","isJumpTheme":false,"ttsUrl":"https://static.tigerbbs.com/7760422ad06b388c60e8166487fc596c","symbols_score_info":{},"content_text":"这是首个基于Transformer的驾驶行为描述框架，可以感知和预测驾驶行为。也就是说，输入车辆视频后，这个算法可以判断车辆行为并告诉你：车在做什么，为什么要这么做。还在玩ChatGPT？已经有自动驾驶算法能告诉你“我在干嘛”。基于视觉和神经网络的自动驾驶算法，虽然能通过传感器数据，以及学习人类的驾驶行为，自主决策并控制车辆。但是，算法基于什么做出的决策？特别是出现故障，也就是决策错误的时候，算法是怎么想的？这些一直被称为自动驾驶算法里的“黑匣子”，让算法缺乏透明度和可解释性。不过，现在有这么一个模型，既能预测车辆控制行为，还能自己解释“我停车是因为红灯亮了，并且有行人在过马路”。模型论文入选ICRA 2023，相关模型已开源。那么，是一个什么样的算法？ADAPT：驾驶行为感知说明大模型这是一种叫ADAPT（Action-aware Driving Caption Transformer）的端到端算法，也是目前第一个基于Transformer的驾驶行为描述框架，可以感知和预测驾驶行为，并且输出自然语言叙述和推理。直白一点说，输入车辆视频后，这个算法可以判断车辆行为并告诉你：车在做什么，为什么要这么做。在论文作者提供的测试视频里，这个算法最终上车的效果是这样的。（红色字是车辆行为，蓝色字是解释）“车在向前开。因为路上没有车。”驾驶行为变化后，算法也能及时感知：“车靠左边停下了。因为要停车。”“车开始移动并且靠右行驶。因为路左边停着车。”算法不仅能识别路口，也能识别骑着车的人。“车在十字路口停下了。因为要避开街上骑着自行车的人，”这是怎么实现的？多任务框架下的联合训练ADAPT框架可以分为两个部分：车辆行为描述（DCG，Driving Caption Generation）和车辆控制信号预测（CSP，Control Signal Prediction）。首先，传感器端输入视频，Video Swin Transformer对车辆视频进行编码，得到的视频特征会输入进各任务模块里。在DCG模块，算法利用Vision-Language Transformer生成两个自然语句，也就是上文中提到的车辆行为描述和原因解释。相同的视频特征也会输入进CSP模块（类似一般基于视觉的自动驾驶系统），输出车辆实际的控制信号序列，并利用Motion Transformer输出模型预测的控制信号，比如速度、方向和加速度。在单个网络中，作者利用车辆实际的控制信号序列和模型预测的控制信号序列，两者的均方误差作为CSP模块的损失函数。而在多任务框架下，通过联合训练DCG和CSP，可以减少车辆决策和文本描述之间的差异，提高控制信号预测的准确率。论文里，作者们在包含控制信号和车辆视频的大规模数据集BDD-X上，利用机器评测和人工评测验证了ADAPT的有效性。机器评测方面，使用的是BLEU4、METEOR、ROUGE-L和CIDEr（对应缩写分别为B4、M、R、C）等多种语言任务常用的指标。最终显示ADAPT达到了当前最优（State-of-the-Art）的结果，ADAPT在动作描述方面比原有先进方法CIDEr高出31.7，在原因解释方面高33.1。人工评测分为动作描述、原因解释和全句三个部分。通过人工判断，ADAPT在这三部分的准确性分别达到了90%，90.3%和82.7%，证明了ADAPT的有效性。在可视化结果里，也能看出ADAPT可以准确识别车辆行为以及决策原因。并且在黑夜、阴雨天等场景下，ADAPT也能保证准确度；即使有雨刷器干扰，ADAPT也可以识别道路上的停止标识。为什么需要ADAPT？自动驾驶行为的可解释性在基于视觉的自动驾驶算法里，比较常见的解释图有视觉注意图（Attention Map），或者成本量图（Cost Volume），但不熟悉自动驾驶算法的人容易对这些图造成误解。上：视觉注意图；下：成本量图因此，ADAPT这种能够生成自然语言、“说人话”的算法，能够帮助用户更好地理解自动驾驶算法在做什么、为什么要这么做，同时还能让用户更信任自动驾驶技术。而对于算法工程师和研究人员来说，当发生极端情况时、或者发生故障（比如判断错误）时，ADAPT可以帮助他们获得更多信息，进而改进算法。作者们将进一步研究如何在模拟器和实际车辆上如何部署ADAPT，以及如何利用文本转语音技术，让生成的句子转化为语音，帮助普通乘客，特别是视力障碍乘客使用。","kind":"highlight","is_publish_news":false,"is_publish_highlight":true,"is_publish_live":false,"is_publish_wemedia":null,"editions":null,"column":"","sentiment":"0","news_tag":"","news_rank":0,"symbols":[],"gpt_button":1},"commentList":[{"id":9943786650,"gmtCreate":1679721300170,"gmtModify":1679721303091,"author":{"id":"4142746771171822","authorId":"4142746771171822","name":"一个正常的名称","avatar":"https://community-static.tradeup.com/news/d508b8060163e453a31dfa85a77c4043","vip":1,"crmLevel":1,"crmLevelSwitch":0,"authorIdStr":"4142746771171822","idStr":"4142746771171822"},"htmlText":"Sksjsksks","listText":"Sksjsksks","text":"Sksjsksks","images":[],"top":1,"highlighted":1,"essential":1,"paper":1,"likeSize":0,"commentSize":0,"repostSize":0,"link":"https://ttm.financial/post/9943786650","repostId":2322314445,"repostType":4,"isVote":1,"likeStatus":false,"favoriteStatus":false,"reportStatus":false,"tweetType":1,"langContent":"EN"},{"id":9943786112,"gmtCreate":1679721288436,"gmtModify":1679721291246,"author":{"id":"4142746771171822","authorId":"4142746771171822","name":"一个正常的名称","avatar":"https://community-static.tradeup.com/news/d508b8060163e453a31dfa85a77c4043","vip":1,"crmLevel":1,"crmLevelSwitch":0,"authorIdStr":"4142746771171822","idStr":"4142746771171822"},"htmlText":"Zksksldk","listText":"Zksksldk","text":"Zksksldk","images":[],"top":1,"highlighted":1,"essential":1,"paper":1,"likeSize":0,"commentSize":0,"repostSize":0,"link":"https://ttm.financial/post/9943786112","repostId":2322314445,"repostType":4,"isVote":1,"likeStatus":false,"favoriteStatus":false,"reportStatus":false,"tweetType":1,"langContent":"EN"},{"id":9943786389,"gmtCreate":1679721282742,"gmtModify":1679721286874,"author":{"id":"4142746771171822","authorId":"4142746771171822","name":"一个正常的名称","avatar":"https://community-static.tradeup.com/news/d508b8060163e453a31dfa85a77c4043","vip":1,"crmLevel":1,"crmLevelSwitch":0,"authorIdStr":"4142746771171822","idStr":"4142746771171822"},"htmlText":"Djdjdjdj","listText":"Djdjdjdj","text":"Djdjdjdj","images":[],"top":1,"highlighted":1,"essential":1,"paper":1,"likeSize":0,"commentSize":0,"repostSize":0,"link":"https://ttm.financial/post/9943786389","repostId":2322314445,"repostType":4,"isVote":1,"likeStatus":false,"favoriteStatus":false,"reportStatus":false,"tweetType":1,"langContent":"EN"},{"id":9943786975,"gmtCreate":1679721275897,"gmtModify":1679721280532,"author":{"id":"4142746771171822","authorId":"4142746771171822","name":"一个正常的名称","avatar":"https://community-static.tradeup.com/news/d508b8060163e453a31dfa85a77c4043","vip":1,"crmLevel":1,"crmLevelSwitch":0,"authorIdStr":"4142746771171822","idStr":"4142746771171822"},"htmlText":"Gaosmm","listText":"Gaosmm","text":"Gaosmm","images":[],"top":1,"highlighted":1,"essential":1,"paper":1,"likeSize":0,"commentSize":0,"repostSize":0,"link":"https://ttm.financial/post/9943786975","repostId":2322314445,"repostType":4,"isVote":1,"likeStatus":false,"favoriteStatus":false,"reportStatus":false,"tweetType":1,"langContent":"EN"}],"isCommentEnd":false,"newsSizeData":{"likeSize":2,"commentSize":4,"repostSize":0,"favoriteSize":0,"likeStatus":false,"favoriteStatus":false},"APP":{"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","isDev":false,"isTTM":true,"deviceId":"web-server-community-laohu8-v3","version":"4.29.3","shortVersion":"4.29.3","platform":"web","vendor":"web","appName":"ttm","isIOS":false,"isAndroid":false,"isTiger":false,"isTHS":false,"isWeiXin":false,"isWeiXinMini":false,"isWeiBo":false,"isQQ":false,"isBaiduSwan":false,"isBaiduBox":false,"isDingTalk":false,"isToutiao":false,"isOnePlus":false,"isHuaWei":false,"isXiaomi":false,"isXiaomiWebView":false,"isOppo":false,"isVivo":false,"isSamsung":false,"isMobile":false},"href":"/m/news/2322314445"}