Generating Autonomous Vehicles Neural Reconstruction With Open-Source Physical AI Agent Skills
TL;DR · AI Summary
Leverage NVIDIA DRIVE to deploy a neural reconstruction launchable with agent skills that automate the full end-to-end pipeline from data ingestion to rendering, eliminating the need for local setup and configurations.
Key Takeaways
- Create and open the launchable in DRIVE cloud in one click, with all agent skill
- Use prompts like re-rendering from a different sensor and compare original vs ne
- Scale with Osmo and Azure to process thousands of clips, pulling raw sensor data
Outline
Jump quickly between sections.
Introduces the integration of NVIDIA DRIVE and Neural Reconstruction for AV Physical AI development workflows.
Explains how Nurek reconstructs sensor data into 3D scenes, and Agent Skills automate end-to-end pipelines.
Navigate to drive.nvidia.com to create the launchable, spin up a preconfigured environment, and access the Nemo Cloud interface.
Demo prompts to re-render clips from new sensor positions and generate side-by-side comparisons with video output.
Fetch raw sensor data and 3D assets from Hugging Face, rebuild USD scenes, and provide orbit renderings for comparison.
Scale using Osmo and Azure to handle thousands of clips, with NVIDIA running millions of simulations weekly for AV validation.
Mindmap
See how the topics connect at a glance.
查看大纲文本（无障碍 / 无 JS 友好）
- NVIDIA DRIVE Neural Reconstruction
- Agent Skills自动化工作流
- 端到端流程：摄入-重建-渲染-对比
- Nemo Flow接口，适配任意Agent
- 云端一键部署
- drive.nvidia.com创建Launchable
- 自动加载环境与凭证
- 进入Nemo Cloud聊天界面
- 典型Prompt与输出
- 重渲染并对比新视角视频
- 生成对比视频与Web查看器
- 支持地面真值侧并排对比
- 数据与资产来源
- 从Hugging Face获取原始数据
- 重建USDC场景并对比
- 从物理AI数据集harvest 3D资产
- 输出USD文件与轨道渲染
- 生产与扩展
- 每周数百万次模拟用于AV验证
- 在Azure用Osmo扩展至500 clips
- GitHub开源skills可自部署
Highlights
Key sentences worth saving and sharing.
Eliminate local setup: DRIVE spins up a fully preconfigured environment with all agent skills and credentials ready.
Agent Skills break tasks into step-by-step pipelines, executing from ingestion to rendering and providing real-time updates.
Prompt example: Render clip ID as if captured from a different sensor, outputting comparison videos and a web viewer link.
Production scale: NVIDIA runs millions of simulations per week to validate AV model development workflows.
Scaling with Osmo/Azure: Reconstruct 500 clips at scale, pull data from Hugging Face, and compare USD assets with ground truth.
GitHub source: Skills are available at github.com/nvidia/skills for self-hosted agent and infrastructure integration.