Huy Ha (@haqhuy) / X

Huy Ha

54 posts

Huy Ha

@haqhuy

PhD @stanford @columbia I train dogs and robots

Palo Alto, CA

Joined October 2022

Pinned
Huy Ha
@haqhuy
Jul 15, 2024
I’ve been training dogs since middle school. It’s about time I train robot dogs too 😛 Introducing, UMI on Legs, an approach for scaling manipulation skills on robot dogs🐶It can toss, push heavy weights, and make your ~existing~ visuo-motor policies mobile!
00:00
Huy Ha
@haqhuy
Jul 28, 2023
How can we put robotics on the same scaling trend as large language models while not compromising on rich low-level manipulation and control?
00:00
Huy Ha
@haqhuy
Jul 15, 2024
Replying to @haqhuy
For stability, we track gripper actions in world frame instead of body frame, like most prior works. So, when our robot is pushed, its arm will compensate the perturbation. To support real world deployment, we mount an iPhone on the dog’s 🍑, running a custom iOS ARKit app.
00:00
Huy Ha
@haqhuy
Jul 15, 2024
Replying to @haqhuy @YihuaiGao and 2 others
To learn more, check out our video, website, paper, and code! 📽️: youtu.be/4Bp0q3xHTxE 🌍: umi-on-legs.github.io 📃: umi-on-legs.github.io/static/umi-on-… 💻:github.com/real-stanford/…
Huy Ha
@haqhuy
Jul 15, 2024
Replying to @haqhuy
UMI on Legs starts with UMI, a handheld gripper with a camera, that can be used to collects robot data without robots. From this data, we train manipulation policies which predicts gripper actions from image inputs. But how should the legs move to track those gripper actions?🤔
00:00
Huy Ha
@haqhuy
Jul 15, 2024
Replying to @haqhuy
For precision, we give the controller a trajectory of targets into the future, which allows anticipation of upcoming motion for dynamic trajectory tracking. For instance, the controller will brace before a fast toss, and mobilize all its legs during a push.
00:00
Huy Ha
@haqhuy
Jul 28, 2023
Replying to @haqhuy
To find out more check out our paper! Arxiv: arxiv.org/abs/2307.14535 Website: cs.columbia.edu/~huy/scalingup/ Code: github.com/columbia-ai-ro… Or, just find out how we render the visualizations 🙃 github.com/columbia-ai-ro…
github.com
GitHub - real-stanford/scalingup: [CoRL 2023] This repository contains data generation and training...
[CoRL 2023] This repository contains data generation and training code for Scaling Up & Distilling Down - real-stanford/scalingup
Huy Ha
@haqhuy
Jul 15, 2024
Replying to @haqhuy
A giant shoutout to my co-first author, @YihuaiGao , whose real world robot engineering wizardy and optimism are inspiring 🚀 Special thanks to @zipengfu and Jie Tan for their RL, control, and quadruped wisdom 🐕 Last but not least, thank you to my advisor, @SongShuran
00:00
Huy Ha
@haqhuy
Jul 15, 2024
Replying to @haqhuy
In a massively parallelized simulation, the robot learns through trial and error how to track UMI trajectories with stability and precision.
00:00
Huy Ha
@haqhuy
Jul 28, 2023
Replying to @haqhuy
Our framework, scaling up and distilling down, provides a comprehensive solution by leveraging language guidance. We scale up automatic language and success labeled robot data generation and distill this data down to an expressive language conditioned multi task policy.
Huy Ha
@haqhuy
Jul 28, 2023
Replying to @haqhuy
Using no human demonstrations, language annotation, and reward specification, our framework outputs robust visuo motor policies which can be used for real world deployment.
00:00
Huy Ha
@haqhuy
Jul 15, 2024
Replying to @ARXrobotics
Thank you so much for your amazing hardware! Wouldn't have been possible if we hadn't switched to ARX5 arms 🦾🦾🦾
Huy Ha
@haqhuy
Jul 18, 2024
Replying to @sidharthtalia
We wanted to use odometry provided by Unitree. However, they only enable odometry during joint control mode. Further, we found that the lidar spinning can cause vibrations and shakes on the arm 🤦This is why we disconnect the lidar entirely during all of our experiments