v1sea

Coming down with the sickness again, maybe there are codebase visualizations that are useful. Interactive with bidirectional editor integration and runtime analysis. Apologies for the slop prototype.

The VGGT vision transformer model couldn't really handle mixed time of day images, but this new Apache-2 licensed model is a big step up. Variable lighting and weather really mess with existing VPS solutions. A popular one needs two scans at least 5 hours apart. map-anything.github.io

The VGGT model works as a VPS with only a few images. One attention head estimates the 6dof camera pose of each image. Here is a rough example with 7 warped Mapillary images and 1 XR headset camera image in red. While the error is about 2m, you couldn't do this at all with SfM.

This is the headset image.

I got some very promising 3d sidewalk point cloud data from unwarping Mapillary data and feeding it to VGGT(Visual Geometry Grounded Transformer). The output seems much better than traditional Structure from Motion + Multi-View Stereo dense reconstruction.

Demo: while you were living in sin I was studying the GNSS.

Demo: while you were living in sin I was studying the GNSS.

Alleyway GPS sensor fusion test. Head position in white. GPS in red. Fused state in green. It is tuned for open fields and forests, but maybe with a bit more math this could hit sub meter accuracy. At this scale the odometry is very accurate.

So you want to build Mountain-Scale AR for current hardware? Maybe like Snap's Spectacles devkit does it now? For vision pro you can do this by reading the headset position(odometry) and a iPhone gps position.

There are companies working on building a 3d map of the world for this, most are hopeless (like MeshMap), but a few have a shot by building the most comprehensive surveillance machine on the planet. The only other use case is for military C2, but I will not jingopost on main.

So you want to build Mountain-Scale AR for current hardware? Maybe like Snap's Spectacles devkit does it now? For vision pro you can do this by reading the headset position(odometry) and a iPhone gps position.

This gets to around 2m accuracy, but to get better you have to combine this with data of the local environment. Hence why I was running mesh registration via ISS + FPFH + TEASER++ on device.

So you want to build Mountain-Scale AR for current hardware? Maybe like Snap's Spectacles devkit does it now? For vision pro you can do this by reading the headset position(odometry) and a iPhone gps position.

The filter also provides variance estimates you can use for tracking quality levels, something Snap didn't expose to users for their north heading component at launch. I found it easiest to pass this all through a state machine.

So you want to build Mountain-Scale AR for current hardware? Maybe like Snap's Spectacles devkit does it now? For vision pro you can do this by reading the headset position(odometry) and a iPhone gps position.

We can unwarp this and get the north heading by estimating a four component state with an Augmented Unscented Kalman Filter. We can't use a basic Kalman Filter because the north heading state introduces non-linearity to our model.

So you want to build Mountain-Scale AR for current hardware? Maybe like Snap's Spectacles devkit does it now? For vision pro you can do this by reading the headset position(odometry) and a iPhone gps position.

The issues are we don't know what way is north, the odometry drifts by say 100m every 1-2km, and the odometry origin 'recenters' if we walk ~0.3km.

I was surprised to find NOAA publishes the 3d weather forecast data for the continental US + Alaska. This appears to match observations of nearby mountains. Very little smoke today. I get why hiking apps don't provide search on this data now :), but there might be a way.

Would anyone actually try a free outdoor navigation app for the vision pro? The tech works in the mountains, but it is too dangerous to recommend outside flat parks. My initial tests were putting a 10m tall pushpin at a gps coordinate and keeping it there for miles.

There is even a nice article about how this is dangerous ❤️ www.cnet.com/tech/computi...

Demo showing Claude building a race car with voxels. The UI is all buttoned up now, hopefully app review approves this feature.

Demo showing Claude building a race car with voxels. The UI is all buttoned up now, hopefully app review approves this feature.

Today was such a beautiful summer day for a hike according to my database.