Dr. Mar Gonzalez-Franco

Over the last two years we have seen an acceleration on AI, but if AI is going to enable humans in their day to day tasks, it most probably will be via XR. The issue then is if a selection has real-world consequences, we will need great precision to interact.

Ego4D is an incredible anthropological dataset. With over 3600 hours of videos (5 months of human activities) in first person view with written narrations from all over the world ego4d-data.org/fig1.html. (Always reminds me of this movie by #IsabelCoixet youtu.be/bYI3A6WLMBU)

But how do we know if our data is any good? We asked 35k humans in Prolific about how correct and useful they found the suggested actions in reference to every video. Now we opensource the new dataset alongside the human ranks. github.com/google/parse... and all the data generation prompts.

Ego4D is an incredible anthropological dataset. With over 3600 hours of videos (5 months of human activities) in first person view with written narrations from all over the world ego4d-data.org/fig1.html. (Always reminds me of this movie by #IsabelCoixet youtu.be/bYI3A6WLMBU)

I love how well the work on bringing avatars to video conferences as an alternative to camera on/off is aging. Check out our work on avatars via virtual cameras on regular conferences on our paper AllTogether. www.microsoft.com/en-us/resear...

Ever wondered why is so hard to write in VR/XR? We too 😅. Check this new work to appear in ACM CHI: xrtexttrove.github.io We (@arpitbhatia.bsky.social et al) analyzed 176 papers and text entry techniques inside XR and found that:

More over... we put all of this in an interactive dataset for people to explore each of the techniques and geek around😎 xrtexttrove.github.io some are wild!

Ever wondered why is so hard to write in VR/XR? We too 😅. Check this new work to appear in ACM CHI: xrtexttrove.github.io We (@arpitbhatia.bsky.social et al) analyzed 176 papers and text entry techniques inside XR and found that:

I personally think the solution will in part become a motion retargeting problem as soon as we can provide good haptic feedback. For as long as the human has to move fingers from point A to point B, we will have physical limitations to typing…

Ever wondered why is so hard to write in VR/XR? We too 😅. Check this new work to appear in ACM CHI: xrtexttrove.github.io We (@arpitbhatia.bsky.social et al) analyzed 176 papers and text entry techniques inside XR and found that:

2) external surfaces and fingertips/transparency of hands seem to be preferable as ways to improve performance compared to midair gestures.

Ever wondered why is so hard to write in VR/XR? We too 😅. Check this new work to appear in ACM CHI: xrtexttrove.github.io We (@arpitbhatia.bsky.social et al) analyzed 176 papers and text entry techniques inside XR and found that:

Ok! I have been using bluesky now for a bit. Unlike Mastodon, I really think it’s properly good and with enough critical mass and I am finally moving out of X. Moving forward: find me at @twimar.bsky.social

At @IEEEVR we present “EmBARDiment: an Embodied AI Agent for Productivity in XR" by @riccardobovoHCI et al. showing how powerful AI agents can be when they are aware of the content around us, driven by contextual inputs like gaze. Web: http://embardiment.github.io Code: soon

https://duruofei.com/papers/Zhu_BeyondThePhone-ExploringContext-AwareInteractionBetweenMobileAndMixedRealityDevices_VR2025.pdf

EmBARDiment explores: - how much we can simplify LLM prompts if the AI agent shares context and content understanding with the user (hint: a lot!). - highly homogeneous vs heterogeneous content. - long vs short contextual memory.

EmBARDiment explores: - how much we can simplify LLM prompts if the AI agent shares context and content understanding with the user (hint: a lot!). - highly homogeneous vs heterogeneous content. - Long vs short contextual memory.

At @IEEEVR we present “EmBARDiment: an Embodied AI Agent for Productivity in XR" by @riccardobovoHCI et al. showing how powerful AI agents can be when they are aware of the content around us, driven by contextual inputs like gaze. Web: http://embardiment.github.io Code: soon

Next week at #IEEEVR we present “EmBARDiment: an Embodied AI Agent for Productivity in XR" by @riccardobovo.bsky.social et al. showing how powerful AI agents can be when they are aware of the content around us, driven by contextual inputs like gaze. Web: embardiment.github.io Code: coming soon

Super proud of our collaboration with academia on advancing inputs for XR receiving a recognition by @acm-sigchi.bsky.social. 🙌 @ejgonz.bsky.social, @kenpfeuffer.bsky.social, @hansgellersen.bsky.social, Hasti, Karan, Max, Andrea