Niladri Shekhar Dutt

🧵1/10 Excited to share our #SIGGRAPH paper "MonetGPT: Solving Puzzles Enhances MLLMs' Image Retouching Skills" 🌟 We explore how to make MLLMs operation-aware by solving visual puzzles and propose a procedural framework for image retouching #MLLM

🧵9/10 We quantitaively evaluate on the Adobe5k dataset as well as conduct user studies by expert and novice users. Our evaluations show that MonetGPT outperforms open-source alternatives and performs comparably to Google Photos AutoEnhance (closed-source).

monetgpt.github.ioMonetGPT

🧵10/10 Lastly, huge thanks to my co-advisors Niloy and Duygu! For more details check out our paper below- 🌐 Project Website: monetgpt.github.io 📄 Arxiv: arxiv.org/abs/2505.06176

🧵7/10 Our puzzle-based training with a 'reasoning as a pathway' approach allows MonetGPT to generate detailed justifications for each edit, delivering truly explainable image retouching

🧵8/10 Photo editing is subjective 🎨. Our framework adapts to user preference by guidance from natural language tags like ‘vibrant’ or ‘retro vibe’ to produce personalized and stylistically distinct retouching plans from the same input image.

🧵5/10 🧩 Puzzle B imparts aesthetic judgement. By ranking professionally edited photos against altered versions, the MLLM learns to recognize the visual characteristics of an optimally adjusted image for any given operation, building an internal aesthetic model.

🧵6/10 🧩 Puzzle C builds planning capabilities. The model learns to generate a complete, multi-step retouching plan to enhance a photo, structuring its reasoning as a sequence of discrete issues and solutions for clarity and control.

🧵3/10 Our key recipe: MLLMs struggle to predict edit values directly. We solve this by generating rich textual reasoning for each puzzle ✍️. We then fine-tune MonetGPT on this data, creating a 'reasoning pathway' that enables it to regress final adjustment values.

🧵4/10 🧩 Puzzle A builds an understanding of individual operations. The MLLM learns to map visual changes in before/after images to a specific tool and its precise parameter value, effectively learning the semantics of our procedural library.

🧵2/10 MLLMs lack the visual understanding to plan edits. 🧠 So, we use expert photos as our ground truth and work backward, procedurally creating puzzles by assuming any change to an expert edit makes it less optimal

10 Apr 2025

Hi Londoners Join us on April 15 for an evening on Gen AI for 3D at UCL! We have an amazing list of keynote speakers and lightning talks. Register at londongenai.github.io Very excited to co-organize this with Michael and Preddy!

Niladri Shekhar Dutt reposted

Ruchira 🏳️‍🌈

@rayruchira.bsky.social

17 Jan 2025

🤔What tasks do we want robots to handle? Are these preferences based on saved time or feelings we associate with the tasks? Introducing Why Automate This?—a study exploring automation preferences across social groups, using feelings & time-spent as key factors. 👇 (1/5)

25 Nov 2024

All the papers I've reviewed still have some reviewers who haven't participated in the discussion/replied to the rebuttal at all. This is true even after the authors have nudged the reviewers a few times already :(

ICLR Conference@iclr-conf.bsky.social

✍️ Reminder to reviewers: Check author responses to your reviews, and ask follow up questions if needed. 50% of papers have discussion - let’s bring this number up!

Niladri Shekhar Dutt reposted

Kosta Derpanis

@csprofkgd.bsky.social

19 Nov 2024

My growing list of #computervision researchers on Bsky. Missed you? Let me know. go.bsky.app/M7HGC3Y

@timinclimate.naruresearch.com.au

19 Nov 2024

I've made a starter pack for researchers working in inverse graphics and 3D vision. Let me know if you would like to be added/ want others to be added (more people joining 🦋 everyday) bsky.app/starter-pack...

bsky.appbsky.app

Niladri Shekhar Dutt@niladridutt.bsky.social

Created a starter pack for researchers working in inverse graphics, 3D vision, and geometry processing. Would love your help to expand this list! go.bsky.app/9uEdjzb

Niladri Shekhar Dutt reposted

Tim Baxter

18 Nov 2024

Bluesky tip for newbies: Bluesky has no real algorithm. Likes here do nothing but show the poster that you enjoyed their content. That kindness is always going to be appreciated by creators, but if you want to help to boost something here, you have to repost it so others see it. 🙂

Niladri Shekhar Dutt reposted

Kosta Derpanis

@csprofkgd.bsky.social

19 Nov 2024

Future Bsky homes for our #computervision conferences! @cvprconference.bsky.social @iccv.bsky.social @eccv.bsky.social @wacvconference.bsky.social Stay tuned!

Niladri Shekhar Dutt reposted

Valentin Deschaintre

@vdeschaintre.bsky.social

18 Nov 2024

Converted my Graphics Research list to a starter pack (not sure what's the difference though). Let me know who we are missing here :) Here goes! go.bsky.app/ApQNTt2

18 Nov 2024

Created a starter pack for researchers working in inverse graphics, 3D vision, and geometry processing. Would love your help to expand this list! go.bsky.app/9uEdjzb