Tom Silver

Tom Silver

@tomssilver.bsky.social

Assistant Professor @Princeton. Developing robots that plan and learn to help people. https://tomsilver.github.io/

481 followers85 following119 posts

Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy" (Allen et al., NeurIPS 2024). I appreciate the fresh perspective on the classic RL paradigm. When two TD estimates disagree, you need memory! PDF: arxiv.org/abs/2407.07333

arxiv.orgMitigating Partial Observability in Sequential Decision Processes via the Lambda DiscrepancyReinforcement learning algorithms typically rely on the assumption that the environment dynamics and value function can be expressed in terms of a Markovian state representation. However, when state i...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Latent Programming Horizons in Coding Agents" (Silva et al., 2026). Now seems like a good time to better understand what coding agents are doing. Here's a nice example of the kind of analysis one can do. PDF: arxiv.org/abs/2607.05188

arxiv.orgLatent Programming Horizons in Coding AgentsA coding agent solving a software-engineering task spends dozens of steps reasoning, editing code, and running tests, yet little is known about what the underlying language model internally represents...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Safe Model-based Reinforcement Learning with Stability Guarantees" (Berkenkamp et al., NeurIPS 2017). Some safe RL guarantees the learned policy is safe; this guarantees safety *during* learning. Important for RL in real! PDF: arxiv.org/abs/1705.08551

arxiv.orgSafe Model-based Reinforcement Learning with Stability GuaranteesReinforcement learning is a powerful paradigm for learning optimal policies from experimental data. However, to find optimal policies, most reinforcement learning algorithms explore all possible actio...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Deliberate Practice: Learning Robot Skills under a Budget" (Vats et al., 2026). Practice your robot skills in a provably optimal way. Big fan of this whole line; see also arxiv.org/abs/2209.13605 & arxiv.org/abs/2505.00490 PDF: arxiv.org/abs/2608.13415

arxiv.orgDeliberate Practice: Learning Robot Skills under a BudgetWe consider the problem of autonomously learning robot skills under a limited practice budget for sequential tasks. We propose an active skill learning algorithm, \emph{Deliberate Practice (DP)}, that...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Sleep-time Compute: Beyond Inference Scaling at Test-time" (Lin et al., 2025). I often wonder what robots should do while they're asleep. One answer: use context to think ahead about likely requests before they arrive. PDF: arxiv.org/abs/2504.13171

arxiv.orgSleep-time Compute: Beyond Inference Scaling at Test-timeScaling test-time compute has emerged as a key ingredient for enabling large language models (LLMs) to solve difficult problems, but comes with high latency and inference cost. We introduce sleep-time...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "People construct simplified mental representations to plan" (Ho et al., Nature 2022). People generate ("construe") simplified task representations on-the-fly. (Robots should too!) Also: awesome first paragraph. PDF: arxiv.org/abs/2105.06948

arxiv.orgPeople construct simplified mental representations to planOne of the most striking features of human cognition is the capacity to plan. Two aspects of human planning stand out: its efficiency and flexibility. Efficiency is especially impressive because plans...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Algorithm Runtime Prediction" (Hutter et al., 2014). "I don't know the answer, but I know I will know the answer in about 10 minutes" (e.g., to a SAT question). Pretty wild that this is possible! PDF: arxiv.org/abs/1211.0906

arxiv.orgAlgorithm Runtime Prediction: Methods & EvaluationPerhaps surprisingly, it is possible to predict how long an algorithm will take to run on a previously unseen input, using machine learning techniques to build a model of the algorithm's runtime as a ...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "A micro Lie theory for state estimation in robotics" (Solà, Deray, & Atchuthan, 2018). Pay the small tax of learning a little Lie theory -- it clarifies many robotics concepts (and much code!). PDF: arxiv.org/abs/1812.01537

arxiv.orgA micro Lie theory for state estimation in roboticsA Lie group is an old mathematical abstract object dating back to the XIX century, when mathematician Sophus Lie laid the foundations of the theory of continuous transformation groups. As it often hap...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "General agents contain world models" (Richens et al., ICML 2025). Brought to my attention by @dabelcs.bsky.social 's beautiful philosophy-laden talk at ICAPS. The explicit construction of a transition model from a policy is cool (Alg 1). PDF: arxiv.org/abs/2506.01622

arxiv.orgGeneral agents contain world modelsAre world models a necessary ingredient for flexible, goal-directed behaviour, or is model-free learning sufficient? We provide a formal answer to this question, showing that any agent capable of gene...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Robust Planning for Multi-stage Forceful Manipulation" (Holladay et al., IJRR 2023). Sec. 6 is the clearest self-contained intro to PDDLStream I've seen. Also: robots with knives and child-proof bottles! (Very impressive) PDF: arxiv.org/abs/2208.00319

arxiv.orgRobust Planning for Multi-stage Forceful ManipulationMulti-step forceful manipulation tasks, such as opening a push-and-twist childproof bottle, require a robot to make various planning choices that are substantially impacted by the requirement to exert...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?" (Li, Kim, & Wang, NeurIPS 2025). Beautiful paper & increasingly important as agents start to ask more Qs. Curious how SOTA models do. PDF: arxiv.org/abs/2503.22674

arxiv.orgQuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?Large language models (LLMs) have shown impressive performance on reasoning benchmarks like math and logic. While many works have largely assumed well-defined tasks, real-world queries are often under...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundary" (Asai & Fukunaga, AAAI 2018). One of the key papers that got me hooked on learning + planning before I started grad school. PDF: arxiv.org/abs/1705.00154

arxiv.orgClassical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic BoundaryCurrent domain-independent, classical planners require symbolic models of the problem domain and instance as input, resulting in a knowledge acquisition bottleneck. Meanwhile, although deep learning h...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Human-Guided Complexity-Controlled Abstractions" (Peng et al., NeurIPS 2023). Selecting the right levels and kinds of abstractions remains important and open for many forms of human-AI / human-robot interaction. PDF: arxiv.org/abs/2310.17550

arxiv.orgHuman-Guided Complexity-Controlled AbstractionsNeural networks often learn task-specific latent representations that fail to generalize to novel settings or tasks. Conversely, humans learn discrete representations (i.e., concepts or words) at a va...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "HG-DAgger: Interactive Imitation Learning with Human Experts" (Kelly et al., 2019). This would definitely be high on my list of "papers to read if you want to understand what robot foundation model startups are doing." PDF: arxiv.org/abs/1810.02890

arxiv.orgHG-DAgger: Interactive Imitation Learning with Human ExpertsImitation learning has proven to be useful for many real-world problems, but approaches such as behavioral cloning suffer from data mismatch and compounding error issues. One attempt to address these ...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "INQUIRE: INteractive Querying for User-aware Informative REasoning" (Fitzgerald et al., CoRL 2022). A very nice way to unify different forms of information gathering and preference learning for human-robot assistance. PDF: proceedings.mlr.press/v205/fitzger...

proceedings.mlr.pressINQUIRE: INteractive Querying for User-aware Informative REasoningResearch on Interactive Robot Learning has yielded several modalities for querying a human for training data, including demonstrations, preferences, and corrections. While prior work in this space ...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "AssistanceZero: Scalably Solving Assistance Games" (Laidlaw et al., ICML 2025). AlphaZero-like combo of learning & planning for assistance games, where robot & human share reward fn that robot doesn't know. + Minecraft! PDF: arxiv.org/abs/2504.07091

arxiv.orgAssistanceZero: Scalably Solving Assistance GamesAssistance games are a promising alternative to reinforcement learning from human feedback (RLHF) for training AI assistants. Assistance games resolve key drawbacks of RLHF, such as incentives for dec...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Cooperative Inverse Reinforcement Learning" (Hadfield-Menell et al., 2016). Seminal work. My favorite part is the simple example showing that demonstration-by-expert is suboptimal. PDF: arxiv.org/abs/1606.03137

arxiv.orgCooperative Inverse Reinforcement LearningFor an autonomous system to be helpful to humans and to pose no unwarranted risks, it needs to align its values with those of the humans in its environment in such a way that its actions contribute to...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "RoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse Tools" (Shi et al., CoRL 2023). So much to like in one paper: planning, learning, deformable manipulation, GNNs, 15 3D-printed tools, and dumplings! PDF: arxiv.org/abs/2306.14447

arxiv.orgRoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse ToolsHumans excel in complex long-horizon soft body manipulation tasks via flexible tool use: bread baking requires a knife to slice the dough and a rolling pin to flatten it. Often regarded as a hallmark ...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Learning Montezuma’s Revenge from a Single Demonstration" (Salimans & Chen, 2018). 1 demo + known world model = very natural and still under-explored problem setting. PDF: arxiv.org/abs/1812.03381

arxiv.orgLearning Montezuma's Revenge from a Single DemonstrationWe propose a new method for learning from a single demonstration to solve hard exploration tasks like the Atari game Montezuma's Revenge. Instead of imitating human demonstrations, as proposed in othe...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Partially Observable Task and Motion Planning with Uncertainty and Risk Awareness" (Curtis et al., RSS 2024). State of the art for TAMP + POMDPs. I learn more every time I read this paper. PDF: arxiv.org/abs/2403.10454

arxiv.orgPartially Observable Task and Motion Planning with Uncertainty and Risk AwarenessIntegrated task and motion planning (TAMP) has proven to be a valuable approach to generalizable long-horizon robotic manipulation and navigation problems. However, the typical TAMP problem formulatio...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Continuous Deep Q-Learning with Model-based Acceleration" (Gu et al., 2016). Got swept away by other deep RL, but I always liked the idea of parameterizing Q in a form where the optimal policy can be derived analytically. PDF: arxiv.org/abs/1603.00748

arxiv.orgContinuous Deep Q-Learning with Model-based AccelerationModel-free reinforcement learning has been successfully applied to a range of challenging problems, and has recently been extended to handle large neural network policies and value functions. However,...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning" (Allen et al., PNAS 2020). Their "Virtual Tools Game" is one I revisit often when brainstorming open challenges. PDF & game: sites.google.com/view/virtual...

sites.google.comvirtualtoolsMany animals, and an increasing number of artificial agents, display sophisticated capabilities to perceive and manipulate objects. But human beings remain distinctive in their capacity for flexible, ...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Bayesian Residual Policy Optimization" (Lee et al., 2020). I like this POMDP approach because it reduces the problem to figuring out a good set of "clairvoyant experts". PDF: arxiv.org/abs/2002.03042

arxiv.orgBayesian Residual Policy Optimization: Scalable Bayesian Reinforcement Learning with Clairvoyant ExpertsInformed and robust decision making in the face of uncertainty is critical for robots that perform physical tasks alongside people. We formulate this as Bayesian Reinforcement Learning over latent Mar...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Differentiable GPU-Parallelized Task and Motion Planning" (Shen et al., RSS 2025). As always, meticulous work from @WillShenSaysHi and team. TAMP + GPU is long overdue! PDF: arxiv.org/abs/2411.11833

arxiv.orgDifferentiable GPU-Parallelized Task and Motion PlanningPlanning long-horizon robot manipulation requires making discrete decisions about which objects to interact with and continuous decisions about how to interact with them. A robot planner must select g...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Kinodynamic Task and Motion Planning using VLM-guided and Interleaved Sampling" (Kwon & Kim, 2025). I particularly like using VLMs to guide backtracking in TAMP. Outperforms PDDLStream and LLM3. PDF: arxiv.org/abs/2510.26139

arxiv.orgKinodynamic Task and Motion Planning using VLM-guided and Interleaved SamplingTask and Motion Planning (TAMP) integrates high-level task planning with low-level motion feasibility, but existing methods are costly in long-horizon problems due to excessive motion sampling. While ...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Elephants Don't Pack Groceries: Robot Task Planning for Low Entropy Belief States" (Adu-Bredu, RAL 2022). Love the focus on planning with "low entropy beliefs" -- not full-fledged POMDPs, but also not full observability. PDF: arxiv.org/abs/2011.09105

arxiv.orgElephants Don't Pack Groceries: Robot Task Planning for Low Entropy Belief StatesRecent advances in computational perception have significantly improved the ability of autonomous robots to perform state estimation with low entropy. Such advances motivate a reconsideration of robot...
Tom Silver
@tomssilver.bsky.social

This week's #PaperILike is "Interleaving Monte Carlo Tree Search and Self-Supervised Learning for Object Retrieval in Clutter" (Huang et al., ICRA 2022). Impressive results on a difficult and subtle problem, with a nice combo of planning + learning. PDF: arxiv.org/abs/2202.01426

arxiv.orgInterleaving Monte Carlo Tree Search and Self-Supervised Learning for Object Retrieval in ClutterIn this study, working with the task of object retrieval in clutter, we have developed a robot learning framework in which Monte Carlo Tree Search (MCTS) is first applied to enable a Deep Neural Netwo...