LLMs are powerful but often struggle with causal reasoning and planning in environments. What if we could enhance their abilities by integrating causal representation learning? Introducing our new framework: Bridging LLMs and Causal World Models for reasoning and planning!

🎯Our goal: Enable LLMs to perform causal inference and planning in environments. How? By combining Causal Representation Learning with LLMs to create a Causal World Model. CRL methods are able to separate and identify causal variables and model the dynamics of the environment.

Our Causal World Model consists of 3 main components: - Causal Encoder: Transform images into disentangled latent causal factors - Causal Transition Model: Predicts next state based on current state and action (acts as a simulator) - Decoder: Generates text descriptions of states

The Causal World Model disentangles latent causal factors, making state transitions more robust and interpretable. Input: Image + text description of the action. Output: Text description of the next state This creates an interpretable interface between the causal model and LLMs.

We compared our approach to a baseline (Reasoning Via Planning) that uses an LLM for both reasoning and world modeling. Results? Our Causal World Model outperforms the baseline in: - Multi-step causal reasoning - Planning (finding action sequences to reach goals)

We found that using textual representations for actions (rather than interaction coordinates) improves causal learning, especially when working with limited data. This makes our approach more efficient and practical than previous methods while maintaining strong performance.

🚀 Key findings: - Integrating causal understanding with LLMs improves reasoning and planning in visual environments - Text-based representations of actions are effective for causal learning, especially in low-data scenarios

Notably, our method performs particularly well in longer planning horizons where estimating the causal effects of actions is crucial while experiencing graceful performance degradation as task complexity increases.

📊 Resources: Full paper: https://t.co/PtwQw5eDDV Code: https://t.co/bkuuK8NPZ4 Project page: https://t.co/xriJthon38

👥 This research is a collaborative effort by: Me, Matthias Lindemann, @phillip_lippe, @egavves, and my wonderful supervisor @iatitov. We welcome discussions and questions about our work in causal AI and language models. #CausalAI #MachineLearning #LLM @EdinburghNLP @AmsterdamNLP

yes