Evaluating predictability in deterministic cellular automata through machine learning

This study investigates the predictability of Elementary Cellular Automata (ECA) — simple yet dynamically rich deterministic systems — using two state-of-the-art machine learning models: Long Short-Term Memory (LSTM) networks and Transformer architectures. Although ECAs are governed by deterministic rules, their behavior often exhibits pseudo-randomness and chaotic dynamics, especially under Class III and IV rules. Our experiments assess the ability of LSTM and Transformer models to predict future configurations of ECAs based on past states. Results reveal that LSTMs excel in modeling rules with short- to mid-range dependencies, achieving accuracies above 99% in structured scenarios, while struggling with chaotic rules such as Rule 30. Conversely, Transformers demonstrate superior performance in capturing long-range dependencies, achieving perfect accuracy for rules like Rule 90 and Rule 62, but incurring higher computational costs. These findings underscore the limits of predictability in deterministic yet complex systems and highlight how different neural architectures are suited to distinct forms of structural complexity. The work contributes to a deeper understanding of machine learning's capacity to model discrete chaotic systems and opens avenues for hybrid approaches in predictive modeling of computationally irreducible dynamics.