Robot-plant bio-hybrid systems are getting increasing attention due to the wide range of applications they offer. Such synergies between robots and natural plants will allow, for example, establishing highly reliable environmental monitoring systems or growing the architecture of our future cities. We explore the latter application where robots exploit the plants’ ability to produce construction material, and plants exploit the robots’ sensing and computational capabilities. In our previous work, we used machine learning techniques to model plant behavior in their early life stages. We collected a 10-point plant stem description dataset and used it to train an LSTM as a forward model that predicts plant dynamics and drives the evolution of plant shaping controllers. Here, we show our vision to model plant behaviors in later stages, where full-plant morphology will be used to train state-of-the-art sequence modeling networks capable of simulating more complex plant dynamics.