When cultural probes emerged in 1999, they were, for the most part, crafted from physical, non-digital, materials used to explore people's lives through playful co-creation and ambiguity. They have since become an important method for design researchers to generate insights into user behavior. Today, there is a growing need for user research to involve remote alternatives, something that was very much amplified during the global COVID-19 pandemic. The ubiquity of smartphones has given researchers the potential to have a remote window into people's lives like never before. This raises questions about the probe-like qualities of smartphone apps from a design research perspective. What options are available to design researchers? What are their strengths and weaknesses? What criteria matter? How can we improve on them? We share what we learned while using a mobile diary app for a remote in-home study with 31 households. We discuss the limitations of the app we ended up using and identify the need for an easy-to-adapt, off-the-shelf digital probe suitable for design researchers. The results and findings are intended to encourage designers to work toward digital probes and provide guidance for those who depend on commercially available mobile dairy apps in the meantime.