Quantifying brain state transition cost is a fundamental problem in systems neuroscience. Previous studies utilized network control theory to measure the cost by considering a neural system as a deterministic dynamical system. However, this approach does not capture the stochasticity of neural systems, which is important for accurately quantifying brain state transition cost. Here, we propose a novel framework based on optimal control in stochastic systems. In our framework, we quantify the transition cost as the Kullback-Leibler divergence from an uncontrolled transition path to the optimally controlled path, which is known as Schrödinger Bridge. To test its utility, we applied this framework to functional magnetic resonance imaging data from the Human Connectome Project and computed the brain state transition cost in cognitive tasks. We demonstrate correspondence between brain state transition cost and the difficulty of tasks. The results suggest that our framework provides a general theoretical tool for investigating cognitive functions from the viewpoint of transition cost. Author Summary In our daily lives, we perform numerous tasks with different kinds and levels of cognitive demand. To successfully perform these tasks, the brain needs to modulate its spontaneous activity to reach an appropriate state for each task. Previous studies utilized optimal control in deterministic systems to measure the cost for the brain state transition. However, no unified framework for quantifying brain state transition cost that takes account of the stochasticity of neural activities has been proposed. Here, we describe a novel framework for measuring brain state transition cost, utilizing the idea of optimal control in stochastic systems. We assessed the utility of our framework for quantifying the cost of transitioning between various cognitive tasks. Our framework can be applied to very diverse settings because of its generality.