Optimality in natural physics: From Stochastic Optimal Control to Schrödinger equation

The hidden “feedback” in Quantum mechanics

Preface

We have demonstrated the underlying optimality in classic Newtonian mechanics in this post. In another post, we’ve also shown that Stochastic Optimal Control Problem(SOCP) can be solved by simulating a diffusing process [1] (interpreted as a classical Path Integral as the figure depicts below). These results prompt an interesting question that will be informally exploited in this post.

Is there a corresponding Optimal Control Problem that leads to Quantum mechanics?

Solving Stochastic Optimal Control via Path Integrals for classic (real numbered) dynamic

Stochastic Hamilton–Jacobi–Bellman equation (S-HJB)

A dynamic system with stochastic perturbation modeled as a Wiener process can be formulated as the following equations. We will only considered a one dimention system in the following derivation.

Stochastic dynamic system with an exogenous noise input “v”

The corresponding time-continuous HJB equation [2] can be derived if we deployed a “closed-loop optimal strategy” with the time integral of Lagrange function “L” as the cost function. Readers may refer to this post for the implication of closed-loop optimal control and the derivation of the HJB.

The derived Stochastic Hamilton–Jacobi–Bellman equation (S-HJB) with a fully actuated control input “f=u”

Like-wise the setup in finding the optimality in classic dynamics, we assumed a fully actuated system without any passive dynamics “dx=udt+dξ”.

The Optimal control u*

By taking the first-order derivative of the term left inside the minimization, the negative state gradient of the optimal cost-to-go “p” and the term Hamiltonian “H” could be defined over the state space with the optimal strategy “u*” applied everywhere. If the relation between the partially differentiated Lagrange “L” and the co-state “p” could be inversed, the Hamiltonian “H” could be further treated as a new function following the Legendre transformation [3].

Solving the optimal control, the definition of co-state “p”, and the solved S-HJB equation

For a single particle sitting in a potential field, the Lagrange and Hamiltonian could be expanded and leads to a more explicit form of the S-HJB equation as shown.

Solved S-HJB for a one-dimension particle under a potential field “q”, notice the mass serves as the coefficient of the quadratic control cost

Without diffusion (v=0)

If there is simply no diffusion at all “v=0”, the S-HJB will shrink to an ordinary HJB. Defining the optimal cost-to-reach “S” (basically the term “Action” in classical dynamics) as the total cost with the optimal cost-to-go subtracted, the final result resembles the well-known Hamilton-Jacobi equation [4]. Please also refer to our previous post for a more comprehensive discussion.

S-HJB with “v=0”, and the transformation between optimal cost-to-go “J” and the optimal cost-to-reach “S”. The last equation is essentially the Hamilton-Jacobi equation.

With real diffusion coefficient (v∈ℝ)

This is exactly the case discussed in this previous post, the S-HJB for the exponentially transformed optimal cost-to-go “J” could be derived similarly as shown. The transformed function “Ψ” could be interpreted as a probabilistic distribution function that follows a diffusion rate of “v” and a dissipation rate “-q/(mv)”. Thermal conduction over a continuous material with a radiation loss is one of the physical pictures for such a dynamic.

S-HJB with real “v”, formed with the transformed cost “Ψ”, resembles a dissipative and diffusion process.

With complex diffusion coefficient (v∈ℂ)

With the diffusing coefficient set specifically as “v=-iℏ/m”, we will be rewarded with the celebrated Schrödinger equation [5]. The inverse relation between the noise “v” and the mass “m” is essential to this result as depicted in the derivation shown above (and reference [1]), while the idea of using a complex diffusing coefficient is not new and may refer to [6].

S-HJB with complex “v”, formed with the transformed cost “Ψ”, leads to the Schrödinger equation

The hidden “feedback” in Quantum mechanics

A closed-loop type optimal controller (strategy) is a very important concept that enables a traceable result under a stochastic system. In the derivation under a real diffusing case, we already assume an interchangeable minimization and expectation by replacing a static optimal control “value ” with an optimal strategy (encoded by the optimal cost-to-go function). This idea can be extended under a complex diffusing coefficient with a Path Integral approach over a “complex cost/action”. This implies that Quantum mechanics can be viewed as a continuously, optimally controlled system.

Quantum mechanic is a feedback controlled optimal solution for some cost function

References

[1] H. J. Kappen, ‘Path integrals and symmetry breaking for optimal control theory’, Journal of statistical mechanics: theory and experiment, τ. 2005, τχ. 11, σ. P11011, 2005.

[2] R. Bellman, R. Corporation, en K. M. R. Collection, Dynamic Programming. Princeton University Press, 1957.

[3] R. K. P. Zia, E. F. Redish, en S. R. McKay, “Making sense of the Legendre transform”, American Journal of Physics, vol 77, no 7, bll 614–622, 2009.

[4] B. Houchmandzadeh, “The Hamilton-Jacobi Equation : an intuitive approach.”, American Journal of Physics, vol 88, no 5, bll 353–359, Apr 2020.

[5] J. Townsend, A Modern Approach to Quantum Mechanics. University Science Books, 2012.

[6] H. H. Rosenbrock, “A variational principle for quantum mechanics,” Physics Letters A, 29-Aug-2002.