Dirac and the Path Integral

It is well known that going from classical mechanics to quantum mechanics is in general not expected to be unique. This should hardly come as a surprise since the classical analogue can be thought of as the $\hbar\,=\,0$ case and knowing $f(0)$ there is no unique way of fixing the $f(x)$! Feynman was of course aware of it, and we have quoted his views explicitly on p.5-6. There are essentially two sources of such non-uniqueness. One is the ordering ambiguity. In classical theory, momentum p and position q, being commutative can be written in any order i.e qp and pq are one and the same. On the other hand, in quantum theory, the corresponding operators ${\hat{q}},\,{\hat{p}}$ do not commute

Consequently, for example, the candidate Hamiltonians (for the moment we take any Hermitian operator as a possible Hamiltonian, ignoring issues of whether they are bounded from below) ${\hat{p}}\,{\hat{q}}\,{\hat{q}}\,{\hat{p}}\,,{\hat{p}}\,{\hat{p}}\,{\hat{q}}\,{\hat{q}}\,$ which are obviously different quantum mechanically have the same classical analogue $p^{2}q^{2}$. The other source of ambiguities, in a sense, are explicitly $\hbar$-dependent terms in the quantum Hamiltonians like $\hbar\,{\hat{\tilde{H}}}$ which vanish in the classical limit understood to be the $\hbar\,\rightarrow\,0$ limit.

Either way, every choice of the quantum Hamiltonian results in general in a different quantum description. Specifically, they lead to different observable consequences. The eigenvalues, wavefunctions, probability amplitudes will be generically different. Therefore, $\langle q_{t+\epsilon}^{\prime}|q_{t}^{\prime}\rangle$ will also be different. These are consequences of the rules for quantum theory, and are logically independent of any considerations based on path-integrals.

Even before considering the implications of these ambiguities for path-integrals, it is important to look at their impact on the missing link of eqn.(4). For reasons just explained, the l.h.s of this eqn takes different values. But all these different quantum Hamiltonians have the same classical limit, and hence the same $S_{cl}$. Consequently, they will have the same value for the r.h.s of eqn.(4)!

So there is a clear inconsistency, in general, between Feynman’s missing link relationsand non-uniqueness of quantisations of classical systems. The Feynman missing link relation can be correct for at best one class of quantisations. The resolution of this inconsistency has to lie in that eqn.(4) can not be true in general but only holds for certain classes of quantisations. There is an interesting anecdote narrated on p.226 of [3]. ”Now he asked Dirac whether the great man had known all along that the two quantities were proportional. ”Are they?” Dirac said. Feynman said yes, they are. After a silence he (Dirac) walked away”. Contrary to what Gleick and numerous others, Feynman included, had stated, that the analogue was, in fact, exactly proportional, we have argued here how they can not be, in general. In fact Gleick goes so far as to assert ”There was a rigorous and potentially useful mathematical bond”. This encounter, according to Gleick is supposed to have taken place in 1947, during the bi-centennial of the founding of Princeton university. In my opinion, Dirac’s response probably meant that Dirac was not sympathetic to the claims of their equivalence. That Dirac, as late as 1967, twenty years after this alleged anecdote, maintained his analogue description [13] is added support to this opinion!

Let us examine carefully how Feynman arrived at eqn.(4). He explicitly showed that for Lagrangians of the type

this eqn. reproduced the Schrödinger equation with the quantum Hamiltonian

To arrive at this, Feynman had to fix the normalisation $A(\epsilon)$ to be

This is indeed a special case in that the quantum Hamiltonian is insensitive to ordering! It could be that the Feynman missing link relation is consistent only for such Hamiltonians which do not have a factoring ambiguity. But even for this special class, the second source of ambiguity is still there, so the situation is not entirely clear. Differently stated, the Feynman missing link picks out one of the (infinitely) many possible quantisations of a classical system.

Whatever was said about the missing link can be immediately to generalized to the Feynman path-integral of eqn.(6). Now, the l.h.s, the Kernel $\langle q_{T}^{\prime}|q_{t}^{\prime}\rangle$ will generically take on different values for different quantum Hamiltonians, albeit with the same classical limit, while the r.h.s is determined entirely by that unique classical limit. The inconsistency manifests again.

This immediately raises a serious issue: if there is no unique path-integral representation, how can one understand the least action principle of classical mechanics? In the Feynman path-integral, the classical action appears explicitly, and applying Dirac’s stationary phase ideas immediately gives rise to the least action principle.

A related issue is, what are the implications of using the eqn.(8) instead of eqn.(4) to construct the path-integrals? Since ${\tilde{S}}$ need not be uniquely determined by $S_{cl}$, there is at least some hope to remove the inconsistency forced on by the latter eqn. In other words, different choices of ${\tilde{S}}$ may take care of the quantisation ambiguities.

In the remainder of this paper, we shall show how Dirac tackles this central question by carefully analysing his paper and the different editions of his book. In fact he comes up with a path-integral far more general than Feynman’s, completely consistent with quantisation ambiguities and yet yields the least action principle of classical mechanics.

As a prelude to Dirac’s proofs of the analogies, we begin by reviewing classical canonical transformations. These are transformations that take an old pair of canonical variables $q,p$ to a new pair $Q,P$. In other words

where ’PB’ stands for Poisson Brackect. All other PB’s are zero. Recall that the so called contact transformations given by $Q\,=\,Q(q,t)$ leave the Euler-Lagrange equations of motion unchanged. Furthermore, these are transformations taking place entirely on the configuration space.

On the other hand, Hamilton’s equations enjoy a much wider freedom i.e all the canonical transformations of the type

They have to however satisfy

where $H^{\prime}(Q,P)$ is the new Hamiltonian. As is well known, such canonical transformations can be described in terms of the so called Generating Functions [20]. For the choice when the generating function S is a function of $(q\,,Q\,,t)$ one gets

This is as far as canonical transformation at some time t are concerned. It is also known in classical mechanics that under time evolutions the change in the dynamical variables (q,p) is also described by canonical transformations! If $q_{t}\,,p_{t}$ are the dynamical variables at initial time t, and, $q_{t^{\prime}}\,,p_{t^{\prime}}$ are dynamical variables at final time $t^{\prime}$, the corresponding canonical transformations obey

Integrating these equations, one can show that S is

i.e S is the action(also Hamilton’s first principal function).

Dirac then proceeds to prove the analogy for quantum mechanics. The proof given in the paper [7] is as follows: he considers the transformation function and defines(I have changed the notation to avoid confusion; this is the same as the earlier introduced eqn.(7))

here primes and double-primes indicate eigenvalues of corresponding operators and the above equation defines ${\tilde{S}}$ which, as already emphasized, has no a priori relation to the classical action, and in fact need not even be real. But the important point is that ${\tilde{S}}$ is an ordinary function and not an operator.

To handle the non-commutativity inherent in quantum mechanics, Dirac introduces a very novel technique [7]:for every operator $\alpha$ he introduces a mixed representation $\langle\,q^{\prime}|\alpha|Q^{\prime}\rangle$. To reduce confusion the variables ${\hat{q}}_{t},{\hat{q}}_{T}$ have been renamed ${\hat{q}}\,,{\hat{Q}}$. The mixed representation can easily be related to the standard representation where matrix elements are evaluated in any basis. It is easy to see that

Dirac then introduces the notion of well-ordered operators. They are operators of the form

By using the commutation rules of quantum mechanics every operator can be brought into a well-ordered form, though in practice it can be quite unwieldy to do so. Given a pair of non-commuting operators ${\hat{q}},{\hat{Q}}$ the well-ordered form of every operator has to be unique. Else, the mixed-representation of that operator would not be unique, which in turn would imply the standard representation in any basis would not be unique.

Then, on using eqn.(20) it is easy to see that

Applying to the momentum operators $p,P$ given by

On using

Dirac shows that

These are exactly of the same form as eqns.(17), except for the time-derivative parts, which we come to next.

The paper [7] appeared in 1932. By that time the first edition of Dirac’s book (1930) [9] had already appeared and this material nor any discussion of the least action principle was in it. In 1935 Dirac came out with the second edition of the book [10] with substantial revisions. Now we take up his proof of the quantum analogue in the second edition.

Dirac completely omits the discussion of the approach based on well-ordered operators. Instead, he provides the following differential equations obtainable from Schrödinger equation:

and, likewise

On applying the well-ordered operator method, one can obtain the temporal differential equations

It is critical to have these in order to integrate them to get the analog of eqn.(18). But the equations in second edition do not seem to transparently lead to eqn.(25)! It is to be appreciated that the eqns.(LABEL:eq:qanalog2,qanalog2edtemp) involve only commuting objects with no explicit trace of the non-commutativity of quantum mechanics.

That brings us to the third edition [11]. In the third edition, Dirac reinstates the proof given in the paper along with the equations of the second edition. Thus the proof of the analogy is mathematically precise. It holds for all values of $\hbar\,,t\,,T$. There is no question of the analogues being either equal or even proportional, even for short temporal separations. A good example in this context are Poisson Brackets in classical mechanics and commutators in quantum mechanics. They are analogues of each other in the sense of obeying the same algebra, but they arev neither equal, nor proportional to each other.

That should answer Feynman’s first question, FQ1, emphatically in the negative.

Dirac sets out to investigate the consistency of the analogy

where we have used the symbol $\sim$ to denote analogy. Before proceding, it is very important to note that, for infinitesimal temporal i separations $B(t,t+\epsilon)$ only depends on the q-values at the two ends. Dirac investigates the consistency by dividing the interval $[t_{i}\,,\,t_{f}]$ into a very large number of intervals with the intervening times labelled as $t_{m}$(note that once again a change in notation has been made whereby $t_{i}\,=\,t;t_{f}\,=\,T$! On the classical side one has

It is very important to bear in mind that $B(t_{i},t_{f})$ depends, in addition to the positions at the initial and final times, on all the intermediate positions $q_{m}$ at $t_{m}$ and that these positions lie on the classical trajectory connecting the initial and final positions. Now Dirac writes down the quantum analogue of the above

All that has been used is the completeness for all the intermediate states. On using the definition of ${\tilde{S}}$ this becomes

where we have used the shorthand notation $\{q_{i}^{\prime}\,\}$ to denote the initial, final as well as all the intermediate positions. This has been delivered entirely by the analogy and is valid for arbitrary $\hbar$. This is what we like to call the Dirac Path-integral.It is for certain a path-integral.

At first sight it might appear that a major weakness of this path-integral is that nothing much seems to be known about ${\tilde{S}}$ making eqn.(32) not particularly useful. We shall now show that that is actually its major strength for it allows this generalized path-integral to be fully compatible with quantisation ambiguities which the Feynman path integral could never be.

Though the analogy is unable to fix ${\tilde{S}}$ there is much that can nevertheless be said about it, as Dirac has already done. In particular, as we approach $\hbar\,\rightarrow\,0$ ${\tilde{S}}$ must increasingly equal the classical action. That is,

We can reexpress this in a different way:if we write

Then

and the final form of the Dirac path-integral would be

Two very important features of this path-integral: though it is valid for all $\hbar$, for very small $\hbar$, the $S_{cl}$ dominates (note that it is not the only term in this limit) and Dirac’s stationary phase arguments immediately lead to the least action principle for classical analogue.

The other is that away from $\hbar\,\rightarrow\,0$, the $S^{*}$ terms are indeed important, and there is no more any conflict between the path-integral and the quantisation ambiguities; both the LHS and RHS are sensitive to these ambiguities and in fact $S^{*}$ can be taken as a way of classifying the ambiguities. This is in stark contrast with the Feynman path-integral for which $S^{*}\,=\,S^{*}_{feyn}$. On using eqn.(4), it is easy to see that

This does satisfy $S^{*}_{feyn}\,\rightarrow\,0$ as $\hbar\,\rightarrow\,0$ as required by eqn.(35).

$S^{*}$ need not be equal to $S^{*}_{feyn}$ in general, but that choice can be taken as a particular choice of quantisation for which the Feynman path-integral is the correct representation. In the absence of a deeper guiding principle for quantum Hamiltonians, any such choice is ad hoc in itself. Only empirical vindications can be used to validate the quantum Hamiltonians.

The LHS of either of these equations is a purely quantum mechanical object, calculable entirely within the quantum mechanical formalism, which necessarily involves non-commutativity. But the RHS is composed of a classical action $S_{cl}$ in the Feynman case, and, $S_{cl}\,+\,S^{*}$ in the Dirac case, along with all the intermediate ’classical positions’ as well as integrations over them. This is nothing short of a miracle as all traces of operators, non-commutativity etc have disappeared in this representation of Quantum Mechanics! Nevertheless, it is completely equivalent to the traditional formulation of quantum mechanics. This magic is entirely due to Dirac and his use of well-ordered operators to completely obviate the explicit use of non-commutativity. Since Feynman’s original construction was based on Dirac’s analogy, which directly resulted from the well-ordered techniques, this particular magic in Feynman path-integrals is a reflection of the magic of Dirac’s techniques.

As is well known, Feynman showed that even the manifestly non-commutative relation

emerges from the purely commutative calculus of path-integrals! Both Feynman’s thesis [8] as well as his paper [4] are a treasure-house for a deeper understanding of the path-integrals. The apparent mystery gets resolved on noting that in the path-integral, momentum does not appear explicitly and all the subtelety resides in a careful construction of it.

We have not demonstrated this in our paper. But we give a fairly good argument for why it should follow from Dirac path-integrals as well. The key observation is that eqn.(50) does not depend on the specific form of $S_{cl}$. For the purposes of this demonstration,one can treat the entire $S_{cl}\,+\,S^{*}$ of Dirac path-integrals as an effective $S_{cl}$. Then Feynman’s demonstration of eqn.(50) can be extended to the Dirac case as well. Of course, this effective can not be a true classical action as it will explicitly involve $\hbar$.

This is an apt place to make some remarks about various alternative approaches to path-integrals and how non-commutativity issues are handled by them. After Feynman’s construction, several approaches to deriving the path integral have appeared in the literature. One of these is to consider the quantum Hamiltonians of the type given in eqn.(11). The terms $T({\hat{p}}),V({\hat{x}})$ do not commute with each other. Working in the Schrödinger picture, the time evolution operator $e^{i\frac{{\hat{H}}{\bar{T}}}{\hbar}}$ (without loss of generality we set $t\,=\,0$ and denote final time as ${\bar{T}}$ to avoid confusion with the kinetic energy T) is rewritten as

on using the famous Trotter’s Formula. One then cleverly uses completeness of both position bases and momentum bases to arrive at the path-integral. But in Dirac’s, and consequently Feynman’s, Trotter’s formula plays no role whatsoever, for reasons explained above. Furthermore, if the Hamiltonians are not of the form of eqn.(11) it is not at all clear if Trotter’s formula would be of any use.

In the quantum histories approach [14, 15], the summands are chains of quantum operators unlike both the Dirac and Feynman path-integrals, where the summands are complex numbers. It would be a very important exercise to understand the inter-relationships between all these approaches. In particular, it will be interesting to explore how Dirac’s well ordered-operators approach can be used in this context.

Lastly, we refer to the remarkable works of Gregor Wentzel [16, 17] which appeared even before the works of Heisenberg and Schrödinger. A good account of Wentzel’s works, and their significance, can be found in [18]. According to these authors, Wentzel, in this important but forgotten work, identified $\int\,L\,dt$ as a phase that governs quantum interference. I have not fully digested Wentzel’s ideas, but they have to be understood. Certainly from the point of view understanding the non-commutativity aspects as well as those of quantisation ambiguities.

In classical mechanics the principle of least action, namely, that the path followed by a system under dynamics is the one that renders the action $S\,=\,\int\,L\,dt$(seen as a functional of paths) stationary i.e

appears from nowhere without any understanding of why it has to be so. Its main raison d’etre in classical mechanics is that it correctly reproduces the equations of motion. The Lagrangian version of the action principle is

while the Hamiltonian version is

The least action principle appears more as a pnemonic, without any deeper physical significance. The path-integral formulation is supposed to provide that significance by showing how it emerges ’naturally from the path-integral in the $\hbar\,\rightarrow\,0$ limit. To get the crux of the argument(first given by Dirac in [7]), consider fixing all intermediate $x^{\prime}$’s except one, say, $x_{k}^{\prime}$(in the Dirac path-integral of eqn.(LABEL:eq:diracpathintegral*) the notation used is $q_{k}^{\prime}$). Any variation, including infinitesimal ones, in $x_{k}^{\prime}$ will cause the phase $e^{\frac{iS_{cl}}{\hbar}}$ to fluctuate very rapidly in this limit(frequency approaching $\infty$), averaging to zero. The only exception to this behaviour would be if $x_{k}^{\prime}$ were the stationary point of $S_{cl}$. Then for sufficiently small variations in $x_{k}^{\prime}$ $S_{cl}$ will not change, and all these nearby paths will make contributions to the path-integral that will constructively interfere. Of course, for variations that are not that small, the earlier argument of destructive interference applies. As emphasized by Dirac, this line of reasoning can be applied to all the intermediate $x^{\prime}$’s with the result that in this limit, the only path that contributes is the one that extremizes the classical action. Feynman was so struck by the beauty of Dirac’s reasoning that he chose to review the salient aspects in his paper [4].

Now let us see how the same arguments can be applied to the Dirac path-integral of eqn.(36). In the limit $\hbar\,\rightarrow\,0$, the $S^{*}$ is sub-dominant compared to $S_{cl}$, so the stationary paths, even for the Dirac case, are i determined by $S_{cl}$ only!

The RHS of the path-integrals either 6, or, eqn.(36), formally look very much like the partition function of a 1-d classical statistical system except for one crucial difference;the integrand of the path-integrals is a complex phase whereas the integrand of a thermal partition function is of the form $e^{-\,\frac{H}{kT}}$. This suggests working out the quantum problem in Euclidean time. It amounts to a Wick rotation $t\,\rightarrow\,i\,\tau$ where $\tau$ is Euclidean time. This in turn leads to $S\,\rightarrow i\,S_{E}$, where $S_{E}$ is the Euclidean Action. It turns out that for most systems be it quantum mechanics or Quantum Field Theories, the Euclidean Action is positive(gravity is an exception). Thus we get the map

Thus $\hbar$ plays the role of temperature and $S_{E}$ the role of the Hamiltonian for the equivalent classical statistical system, necessarily in one higher spatial dimension!

This very deep connection could only be established in the path-integral approach. From a foundational point of view, this is the only constructive approach to quantum field theories at present. By ’constructive’ one means that all mathematical manipulations are well defined! Strictly speaking, the space-time continuum has to be approximated by discrete set of points. This is in sharp contrast to Quantum Electrodynamics of Dirac, Feynman, Schwinger, Tomonaga and Dyson where almost every manipulation leads to infinities. Dirac found the situation highly unsatisfactory and while stating ”… it is this high energy part that is responsible for the infinities”, goes on to add (section 81 Interpretations in the chapter on Quantum Electrodynamics in the revised fourth edition of his book [13]) ”It appears that we must modify the high energy part of the interaction. At present there does not exist any detailed theory of the other particles and so it is not possible to say how it ought to be modified. The best we can do is to cut it out from the theory altogether, and so remove the infinities. The precise form of the cut-off and the energy where it is applied will be left unspecified. Of course, the cut-off spoils the relativistic invariance of the theory. This is a blemish which can not be avoided in our present state of ignorance of high-energy processes.”

This was said in 1967 and subsequently there have been such high-energy modifications to QED like Quantum Chromodynamics and the Standard model of electroweak interactions. While these have ameliorated many of the defects of QED, they too are beset with infinities and the high energy modifications have not fixed the blemishes alluded to by Dirac. Though Superstring theories are claimed to be finite, we shall not comment on them here.

This equivalence of quantum field theories on a Lattice to classical statistical mechanics has had great impact and applicability. Chief among them is Lattice Gauge Theory [2] for QCD. The classical statistical mechanics systems can be numerically simulated relying on numerous techniques from statistical mechanics. There has been explosive growth in this area [19]. These formulations are of the cut-off type, with the cut-off operating at the level of the entire theory instead of process by process. The loss of relativistic invariance by discretisation is obvious.