Training a DNN with stochastic inputs with analysis of the robustness against input process and weight matrix fluctuations

x(t) € is the input process. y(t) € is the desired output process. It may for example be obtained from the input via nonlinear distortions with additive noise:

y(t) = ^(^(t)) + v(t)

The DNN output , assuming that it has just one layer is

z(t) = f(Wx(t) + b)

where W e R^JWxd is the weight matrix, b € R‘^v/ is the bias vector and f : R‘^v/ -> is a nonlinear map. If it has two layers then the output will be of the form

z(t) = f₂(W₂f₁(W₁x(t) + + b₂)

More generally, if it has L layers, then

Z(t) = /L(lVL/L-l(WT-l/£-2(..J₂(VV₂/l(IVl^(i) + h) + M-) + *>l)

We write this as

z(t) = f(W,b,x(t))

where

W = (W₁,...,W_L),fe=(b₁,...,6_L)

Suppose the bias b is fixed so we can write

z(t) = f(W,x(t))

We choose W so that

E[G/(t)-z(t))²]

is a minimum. Assume that this minimum is attained when W = Wo. Now suppose Wo gets perturbed slightly to Wo + JW and simultaneously the input process x(t) gets perturbed slightly to x(t) + 8x(t). We wish then to design the weight matrix perturbation fiW so that the mean square error E(y(t) — /(Wo + fiW, x(t) + fe(t)))² is still a minimum. Using Taylor series, we expand

/(Wo + d'W, x + fe) = _ea;p((JW V_lv) + (fe, V,))/( W_o, x)

« /(Wo, rro) + [(¿W, VW) + (fe, VJ]/(W_o, ¿’o)

+(1/2)[(<5W, V_vv) + (fix, V_x)]²f(W₀,x₀)

In this approximation, we have retained only upto quadratic order terms in <5W, fix. Then,

E(d'W) = E(y(t) - /(Wo + SW, x(t) + fe(t)))²

« E[(e(t) - ((¿W, V_vv) + (fe(t), V_x.))/(w₀, x(t)) -(1/2)[(d'W, V_lv) + (fix, VJ]²/(W₀, x(t))]²

Quantum Boltzmann equation

An approximate derivation. The mixed N particle state p(t) Є is

assumed to have all same marginals of any given order. Thus, if k > 1 and ¿i < ¿2 < ••• < ik < N and (Ji,..., jN-i) is the complement of {«i,..., i_k), then we are assuming that

Pi2...k(t) = Tr_k+1....,N(p(.t)) = T_rjl.....j_N__k)(p(t))

The Hamiltonian of the system is

N

Н = ^^На+ E

a=l l

Pi2Pi(0 <8>Pi(t) + 912(0,

P123 = P12 ® Pl + 9123 = Pl ® pl <8> pl + 912 ® Pl + 9123

Then,

*/12(0 = Up'i ® pl + Pi ® Pl + 912') =

[Hi + H‘2 + Vi₂, p ® pi + 9i₂] + (A — 2)Тгз[Тіз + +23,pi₂ ® pi + 9133]

Using the equation of motion for pi, this gives the following exact equation followed by the approximate equation based on treating 9123 as small and of the second order of smallness as compared to V_ai>,

19'12 = [Hi + H₂ + +12,912] + (Л^г — 2)Тгз[кіз + +23,912 ® pi + 9123]

~ [Hi + H₂ + +12,912] + (A — 2)Тгз[Уіз + +23, Pi ® pi ® pi +912 <8> pi]

= [Hi + H₂ + +12,912] + (A^r - 2)7V₃[Vi3 + +23,912 ® Pi]

If we no regard 912 as the same order of smallness as V_ab, then this last equation further approximates to

*P12 ^— №1 + H₂ + +12,912]

which gives

£12^) = exp{-itad{H_x + H₂ + V₁₂))(pi₂(0)) = Ti₂(t)(gi2(0))

say. Then, the equation for pi becomes with this approximation

ip'i = [Hi, pi] + (N - l)Tr₂[Vi2,pi ® pi + 512]

= [Hi, pi] + (N - l)Tr₂[Vi2,pi <8> pi] + (H - l)Tr₂[V₁₂,T₁₂(i)(i?i₂(0))]

This may be termed as the quantum Boltzmann equation.

List of Ph.D scholars supervised by Har-ish Parthasarathy with a brief summary of their theses

[1] S.N.Sharma: Applications of nonlinear filtering theory to certain problems in classical mechanics.

(NSIT-DU, 2004)

Summary: The stochastic two body gravitational problem in the presence of interplanetary dust modeled using Brownian motion has the form

r"(t) - r(t)0'^[1] (t) = -GM/r¹ + A(t),

r(t)6»"(i) + = fe(t)

where f_r,fe are white noise processes, ie, formal differentials of Brownian motion. These equations are cast in the from of four coupled nonlinear Ito stochastic differential equations and mean and covariance propagation equations for this system are obtained by expanding the variables r(t), 0(t), r'(t), 0'(t) about their mean values and applying Ito’s formula for Brownian motion to obtain the covariance propagation equations. Further, by taking noisy measurements on the position of the body, the EKF is applied to obtain real time estimates of the trajectory. Sharma then applies the Kushner nonlinear filtering theory to the nonlinear Van-der-Pol and Duffing oscillator taking cubic corrections into account, ie, he goes a step further than the EKF which is based on expanding upto quadratic terms. More accurate filtering results are obtained by considering the joint evolution of the first three conditional moments.

phase algorithms using stochastic differential equations driven by Brownian motion.

(NSIT-DU, 2009)

[4] Sudipta Majumdar:Modeling and parameter estimation in nonlinear transistor circuits using Volterra approximations combined with wavelet based compression for data storage for the purpose of estimation.

(NSIT-DU, 2010)

[5] Arthi Vaish:Finite element method for determining the modes in waveguides having various kinds of cross section and with inhomogeneous and anisotropic media filling the guide taking into account background gravitational perturbations in the form of a curved space-time metric.

(NSIT-DU, 2011)

[6] Akash Rathee:Study of higher harmonic generation in nonlinear transistor circuits using Fourier series and perturbation theory.

(NSIT-DU, 2011)

[7] Raj eev Srivastava: Image modeling, smoothing and enhancement using partial differential equations with emphasis on diffusion equations with intensity dependent diffusion matrix coefficient.

(NSIT-DU, 2009)

[8] Rajveer S.Yaduvanshi: Magneto-hydrodynamic antenna construction analysis using Navier-Stokes and Boltzmann kinetic transport equation.

(NSIT-DU, 2010)

[9] Lalit Kumar: Studies in transmission line and waveguide analysis taking hysteresis and capacitive nonlinearities and quantum mechanical effects of transmission line and waveguide fields on atoms and quantum harmonic oscillators.

(NIT, 2018)

[10] Kumar Gautam: Quantum gate design by perturbing real quantum systems with electromagnetic fields.

(NSIT-DU, 2017)

[11] Rohit Singla: Studies in robot trajectory tracking and dynamic parameter estimation in the presence of noise and in master-slave teleoperation based tracking using adaptive control algorithms.

(Yet to submit, has published three technical papers in the impact factor-4 Springer journal ’’Nonlinear Dynamics”)

[12] Navneet Sharma: Quantum parameter estimation using search algorithms with applications to quantum communication and quantum gate design.

Given an unperturbed Hamiltonian Hq and a time dependent perturbation operator V(t) = ^fcVfc(t), the problem is to estimate the parameter vector 0 = (0fc) based on measuring observables X_a, a = 1, 2,..., n on the system state at different times taking into account the collapse postulate of quantum mechanics. For example, let U(t|0) denote the unitary evolution:

p

iU'(t, s|0) = (Ho + E Wt))U{t, s|0), t > s, k=l

U(s,s) = I

Let {M_a : a = 1,2, ..., r} be a POVM. Then, if p(0) is the initial state of the system, the probability of measuring the outcomes a-i,02,...,a_s at times ¿1 < Î2 < ••• < t_s is given by

F(ai, ...,a_s,ti,...,t_s) =

where

Ea = x/M^.

We then estimate 9 by applying the maximum likelihood method to this probability.

(Yet to defend, all examiner reports recommend award of degree)

[13] Pravin Malik: Antenna design using numerical solution of integral equations arising from the boundary conditions on the antenna surface.

[14] Manisha Khulbe: Studies in electromagnetic wave propagation in inhomogeneous, anisotropic and field dependent (nonlinear) media with applications to estimating the medium parameters from discrete measurements of the electromagnetic field at different space-time points. Applications to antenna design for wave propagation in nonlinear media are also considered. For this, perturbative expansions of the medium permittivity and permeability as Taylor series in the electric and magnetic fields as well as expansion of the field independent coefficients of the permittivity and permeability in terms of basis functions is performed. These expansions are substituted into the Maxwell equations to obtain a sequence of linear equations for each perturbative order. Boundary conditions of the electromagnetic fields on the antenna surface are applied to derive integral equations for the induced surface current density.

• [1] Vipin Behari Vats: Parameter estimation algorithms in nonlinear systems using nonlinear LMS algorithm with a study of the behaviour of the Lyapunov exponents of autonomous nonlinear systems for small initial perturbations around a fixed point. (NSIT-DU, 2007) 2 Tarun K.Rawat: Applications of stochastic nonlinear filtering theory to trajectory maneouvering of spacecrafts and convergence analysis of least mean