DeepNLP Equation Formula Database and Directory Latex Code & Learning Explanation

Navigation

economics
nlp
physics
ai
math
machine learning
statistics

Equation Database

economics

Cox-Ingersoll-Ross CIR
Equation of Exchange
Geometric Brownian Motion SDEs
Ito Lemma
Risk-Neutral Valuation and Power Contracts
Sharpe Ratio
Stock Prices as Geometric Brownian Motion

Contrastive Preference Optimization CPO
Direct Policy Optimization DPO
Direct Preference Optimization DPO
Generalized Knowledge Distillation GKD
Group Relative Policy Optimization GRPO
KTO Kahneman-Tversky Optimisation Equation
LOW RANK ADAPTATION LORA
RLHF Reinforcement Learning from Human Feedback

Electric Oscillations
General Relativity
Maxwell Equations Integral
Mechanic Oscillations
Waves In Long Conductors

Contrastive Preference Optimization CPO
Generalized Knowledge Distillation GKD
Group Relative Policy Optimization GRPO
KTO Kahneman-Tversky Optimisation Equation
LOW RANK ADAPTATION LORA
RLHF Reinforcement Learning from Human Feedback

Arithmetic and Geometric Progressions
Binomial Expansion
Convergence of Series
Determinants of a Matrix
Diffusion Conduction Equation
Eigenvalues and Eigenvectors
Fourier Series
Fourier Transforms
Heat Equation
Laplace Transform

Average Treatment Effect ATE
Bound on Target Domain Error
Conditional Average Treatment Effect CATE
Diffusion Model Forward Process
Diffusion Model Reverse Process
Diffusion Model Variational Lower Bound
Diffusion Model Variational Lower Bound Loss
Domain Adaptation H-Divergence
Domain-Adversarial Neural Networks DANN
Generative Adversarial Networks GAN

Exponential Distribution
Fisher Skewness
Gamma Distribution
Normal Gaussian Distribution

economics

Cox-Ingersoll-Ross CIR

#Financial #Economics

$$\mathrm{d} r(t) = a[b - r(t)] \mathrm{d} t + \sigma \sqrt{r(t)} \mathrm{d} Z(t) \\ P(r, t, T) = A(T-t)e^{-rB(T-t)} \\ \gamma = \sqrt{(a-\bar{\phi})^{2} + 2 \sigma^{2}} \\ q(r, t, T) = \sigma \sqrt{r} B(T-t) \\ \text{yield to maturity} \\ \frac{2ab}{ a - \bar{\phi} + \gamma}$$

Equation of Exchange

#Economics #MacroEconomics

$$\text{MV} = \text{PQ}$$

Geometric Brownian Motion SDEs

#Financial #Economics

$$\mathrm{d}Y(t) = \mu Y(t)dt + \sigma Y(t) \mathrm{d}Z(t) \\ \mathrm{d}[\ln Y(t)] = (\mu - \frac{\sigma^2}{2}) \mathrm{d}t + \sigma \mathrm{d}Z(t) \\ Y(t) = T(0) e^{(\mu - \frac{\sigma^2}{2})t + \sigma Z(t)}$$

Ito Lemma

#Financial #Economics

$$\mathrm{d}X(t) = a(t, X(t)) \mathrm{d}t + b(t, X(t))\mathrm{d} Z(t) \\ Y(t) = f(t, X(t)) \mathrm{d}t \\ \mathrm{d} Y(t) = f_{t}(t, X(t)) + f_{x}(t, X(t))\mathrm{d} X(t) + \frac{1}{2} f_{xx}(t, X(t))[\mathrm{d}X(t)]^{2} \\ [\mathrm{d} X(t)]^{2} = b^{2}(t, X(t))\mathrm{d} t$$

Risk-Neutral Valuation and Power Contracts

#Financial #Economics

$$\frac{\mathrm{d}S(t)}{S(t)} = (r - \delta) \mathrm{d}t + \sigma \mathrm{d} \tilt{Z}(t) \\ \tilt{Z}(t) = Z(t) + \phi t \\ V(S(t), t) = e^{-r(T-t)} E^{*}[V(S(T), T) | S(T)] \\ F^{p}_{t, T}(S^{a}) = S^{a}(t) e ^{ (-r + a(r-\delta) + \frac{1}{2} a(a-1)\sigma^{2})(T-t)}$$

Sharpe Ratio

#Financial #Economics

$$\frac{\mathrm{d}X(t)}{X(t)} = m \mathrm{d}t + s \mathrm{d}Z(t) \\ \phi = \frac{m + \delta -r }{s} \\ \phi = \frac{a - r}{\sigma}$$

Stock Prices as Geometric Brownian Motion

#Financial #Economics

$$\frac{\mathrm{d}S(t)}{S(t)} = (a - \delta) \mathrm{d}t + \sigma \mathrm{d}Z(t) \\ S(t) = S(0) e^{(a - \delta - \frac{\sigma^{2}}{2})t + \sigma Z(t)} \\ \mathrm{d}[\ln S(t)] = (a - \delta - \frac{\sigma^{2}}{2}) \mathrm{d}t + \sigma \sigma \mathrm{d} Z(t) \\ S(t) \sim \ln( \ln S(0) + (a - \delta - \frac{\sigma^2}{2})t, \sigma^{2}t)$$

Contrastive Preference Optimization CPO

#AI #nlp #llm #RLHF

$$\mathcal{L}(\pi_\theta;\pi_{\text{ref}}) = -\mathbb{E}_{(x,y_w,y_l) \sim \mathcal{D}} \Big[ \log \sigma \Big( \beta \log \frac{\pi_{\theta}(y_w | x)}{\pi_{\text{ref}}(y_w | x)} - \beta \log \frac{\pi_{\theta}(y_l | x)}{\pi_{\text{ref}} (y_l | x)} \Big) \Big] $$ $$ \mathcal{L}(\pi_\theta;U) = -\mathbb{E}_{(x,y_w,y_l) \sim \mathcal{D}} \Big[ \log \sigma \Big( \beta \log \pi_{\theta}(y_w | x) \nonumber \\ - \beta \log \pi_{\theta}(y_l | x) \Big) \Big] $$ $$ \min_\theta \mathcal{L}(\pi_\theta, U) \notag \text{ s.t. } \mathbb{E}_{(x,y_w) \sim \mathcal{D}}\Big [ \mathbb{KL}(\pi_w(y_w|x)||\pi_\theta(y_w|x))\Big] < \epsilon $$ $$ \min_\theta\underbrace{ \mathcal{L}(\pi_\theta, U)}_{\mathcal{L}_\text{prefer}} \underbrace{-\mathbb{E}_{(x,y_w) \sim \mathcal{D}} [\log \pi_\theta(y_w| x)]}_{\mathcal{L}_\text{NLL}} $$

Direct Policy Optimization DPO

#nlp #llm #RLHF

$$\pi_{r} (y|x) = \frac{1}{Z(x)} \pi_{ref} (y|x) \exp(\frac{1}{\beta} r(x,y) ) , r(x,y) = \beta \log \frac{\pi_{r} (y|x)}{\pi_{ref} (y|x)} + \beta \log Z(x) , p^{*}(y_{1} > y_{2} |x) = \frac{1}{1+\exp{(\beta \frac{\pi^{*} (y_{2}|x)}{\pi_{ref} (y_{2}|x)} - \beta \frac{\pi^{*} (y_{1}|x)}{\pi_{ref} (y_{1}|x)} )}} , \mathcal{L}_{DPO}(\pi_{\theta};\pi_{ref}) = -\mathbb{E}_{(x, y_{w},y_{l}) \sim D } [\log \sigma (\beta \frac{\pi_{\theta} (y_{w}|x)}{\pi_{ref} (y_{w}|x)} - \beta \frac{\pi_{\theta} (y_{l}|x)}{\pi_{ref} (y_{l}|x)} )] , \nabla \mathcal{L}_{DPO}(\pi_{\theta};\pi_{ref}) = - \beta \mathbb{E}_{(x, y_{w},y_{l}) \sim D } [ \sigma ( \hat{r}_{\theta} (x, y_{l}) - \hat{r}_{\theta} (x, y_{w})) [\nabla_{\theta} \log \pi (y_{w}|x) - \nabla_{\theta} \log \pi (y_{l}|x) ] ] , \hat{r}_{\theta} (x, y) = \beta \log (\frac{\pi_{\theta} (y|x)}{\pi_{ref} (y|x)})$$

Direct Preference Optimization DPO

#nlp #llm #RLHF

$$\pi_{r} (y|x) = \frac{1}{Z(x)} \pi_{ref} (y|x) \exp(\frac{1}{\beta} r(x,y) ) $$ $$ r(x,y) = \beta \log \frac{\pi_{r} (y|x)}{\pi_{ref} (y|x)} + \beta \log Z(x) $$ $$ p^{*}(y_{1} > y_{2} |x) = \frac{1}{1+\exp{(\beta \frac{\pi^{*} (y_{2}|x)}{\pi_{ref} (y_{2}|x)} - \beta \frac{\pi^{*} (y_{1}|x)}{\pi_{ref} (y_{1}|x)} )}} $$ $$ \mathcal{L}_{DPO}(\pi_{\theta};\pi_{ref}) = -\mathbb{E}_{(x, y_{w},y_{l}) \sim D } [\log \sigma (\beta \frac{\pi_{\theta} (y_{w}|x)}{\pi_{ref} (y_{w}|x)} - \beta \frac{\pi_{\theta} (y_{l}|x)}{\pi_{ref} (y_{l}|x)} )] $$ $$ \nabla \mathcal{L}_{DPO}(\pi_{\theta};\pi_{ref}) = - \beta \mathbb{E}_{(x, y_{w},y_{l}) \sim D } [ \sigma ( \hat{r}_{\theta} (x, y_{l}) - \hat{r}_{\theta} (x, y_{w})) [\nabla_{\theta} \log \pi (y_{w}|x) - \nabla_{\theta} \log \pi (y_{l}|x) ] ] $$ $$ \hat{r}_{\theta} (x, y) = \beta \log (\frac{\pi_{\theta} (y|x)}{\pi_{ref} (y|x)})$$

Generalized Knowledge Distillation GKD

#AI #nlp #llm #RLHF

$$L_\mathrm{GKD}(\theta) := (1 - \lambda) \mathbb{E}_{(x, y) \sim (X, Y)} \big[ \mathcal{D}(p_{T} \| p_{S}^\theta)(y|x) \big] + \lambda \mathbb{E}_{x\sim X} \Big[\mathbb{E}_{y \sim p_{S} (\cdot|x)} \big[\mathcal{D}(p_{T} \| p_{S}^\theta)(y|x)\big]\Big] $$

Group Relative Policy Optimization GRPO

#AI #Machine Learning #NLP

$$\mathcal{J}_{GRPO}(\theta) = \mathbb{E}{[q \sim P(Q), \{o_i\}_{i=1}^G \sim \pi_{\theta_{old}}(O|q)]} $$ $$ \frac{1}{G}\sum_{i=1}^G\frac{1}{|o_i|} \sum_{t=1}^{|o_i|} \left\{ \min \left[ \frac{\pi_\theta(o_{i,t} | q, o_{i,\lt t})}{\pi_{\theta_{old}}(o_{i,t} | q, o_{i,\lt t})} \hat{A}_{i,t}, \text{clip} \left( \frac{\pi_\theta(o_{i,t} | q, o_{i,\lt t})}{\pi_{\theta_{old}}(o_{i,t} | q, o_{i,\lt t})}, 1 - \epsilon, 1 + \epsilon \right) \hat{A}_{i,t} \right] - \beta \mathbb{D}_{KL}\left[\pi_{\theta} || \pi_{ref}\right]\right\}$$

KTO Kahneman-Tversky Optimisation Equation

#nlp #llm #AI

$$f(\pi_\theta, \pi_\text{ref}) = \mathbb{E}_{x,y\sim\mathcal{D}}[ a_{x,y} v(r_\theta(x,y) - \mathbb{E}_{Q}[r_\theta(x, y')])] + C_\mathcal{D}$$

LOW RANK ADAPTATION LORA

#AI #nlp #llm #RLHF

$$W_{0} + \Delta W_{0} = W_{0} + BA, h=W_{0}x + \Delta W_{0}x = W_{0}x + BAx, \text{Initialization:} A \sim N(0, \sigma^{2}), B = 0$$

RLHF Reinforcement Learning from Human Feedback

#AI #nlp #LLM #equation

$$p^*(y_w \succ y_l|x) = \sigma(r^*(x,y_w) - r^*(x,y_l)) $$ $$ \mathcal{L}_R(r_\phi) = \mathbb{E}_{x,y_w,y_l \sim D}[- \log \sigma(r_\phi(x, y_w) - r_\phi(x, y_l))] $$ $$ \mathbb{E}_{x \in D, y \in \pi_\theta} [r_\phi(x,y)] - \beta D_{\text{KL}}(\pi_\theta(y|x) \| \pi_{\text{ref}}(y|x)) $$

Electric Oscillations

#physics #oscillations

$$\text{Impedance} \\ Z=R+ix \\ \text{Series connection} \\ V=IZ, Z_{\rm tot}=\sum_i Z_i~,~~L_{\rm tot}=\sum_i L_i~,~~ \frac{1}{C_{\rm tot}}=\sum_i\frac{1}{C_i}~,~~Q=\frac{Z_0}{R}~,~~ Z=R(1+iQ\delta) \\ \text{Parallel connection} \\ \frac{1}{Z_{\rm tot}}=\sum_i\frac{1}{Z_i}~,~~ \frac{1}{L_{\rm tot}}=\sum_i\frac{1}{L_i}~,~~ C_{\rm tot}=\sum_i C_i~,~~Q=\frac{R}{Z_0}~,~~ Z=\frac{R}{1+iQ\delta}$$

General Relativity

#physics #relativity

$$\frac{\mathrm{d}^{2} x^{\alpha}}{\mathrm{d} s^{2}}+\Gamma^{\alpha}_{\beta\gamma}\frac{\mathrm{d} x^{\beta}}{\mathrm{d} s}\frac{\mathrm{d} x^{\gamma}}{\mathrm{d} s}=0$$

Maxwell Equations Integral

#physics #maxwell #electricity #magnetism

$$\oiint (\vec{D}\cdot \vec{n}) \mathrm{d}^{2}A=Q_{\text{free,included}}\\ \oiint (\vec{B}\cdot \vec{n}) \mathrm{d}^{2}A=0 \\ \oint \vec{E} \mathrm{d}\vec{s}=-\frac{\mathrm{d}\Phi}{\mathrm{d}t}\\ \oint \vec{H} \mathrm{d}\vec{s}=I_{\text{free,included}}+\frac{\mathrm{d}\Psi }{\mathrm{d}t}$$

Mechanic Oscillations

#physics #mechanic #oscillations

$$m\ddot{x}=F(t)-k\dot{x}-Cx \\ F(t)=\hat{F}\cos(\omega t) \\ -m\omega^2 x=F-Cx-ik\omega x \\ \omega_0^2=C/m \\ x=\frac{F}{m(\omega_0^2-\omega^2)+ik\omega} \\ \dot{x}=\frac{F}{i\sqrt{Cm}\delta+k} \\ \delta=\frac{\omega}{\omega_0}-\frac{\omega_0}{\omega} \\ Z=F/\dot{x} \\ Q=\frac{\sqrt{Cm}}{k}$$

Waves In Long Conductors

#physics #waves #oscillations

$$Z_0=\sqrt{\frac{dL}{dx}\frac{dx}{dC}} \\ v=\sqrt{\frac{dx}{dL}\frac{dx}{dC}}$$

Contrastive Preference Optimization CPO

#AI #nlp #llm #RLHF

Generalized Knowledge Distillation GKD

#AI #nlp #llm #RLHF

Group Relative Policy Optimization GRPO

#AI #Machine Learning #NLP

KTO Kahneman-Tversky Optimisation Equation

#nlp #llm #AI

$$f(\pi_\theta, \pi_\text{ref}) = \mathbb{E}_{x,y\sim\mathcal{D}}[ a_{x,y} v(r_\theta(x,y) - \mathbb{E}_{Q}[r_\theta(x, y')])] + C_\mathcal{D}$$

LOW RANK ADAPTATION LORA

#AI #nlp #llm #RLHF

$$W_{0} + \Delta W_{0} = W_{0} + BA, h=W_{0}x + \Delta W_{0}x = W_{0}x + BAx, \text{Initialization:} A \sim N(0, \sigma^{2}), B = 0$$

RLHF Reinforcement Learning from Human Feedback

#AI #nlp #LLM #equation

Arithmetic and Geometric Progressions

#math #arithmetic #geometric #progressions

$$S_{n}=a+(a+d)+(a+2d)+...+[a+(n-1)d]=\frac{n}{2}[2a+(n-1)d] \\ S_{n}=a+ar+ar^{2}+...+ar^{n-1}=a\frac{1-r^{n}}{1-r}$$

Binomial Expansion

#math #binomial #expansion

$$(1+x)^{n}=1+nx+\frac{n(n-1)}{2!}x^{2}+...+ C^{r}_{n}x^{r} +...+x^{n} \\ C^{r}_{n}=\frac{n!}{r!(n-r)!}$$

Convergence of Series

#math #convergence #series

$$S_{n}=u_{1}+u_{2}+...+u_{n} \\ \text{Converge AS } n \rightarrow \infty \text{, If} \lim_{n \rightarrow \infty} |\frac{u_{n+1}}{u_{n}}| < 1$$

Determinants of a Matrix

#math #determinants

$$|A|=\sum_{i,j,k,...}\epsilon_{ijk}A_{1i}A_{2j}A_{3k}...$$

Diffusion Conduction Equation

#math #diffusion #conduction

$$\kappa u_{xx} = u_{t} \\ \frac{\partial \Psi}{\partial t}=\kappa \triangledown^{2} \Psi$$

Eigenvalues and Eigenvectors

#math #eigenvalues #eigenvectors

$$A\mathbf{u}=\lambda\mathbf{u} \\ P_{n}(\lambda)=|A-\lambda I|$$

Fourier Series

#math #fourier series

$$y(x)=c_{0}+\sum^{M}_{m=1}c_{m}\cos mx+\sum^{M^{'}}_{m=1}s_{m}\sin mx \\ c_{0}=\frac{1}{2\pi}\int^{\pi}_{-\pi}y(x) \mathrm{d} x \\ c_{m}=\frac{1}{\pi}\int^{\pi}_{-\pi}y(x) \cos mx \mathrm{d} x \\ s_{m}=\frac{1}{\pi}\int^{\pi}_{-\pi}y(x) \sin mx \mathrm{d} x$$

Fourier Transforms

#math #fourier transforms

$$y(t)=\frac{1}{2\pi}\int_{-\infty}^{\infty} \hat{y}(\omega) e^{i\omega t} \mathrm{d} \omega \\ \hat{y}(\omega)=\int_{-\infty}^{\infty} y(t) e^{-i\omega t} \mathrm{d} t$$

Heat Equation

#math #heat equation

$$u_{t}={\alpha}^{2}u_{xx}$$

Laplace Transform

#math #laplace transform

$$\bar{y}(s)=\mathcal{L}(y(t))=\int^{\infty}_{0} e^{-st} y(t) \mathrm{d}t$$

Maclaurin Series

#math #maclaurin series

$$y(x)=y(a+u)=y(0)+x\frac{\mathrm{d} y}{\mathrm{d} x}+\frac{1}{2!}x^{2}\frac{\mathrm{d}^2 y}{\mathrm{d} x^{2}}+\frac{1}{3!}x^{3}\frac{\mathrm{d}^3 y}{\mathrm{d} x^{3}}+...$$

Plane Wave Expansion

#math #plane wave expansion

$$\exp(ikz)=\exp(ikr\cos \theta=\sum^{\infty}_{l=0}(2l+1)i^{l}j_{l}(kr)P_{l}(\cos \theta)$$

Power Series with Real Variables

#math #power #series

$$e^{x}=1+x+\frac{x^{2}}{2!}+...+\frac{x^{n}}{n!}+... \\ \ln(1+x) = x - \frac{x^{2}}{2} + \frac{x^{3}}{3} + ... + (-1)^{n+1}\frac{x^{n}}{n!} +... \\ \cos(x) = \frac{e^{ix}+e^{-ix}}{2}=1-\frac{x^{2}}{2!}+\frac{x^{4}}{4!}-\frac{x^{6}}{6!}+...\\ \sin(x) = \frac{e^{ix}-e^{-ix}}{2i}=x-\frac{x^{3}}{3!}+\frac{x^{5}}{5!}+...$$

Regression Least Squares Fitting

#math #regression

$$y_{i}=\alpha + \beta (x_{i} - \bar{x}) + \epsilon_{i} \\ \hat{\alpha}=\bar{y} \\ \hat{\beta}=\frac{s^{2}_{xy}}{s^{2}_{x}}$$

Taylor Series

#math #taylor series

$$y(x)=y(a+u)=y(a)+u\frac{\mathrm{d} y}{\mathrm{d} x}+\frac{1}{2!}u^{2}\frac{\mathrm{d}^2 y}{\mathrm{d} x^{2}}+\frac{1}{3!}u^{3}\frac{\mathrm{d}^3 y}{\mathrm{d} x^{3}}+...$$

Wave Equation

#math #wave equation

$$u_{tt}=c^{2}u_{xx}$$

Bessel Equation

#math #bessel equation

$$x^{2}\frac{\mathrm{d}^{2} y}{\mathrm{d} x^{2}}+x\frac{\mathrm{d} y}{\mathrm{d} x}+(x^{2}-m^{2})y=0 \\ J_{m}(x)=\sum^{\infty}_{k=0}\frac{(-1)^{k} (x/2)^{m+2k}}{k!(m+k)!}$$

Complex Numbers

#math #complex numbers

$$z=x+iy=r(\cos \theta + i \sin \theta)=re^{i(\theta+2n\pi)} \\ z^{*}=x-iy=r(\cos \theta - i \sin \theta)=re^{-i\theta} \\ zz^{*}=|z|^{2}=x^{2}+y^{2}$$

De Moivre's Theorem

#math #complex number #De Moivre

$$(\cos \theta + i \sin \theta)^{n}=e^{in\theta}=\cos n\theta + i \sin n\theta$$

Euler identity

#math #euler

$$e^{i\pi} + 1 = 0$$

Exponential Distribution

#Math #Statistics

$$f(x, \lambda)=\lambda e^{-\lambda x} \\ F(x, \lambda)=1 - e^{-\lambda x}$$

Fisher Skewness

#Math #Statistics

$$\gamma_1 = \frac{{\mu_3 }}{{\mu_2 ^{{3 \mathord{\left/ {\vphantom {3 2}} \right. \kern-\nulldelimiterspace} 2}} }} = \frac{{\mu_3 }}{{\sigma ^3 }}$$

Gamma Distribution

#Math #Statistics

$$\Gamma \left( a \right) = \int\limits_0^\infty {s^{a - 1} } e^{ - s} ds \\ P(x) = \frac{x^{\alpha-1} e^{-frac{x}{\theta}}}{\Gamma(\alpha) \theta^{\alpha}} \\ \mu = \alpha \theta \\ \sigma^{2} = \alpha \theta^{2} \\ \gamma_{1} = \frac{2}{\sqrt{\alpha}} \\ \gamma_{2} = \frac{6}{\alpha}$$

Gaussian Integral

#math #calculus

$${\displaystyle \int _{-\infty }^{\infty }e^{-x^{2}}\,dx={\sqrt {\pi }}.} $$

Laplace Equation

#math #laplace equation

$$u_{xx} = 0$$

Legendre Equation

#math #legendre equation

$$(1-x^{2})\frac{\mathrm{d}^{2} y}{\mathrm{d} x^{2}}-2x\frac{\mathrm{d} y}{\mathrm{d} x}+l(l+1)y=0 \\ P_{l}(x)=\frac{1}{2^{l}l!}(\frac{\mathrm{d}}{\mathrm{d} x})^{l}(x^2-1)^{l}\\ P_{l}(x)=\frac{1}{l}[(2l-1)xP_{l-1}(x)-(l-1)P_{l-2}(x)]$$

Normal Gaussian Distribution

#Math #Statistics

$$X \sim \mathcal{N}(\mu,\sigma^2) \\ f(x)=\frac{1}{\sigma\sqrt{2\pi}}\exp{[-\frac{(x-\mu)^{2}}{2\sigma^{2}}]}$$

Power Series for Complex Variables

#math #complex variables

$$e^{z}=1+z+\frac{z^{2}}{2!}+\frac{z^{3}}{3!}+...+\frac{z^{n}}{n!}+...\\ \sin z=z-\frac{z^{3}}{3!}+\frac{z^{5}}{5!}-...\\ \cos z=1-\frac{z^{2}}{2!}+\frac{z^{4}}{4!}-...\\ \ln (1+z)=1-\frac{z^{2}}{2!}+\frac{z^{3}}{3!}-...\\ (1+z)^{n}=1+nz+\frac{n(n-1)}{2!}z^{2}+\frac{n(n-1)(n-2)}{3!}z^{3}+...$$

Pythagorean Theorem

#math #geometry

$$a^{2}+b^{2}=c^{2}$$

Riemann Hypothesis

#math

$$\zeta (s)=\sum _{n=1}^{\infty }{\frac {1}{n^{s}}}={\frac {1}{1^{s}}}+{\frac {1}{2^{s}}}+{\frac {1}{3^{s}}}+\cdots $$ $$ \zeta (s)=\prod _{p{\text{ prime}}}{\frac {1}{1-p^{-s}}}={\frac {1}{1-2^{-s}}}\cdot {\frac {1}{1-3^{-s}}}\cdot {\frac {1}{1-5^{-s}}}\cdot {\frac {1}{1-7^{-s}}}\cdots$$

Spherical Harmonics Equation

#math #spherical harmonics

$$[\frac{1}{\sin \theta} \frac{\partial}{\partial \theta}(\sin \theta \frac{\partial}{\partial \theta}) + \frac{1}{\sin^{2} \theta} \frac{\partial^{2}}{\partial \phi^{2}}) ] Y^{m}_{l} + l(l+1) Y^{m}_{l}=0 \\ Y^{m}_{l}(\theta,\phi)=\sqrt{\frac{2l+1}{4 \pi} \frac{(l-|m|)!}{(l+|m|)!}}P^{m}_{l}(\cos \theta) e^{im \phi} \times \begin{cases}(-1)^{m} & m\ge 0 \\ 1 & m <0 \end{cases}$$

gaussian process

#math #gaussian process

$$\log p(y|X) \propto -[y^{T}(K + \sigma^{2}I)^{-1}y+\log|K + \sigma^{2}I|] \\ f(X)=\[f(x_{1}),f(x_{2}),...,f(x_{N}))\]^{T} \sim \mathcal{N}(\mu, K_{X,X}) \\ f_{*}|X_{*},X,y \sim \mathcal{N}(\mathbb{E}(f_{*}),\text{cov}(f_{*})) \\ \text{cov}(f_{*})=K_{X_{*},X_{*}}-K_{X_{*},X}[K_{X,X}+\sigma^{2}I]^{-1}K_{X,X_{*}}$$

Average Treatment Effect ATE

#machine learning #causual inference

$$\text{ATE}:=\mathbb{E}[Y(1)-Y(0)]$$

Bound on Target Domain Error

#machine learning #transfer learning

$$\epsilon_{T}(h) \le \hat{\epsilon}_{S}(h) + \sqrt{\frac{4}{m}(d \log \frac{2em}{d} + \log \frac{4}{\delta })} + d_{\mathcal{H}}(\tilde{\mathcal{D}}_{S}, \tilde{\mathcal{D}}_{T}) + \lambda \\ \lambda = \lambda_{S} + \lambda_{T}$$

Conditional Average Treatment Effect CATE

#machine learning #causual inference

$$\tau(x):=\mathbb{E}[Y(1)-Y(0)|X=x]$$

Diffusion Model Forward Process

#machine learning #diffusion

$$q(x_{t}|x_{t-1})=\mathcal{N}(x_{t};\sqrt{1-\beta_{t}}x_{t-1},\beta_{t}I) \\q(x_{1:T}|x_{0})=\prod_{t=1}^{T}q(x_{t}|x_{t-1})$$

Diffusion Model Reverse Process

#machine learning #diffusion

$$p_\theta(\mathbf{x}_{0:T}) = p(\mathbf{x}_T) \prod^T_{t=1} p_\theta(\mathbf{x}_{t-1} \vert \mathbf{x}_t) \\ p_\theta(\mathbf{x}_{t-1} \vert \mathbf{x}_t) = \mathcal{N}(\mathbf{x}_{t-1}; \boldsymbol{\mu}_\theta(\mathbf{x}_t, t), \boldsymbol{\Sigma}_\theta(\mathbf{x}_t, t))$$

Diffusion Model Variational Lower Bound

#machine learning #diffusion

Diffusion Model Variational Lower Bound Loss

#machine learning #diffusion

$$\begin{aligned} L_\text{VLB} &= L_T + L_{T-1} + \dots + L_0 \\ \text{where } L_T &= D_\text{KL}(q(\mathbf{x}_T \vert \mathbf{x}_0) \parallel p_\theta(\mathbf{x}_T)) \\ L_t &= D_\text{KL}(q(\mathbf{x}_t \vert \mathbf{x}_{t+1}, \mathbf{x}_0) \parallel p_\theta(\mathbf{x}_t \vert\mathbf{x}_{t+1})) \text{ for }1 \leq t \leq T-1 \\ L_0 &= - \log p_\theta(\mathbf{x}_0 \vert \mathbf{x}_1) \end{aligned}$$

Domain Adaptation H-Divergence

#machine learning #transfer learning

$$d_{\mathcal{H}}(\mathcal{D},\mathcal{D}^{'})=2\sup_{h \in \mathcal{H}}|\Pr_{\mathcal{D}}[I(h)]-\Pr_{\mathcal{D}^{'}}[I(h)]|$$

Domain-Adversarial Neural Networks DANN

#machine learning #transfer learning

$$\min [\frac{1}{m}\sum^{m}_{1}\mathcal{L}(f(\textbf{x}^{s}_{i}),y_{i})+\lambda \max(-\frac{1}{m}\sum^{m}_{i=1}\mathcal{L}^{d}(o(\textbf{x}^{s}_{i}),1)-\frac{1}{m^{'}}\sum^{m^{'}}_{i=1}\mathcal{L}^{d}(o(\textbf{x}^{t}_{i}),0))]$$

Generative Adversarial Networks GAN

#machine learning #gan

$$\min_{G} \max_{D} V(D,G)=\mathbb{E}_{x \sim p_{data}(x)}[\log D(x)]+\mathbb{E}_{z \sim p_{z}(z)}[\log(1-D(G(z)))]$$

Graph Attention Network GAT

#machine learning #graph #GNN

$$h=\{\vec{h_{1}},\vec{h_{2}},...,\vec{h_{N}}\}, \\ \vec{h_{i}} \in \mathbb{R}^{F} \\ W \in \mathbb{R}^{F \times F^{'}} \\ e_{ij}=a(Wh_{i},Wh_{j}) \\ k \in \mathcal{N}_{i},\text{ neighbourhood nodes}\\ a_{ij}=\text{softmax}_{j}(e_{ij})=\frac{\exp(e_{ij})}{\sum_{k \in \mathcal{N}_{i}} \exp(e_{ik})}$$

Graph Convolutional Networks GCN

#machine learning #graph #GNN

$$H^{(l+1)}=\sigma(\tilde{D}^{-\frac{1}{2}}\tilde{A}\tilde{D}^{-\frac{1}{2}}H^{l}W^{l})\\ \tilde{A}=A+I_{N}\\ \tilde{D}_{ii}=\sum_{j}\tilde{A}_{ij} \\ H^{0}=X \\ \mathcal{L}=-\sum_{l \in Y}\sum^{F}_{f=1} Y_{lf} \ln Z_{lf}$$

Graph Laplacian

#machine learning #graph #GNN

$$L=I_{N}-D^{-\frac{1}{2}}AD^{-\frac{1}{2}} \\ L=U\Lambda U^{T}$$

GraphSage

#machine learning #graph #GNN

$$h^{0}_{v} \leftarrow x_{v} \\ \textbf{for} k \in \{1,2,...,K\} \text{do}\\ \textbf{for} v \in V \text{do} \\ h^{k}_{N_{v}} \leftarrow \textbf{AGGREGATE}_{k}(h^{k-1}_{u}, u \in N(v)); \\ h^{k}_{v} \leftarrow \sigma (W^{k} \textbf{concat}(h^{k-1}_{v},h^{k}_{N_{v}})) \\ \textbf{end} \\ h^{k}_{v}=h^{k}_{v}/||h^{k}_{v}||_{2},\forall v \in V \\ \textbf{end} \\ z_{v} \leftarrow h^{k}_{v} \\ J_{\textbf{z}_{u}}=-\log (\sigma (\textbf{z}_{u}^{T}\textbf{z}_{v})) - Q \mathbb{E}_{v_{n} \sim p_n(v)} \log(\sigma (-\textbf{z}_{u}^{T}\textbf{z}_{v_{n}}))$$

Group Relative Policy Optimization GRPO

#AI #Machine Learning #NLP

Maximum Mean Discrepancy MMD

#machine learning #mmd

$$\textup{MMD}(\mathbb{F},X,Y):=\sup_{f \in\mathbb{F}}(\frac{1}{m}\sum_{i=1}^{m}f(x_{i}) -\frac{1}{n}\sum_{j=1}^{n}f(y_{j}))$$

Model-Agnostic Meta-Learning MAML

#machine learning #meta learning

$$\min_{\theta} \sum_{\mathcal{T}_{i} \sim p(\mathcal{T})} \mathcal{L}_{\mathcal{T}_{i}}(f_{\theta^{'}_{i}}) = \sum_{\mathcal{T}_{i} \sim p(\mathcal{T})} \mathcal{L}_{\mathcal{T}_{i}}(f_{\theta_{i} - \alpha \nabla_{\theta} \mathcal{L}_{\mathcal{T}_{i}} (f_{\theta}) })$$

S-Learner

#machine learning #causual inference

$$\mu(x,w)=\mathbb{E}[Y_{i}|X=x_{i},W=w] \\ \hat{\tau}(x)=\hat{\mu}(x,1)-\hat{\mu}(x,0)$$

T-Learner

#machine learning #causual inference

$$\mu_{0}(x)=\mathbb{E}[Y(0)|X=x],\mu_{1}(x)=\mathbb{E}[Y(1)|X=x],\\ \hat{\tau}(x)=\hat{\mu}_{1}(x)-\hat{\mu}_{0}(x)$$

TransE

#machine learning

$$\mathcal{L}=\sum_{(h,r,t) \in S} \sum_{(h^{'},r^{'},t^{'}) \in S^{'}_{(h,r,t)}} \[ \gamma + d(h + r, t) - d(h^{'} + r^{'}, t^{'}) \]_{+} \\ S^{'}_{(h,r,t)}=\{(h^{'},r,t)|h^{'} \in E\} \cup \{(h,r,t^{'})|t^{'} \in E\} \\ d(h + r, t)=||h + r - t||^{2}_{2}$$

Variational AutoEncoder VAE

#machine learning #VAE

$$\log p_{\theta}(x)=\mathbb{E}_{q_{\phi}(z|x)}[\log p_{\theta}(x)] \\ =\mathbb{E}_{q_{\phi}(z|x)}[\log \frac{p_{\theta}(x,z)}{p_{\theta}(z|x)}] \\ =\mathbb{E}_{q_{\phi}(z|x)}[\log [\frac{p_{\theta}(x,z)}{q_{\phi}(z|x)} \times \frac{q_{\phi}(z|x)}{p_{\theta}(z|x)}]] \\ =\mathbb{E}_{q_{\phi}(z|x)}[\log [\frac{p_{\theta}(x,z)}{q_{\phi}(z|x)} ]] +D_{KL}(q_{\phi}(z|x) || p_{\theta}(z|x))\\$$

Exponential Distribution

#Math #Statistics

$$f(x, \lambda)=\lambda e^{-\lambda x} \\ F(x, \lambda)=1 - e^{-\lambda x}$$

Fisher Skewness

#Math #Statistics

$$\gamma_1 = \frac{{\mu_3 }}{{\mu_2 ^{{3 \mathord{\left/ {\vphantom {3 2}} \right. \kern-\nulldelimiterspace} 2}} }} = \frac{{\mu_3 }}{{\sigma ^3 }}$$

Gamma Distribution

#Math #Statistics

Normal Gaussian Distribution

#Math #Statistics

$$X \sim \mathcal{N}(\mu,\sigma^2) \\ f(x)=\frac{1}{\sigma\sqrt{2\pi}}\exp{[-\frac{(x-\mu)^{2}}{2\sigma^{2}}]}$$

Chatbot close

Bot
Hi TEMP_c922ff37,
How can I help you today?

Send

Navigation

Equation Database

EQUATION LIST

economics

nlp

physics

ai

math

machine learning

statistics