Equivariance and Polynomials

In this section we will define what it means for a function to be equivariant, and what tools are available to create equivariant polynomials. This latter is of great importance as almost any functions can be approximated by polynomials, and that the role of our neural network is to estimate an (equivariant) function.

Equivariance

We say that a function \(f: V \rightarrow V'\) is equivariant to the action of the symmetry element \(g\), represented by \(D(g)\) on it’s input \(x\), if it gives rise to a similar action \(D'(g)\) of its output, i.e:

\[ f(D(g)x) = D'(g)f(x). \]

This principle is illustrated in figure 1.

For a concreate example, consider a 3D rotation of \(\theta\) degrees around the x axis:

\[\begin{split} R_{x}(\theta )=\begin{bmatrix}1&0&0\\0&\cos \theta &-\sin \theta \\[3pt]0&\sin \theta &\cos \theta \\[3pt]\end{bmatrix}, \end{split}\]

and the following functions function:

\[\begin{split} f: \begin{bmatrix} x \\ y \\ z \end{bmatrix} \rightarrow \begin{bmatrix} x^2+y^2+z^2 \end{bmatrix} \end{split}\]

Then \(f\) is equivariant to rotation as:

\[\begin{split} f(R_x(\theta) \vec{x}) = f(\begin{bmatrix} x \\ cos(\theta)y-sin(\theta)z \\ sin(\theta)y+cos(\theta)z \end{bmatrix} ) = \begin{bmatrix} x^2+y^2+z^2 \end{bmatrix} = R'_x(\theta) f(x) = f(x). \end{split}\]

especialy in this case the function is actually invariant to rotations: \(R'_x(\theta) =1 \space \forall \theta \in \mathcal{R}\).

An example of a non equivariant polynomials would be \(p\):

\[\begin{split} p : \begin{bmatrix} x \\ y \\ z \end{bmatrix} \rightarrow \begin{bmatrix} x^2+2(y-3z) \\ y^2x \\ \end{bmatrix}. \end{split}\]

Fig.1 - Principle of equivariance.

Equivariant polynomials

In this section we will discuss the tools to create equivariant polynomials.

We will first show that the compositon of 2 equivariant polynomial is equivariant:

Consider two equivariant function:

\[ f: V_1 \rightarrow V_3, \]

\[ g: V_2 \rightarrow V_3, \]

then:

Addition of equivariant polynomials is equivariant:

Indeed, \(f+g\) is equivariant as:

\[ D_3(g)(f(x)+g(x)) = D_1(g)f(x)+D_2(g)g(x) = f(D_1(g)x)+g(D_2(g)x). \]

Composition of equivariant polynomials is equivariant:

Indeed, \(f \circ g\) is equivariant as:

\[ D_3(g)(f(g(x))) =f( D_2(g) g(x)) = f(g(D_1(g)x)). \]

The Tensor Product of two polynomial is equivariant:

It is denoted \(f \otimes g\). The Tensor Product section is dedicated to it.