
Outer Product of Two Vectors
When we talk about the outer product of two vectors, we're referring to the resulting matrix in which every entry is the product of an element from the first vector and an element from the second.
This operation is often recognized by different names such as the tensor product.
If our two vectors possess dimensions of n and m respectively, their outer product would yield an n x m matrix.
$$ \vec{a} \otimes \vec{b} = \begin{pmatrix} a_1 \\ a_2 \\ \vdots \\ a_n \end{pmatrix} \otimes \begin{pmatrix} b_1 \\ b_2 \\ \vdots \\ b_m \end{pmatrix} = \begin{pmatrix} a_1 \cdot b_1 & a_1 \cdot b_2 & ... & a_1 \cdot b_m \\ a_2 \cdot b_1 & a_2 \cdot b_2 & ... & a_2 \cdot b_m \\ \vdots \\ a_n \cdot b_1 & a_n \cdot b_2 & ... & a_n \cdot b_m \end{pmatrix} $$
In the outer product, the two vectors can have varying component counts (n≠m); they don't necessarily need to match (n=m).
On a broader scale, the outer product of two tensors (multi-dimensional arrays of numbers) yields a tensor.
The outer product distinguishes itself from other linear algebra operations like the scalar product and the Kronecker product. Unlike the scalar product, which merges two vectors to produce a scalar, the outer product creates a matrix from two vectors. It also deviates from the Kronecker product, which forms a matrix from two matrices. Lastly, it should not be confused with standard matrix multiplication, which is carried out row-by-column.
What's its purpose?
The outer product is a key concept within linear algebra and finds application in various fields like quantum physics, signal processing, and image compression.
A simple example
Take the two-dimensional vectors [1, 2] and [3, 4].
$$ \vec{v} = \begin{pmatrix} 1 \\ 2 \end{pmatrix} $$
$$ \vec{w} = \begin{pmatrix} 3 \\ 4 \end{pmatrix} $$
The outer product of these two vectors results in a 2x2 matrix, calculated as follows:
$$ \vec{v} \otimes \vec{w} = \begin{pmatrix} 1 \\ 2 \end{pmatrix} \otimes \begin{pmatrix} 3 \\ 4 \end{pmatrix} = \begin{pmatrix} 1 \cdot 3 & 1 \cdot 4 \\ 2 \cdot 3 & 2 \cdot 4 \end{pmatrix} $$
So, the outer product of the vectors [1, 2] and [3, 4] results in the matrix [[3, 4], [6, 8]].
$$ \vec{v} \otimes \vec{w} = \begin{pmatrix} 3 & 4 \\ 6 & 8 \end{pmatrix} $$
The final result is a matrix.
In this case, the matrix has the same number of rows and columns because the vectors are of the same size (n=m=2). However, this isn't a rule - in the outer product, the vectors can have different dimensions.
Properties of the Outer Product
The outer product of vectors fulfills the following properties:
- The transpose of the outer product of two vectors u and v is equal to the outer product of the vectors v and u swapped
In other words, if you swap the vectors in the outer product and then take the transpose, you'll get the same result as if you took the transpose of the original outer product. $$ (\vec{u}⊗\vec{v})^T=(\vec{v}⊗\vec{u}) $$ - The distributive property with respect to vector addition
In simpler terms, this means that if you add two vectors, v and w, and then calculate the outer product with a third vector, u, the result will be the same as if you separately calculated the outer product of v with u and of w with u, and then added these two results. $$ (\vec{v}+\vec{w})⊗\vec{u}=\vec{v}⊗\vec{u}+\vec{w}⊗\vec{u} $$ $$ \vec{u}⊗(\vec{v}+\vec{w})=\vec{u}⊗\vec{v}+\vec{u}⊗\vec{w} $$ - Linearity with respect to multiplication by a scalar
In particular, multiplying a scalar by the outer product of two vectors k(v⊗u) is equivalent to multiplying the scalar by one of the vectors before calculating the outer product i.e., (kv)⊗u or v⊗(ku). In other words, scalar multiplication can be "pulled inside" the outer product operation. $$ k(\vec{v}⊗\vec{u})=(k\vec{v})⊗\vec{u}=\vec{v}⊗(k\vec{u}) $$ - The associative property of the outer product
When you perform the outer product between three vectors u, v and w, the order in which you perform the operations doesn't change the final result. In other words, you can first calculate the outer product between u and v and then between the result and w, or first calculate the outer product between v and w and then between u and the result, obtaining the same final result. $$ (\vec{u}⊗\vec{v})⊗\vec{w} = \vec{u}⊗(\vec{v}⊗\vec{w}) $$
Differences between the Outer and Cross Product
The outer and cross product aren't the same, as they reference two distinct mathematical operations between vectors.
- The outer product takes in two vectors of any dimension and outputs a matrix with all the possible products of the components of the two vectors.
- The cross product is specific to three-dimensional vectors. It takes in two vectors and produces a third, orthogonal to the plane formed by the initial two vectors.
Generally, in mathematics, "outer product" describes a binary operation that gives a result not belonging to the same set as the operands.
Although, in linear algebra, "outer product" is sometimes used synonymously with "cross product". This is because the cross product is a binary operation where the operands are two vectors of a plane and the result is an orthogonal vector that does not lie in the same plane. Therefore, you can consider the cross product as a particular case of the outer product. However, generally speaking, the two terms do not denote the exact same operation.
In summary, the outer and cross product are two unique mathematical operations involving vectors, each with distinct objectives and outcomes.