1. Introduction

In this tutorial, we’ll explore the concept of convolution, distinguishing between linear and circular convolution.

2. Linear Convolution: Intuitive Understanding and Mathematical Model

Convolution combines two functions to produce a third function, showing how one modifies the other.

In this section, we’ll offer an intuitive understanding of linear convolution before presenting its mathematical model.

2.1. Intuitive Understanding: A Bakery’s Delivery Schedule

Here, let’s use a bakery’s delivery schedule to understand the concept of linear convolution.

If we pay a dollar on monday, we get a loaf of bread on tuesday and wednesday in that order.  The next two figures illustrate this schedule and its timeline.

Bakery's Schedule

Schedule In Timeline

Here, x(t) is the amount we pay to the bakery (input), y(t) is the number of loaves we receive each day (output), and h(t) is the bakery’s delivery schedule (impulse response).

If we pay a dollar on the first and sixth days, we receive a loaf on the second, third, seventh, and eighth days based on the bakery’s schedule. The next figure illustrates this:

New Schedule Timeline

Regardless of when we pay, the bakery’s response remains consistent. This indicates that the bakery’s policy is unaffected by changes in time, making it a Time-Invariant system.

Let’s imagine another scenario. If the buyer pays one dollar to the baker two days in a row, what happens?

Stacking Property

We discover that the loaves of bread stack upon one another and scale the output as we slide through the baker’s impulse response. The next figure also illustrates this stacking property:

Linear System

Here, the buyer paid two dollars in one day. The output is stacked and scaled based on the bakery’s impulse response. This stacking and scaling property define a Linear system. Combining these properties, we conclude our bakery’s schedule is a Linear Time-Invariant (LTI) system.

Let’s consider another bakery with an LTI system and a different delivery policy: paying a dollar today gets you three loaves tomorrow, two loaves the next day, and one loaf on the third day:

New Bakery's Schedule

Here is how this LTI system will look like on a timeline:

New Bakery's Policy Timeline

This is another LTI system timeline:

Another Schedule Timeline

2.2. Forming and Interpreting the Vectors and Matrices

Let us capture all this information using vectors, matrices, and linear algebra.

Algebra

We know that the matrix multiplication of x(t) and h(t) will yield y(t).

Using our timeline analogy and the concepts of sliding, stacking, and scaling, we show how h(t) and y(t) behave as we shift the input x(t) over time.

Starting with a simple input vector x(t) = [1, 0, 0, 0, 0, 0, 0], here is how the matrix will look like:

First Step Matrix

If we shift the input x(t) downward by one step, the impulse response h(t) also shifts downward by one step, resulting in the same output but shifting downward by one step. This is shown in the next figure:

Second Step Matrix

if we keep repeating this pattern for the other inputs, we obtain these patterns for h(t) and y(t):

Third Step Matrix Fourth Step Matrix Fifth Step Matrix Sixth Step Matrix Seventh Step Matrix

within each column of h(t), we have its shifted version, as shown in the next figure:

Column Shifting Matrix

Along the row, we also get a copy of h(t), but it’s a reflected and shifted version of h(t) as shown next:

Row Shifting Matrix

From our understanding of the pattern of the last two figures, we can generalize our matrix now as this:

matrix

From matrix multiplication, we understand that the output vector is the dot product of the input vector and the row vector of the reflected and shifted impulse response. We express the output vector as a dot product of x(t) and h(t):

Matrix Dot Product

The negative sign at A reflects the impulse response, while the negative sign at B shifts it.

Let’s illustrate how x(t) \cdot h(t) translates to  x(t) \cdot h(-(t-t))

For the first row, t = 0, we have:

Row1 Dot Product

For the third row, t = 0, 1 , 2, we have:

Row3 Dot Product

As time increases, so does the shift, indicating that the output is a function of the shift.

To avoid having h(-(t-t)) in a generalized form, we replace the input time variable with k and maintain the output time variable as t.

With this, we obtain the generalized expression for the output vector:

matrix

We can express the dot product explicitly with sigma notation:

y(t) = \sum_{k=-\infty}^{\infty} x[k] h[t - k]

To represent the system with a continuous function, we convert the sigma sum into an integral:

y(t) = \int_{-\infty}^{\infty} x(k) h(t - k) dk.

2.3. Mathematical Definition

Linear convolution combines two LTI sequences or time functions to produce a third sequence or function. It involves multiplying each sample of one function with the corresponding sample of the other function shifted by k, then summing up the results over all possible values of k .

In discrete LTI systems, the formula for convolution is:

    [y[n] = x[n] * h[n] = \sum_{k=-\infty}^{\infty} x[k] h[n - k]]

In continuous-time LTI systems, it becomes:

    [y(t) = x(t) * h(t) = \int_{-\infty}^{\infty} x(k) h(t - k) dk]

The asterisk (*) denotes the convolution operator.

Now, let’s explore the various methods of computing linear convolution.

3. Linear Convolution: Methods of Computation

This article focuses on the short data sequences. Let’s take a look at an example:

3.1. An Example

We are to determine the response of an LTI system whose input x(n) and impulse response h(n) are given as:

\mathbf{x(k)} = \{ 1, 2, 3, 2 \}\underset{\text{x(0) = } 1}{\xleftarrow{\hspace*{1cm}}}, and

\mathbf{h(k)} = \{ 1, 2, 1,-1 \}\underset{\text{h(0) = } 2}{\xleftarrow{\hspace*{1cm}}}.

To solve this, we note the following:

  • Let the length of the first sequence be L and the second sequence is M. Then, the length of the output sequence y(n) is L + M - 1
  • If the first sequence starts at n = n and the second at n = m, the output sequence will begin at n + m and end at (n + m) + (L + M - 2)

For our example:

  • L = 4, and M = 4, then, y(n) = 4 + 4 - 1 = 7. y(n) will have a length of 7
  • Also, n = 0, and m = -1, then, y(n) will start at n + m = 0 + (-1) = -1, and ends at (n + m) + (L + M - 2) = (0 + (-1) + (4 + 4 - 2)) = -1 + 6 = 5

Now, let’s investigate the graphical method.

3.2. The Graphical Method

Using the graphical method, we aim to visually overlap and shift sequences to compute their convolution by summing the products of their overlapping values.

We present the steps to follow in Table 1:

Steps

Process

1

Obtain the reverse sequence h(-k)

2

Shift h(-k) by n samples to get h(n-k). If n >= 0, h(-k) will be shifted to the right by n samples. But if n < 0, h(-k) will be shifted to the left by n samples

3

Perform the convolution sum, i.e., the sum of x(k) and h(n-k) to obtain y(n)

4

Repeat steps 1 to 3 for the next convolution value y(n)

In solving the problem using the graphical method, we first draw the graph of both sequences:

Two Inputs Graphs

To obtain  y(0) at n = 0, we reverse h(k) to h(-k). Then, we multiply x(k) by h(-k) and sum the products. The next plots show how we obtain y(0):

First Output Graph

Our output y(n) starts at n = -1, so we need to obtain y(-1). To do this, we compute h(-1-k), shifting h(-k) leftward. The next plots show this operation:

Second Output Graph

Next is to compute the output y(1) for n = +1 by solving for h(1-k) shifting h(-k) rightward. This operation is shown in the next plots:

Third Output Graph

For the output y(2), its graph looks like this:

Fourth Output Graph

Here is the plot for y(3):

Fifth Output Graph

for y(4):

Sixth Output Graph

and here is for y(5):

Seventh Output Graph

The output sequence and its graph is shown next:

\mathbf{y(n)} = \{ 1, 4, 8, 9, 5, -1, -2 \}\underset{\text{y(0) = } 4}{\xleftarrow{\hspace*{1cm}}},

Final Output Graph

Now, let’s attempt to solve this same given problem using the tabular method.

3.2. The Tabular Method

Using the tabular method, we systematically align and multiply sequences to compute their convolution step by step.

Table 2 provides the steps to follow:

Steps

Process

1

List the index k covering a sufficient range

2

List the input x(k)

3

Obtain the reversed sequence h(k), and align the rightmost element of h(n-k) to the leftmost element of x(k)

4

Cross-multiply and sum the non-zero overlap terms

5

Slide h(n-k) to the right by one step

6

Repeat step 4. Stop if all output values are zero or if required

Let’s now solve the given problem by following the steps, we show the result in Table 3:

k

-3

-2

-1

0

1

2

3

4

5

6

y(n)

x(k)

1

2

3

2

h(k)

1

2

1

-1

h(-k)

-1

1

2

1

h(-1-k)

-1

1

2

1

y(-1) = x(k)h(-1-k) = 0+0+0+1 = 1

h(0-k)

-1

1

2

1

y(0) = x(k)h(0-k) = 0+0+2+2 = 4

h(1-k)

-1

1

2

1

y(1) = x(k)h(1-k) = 0+1+4+3 = 8

h(2-k)

-1

1

2

1

y(2) = x(k)h(2-k) = -1+2+6+2 = 9

h(3-k)

-1

1

2

1

y(3) = x(k)h(3-k) = 0-2+3+4 = 5

h(4-k)

-1

1

2

1

y(4) = x(k)h(4-k) = 0+0-3+2 = -1

h(5-k)

-1

1

2

1

y(5) = x(k)h(5-k) = 0+0+0-2 = -2

Here is the output sequence:

\mathbf{y(n)} = \{ 1, 4, 8, 9, 5, -1, -2 \}\underset{\text{y(0) = } 4}{\xleftarrow{\hspace*{1cm}}},

This is the same as that of the graphical method.

Let’s look at the matrix method.

3.3. Matrix Method

We convert sequences into matrices using the matrix method to compute their convolution through matrix multiplication efficiently.

Table 4 provides the steps to follow:

Steps

Process

1

Arrange x(k) as a column

2

Arrange h(k) as a row

3

Obtain the element of a two-dimensional array by multiplying the corresponding row elements with the column elements

4

The sum of the diagonal element gives the samples y(n)

Let’s solve the given problem using these steps:

Matrix Computation Method

Here is the output sequence:

\mathbf{y(n)} = \{ 1, 4, 8, 9, 5, -1, -2 \}\underset{\text{y(0) = } 4}{\xleftarrow{\hspace*{1cm}}},

The same as that of the graphical and tabular methods.

4. Circular Convolution

Using circular convolution, we compute the convolution of two sequences assuming they are periodic, resulting in an output of the same length.

The mathematical formula for circular convolution is:

    [  y(m) = x(n) \circledast h(n) = \sum_{n = 0}^{N-1} x(n) h((m - n))_N]

for   m = 0, 1, 2, ..., N-1

Let’s explore the short sequence methods.

4.1. Concentric Circle Method

Using the concentric circle method, we visually align and rotate sequences to compute their circular convolution efficiently.

Table 5 provides the steps to follow:

Steps

Process

1

Draw two concentric circles, one will represent x(n), and the other h(n)

2

Place the elements x(n) on the inner circle and h(n) on the outer circle at corresponding positions

3

Align the first element of x(n) with the first element of h(n)

4

For the current alignment, multiply corresponding elements of x(n) and h(n), then sum these products to het y(n)

5

Rotate the outer circle h(n) one position clockwise

6

Repeat the multiplication and summation process for each new alignment to get other output y(n)

7

Continue rotating the outer circle and computing sums until you have completed N rotations, generating all elements of y(n)

8

The sequence y(n) represents the circular convolution result

We illustrate this method with an example:

4.2. An Example

We are to determine the response of an LTI system whose input x(n) and impulse response h(n) are given as:

\mathbf{x(k)} = \{ 0, 1, 2, 3 \}\underset{\text{x(0) = } 0}{\xleftarrow{\hspace*{1cm}}}, and,

\mathbf{h(k)} = \{ 2, 1, 3, 4 \}\underset{\text{h(0) = } 2}{\xleftarrow{\hspace*{1cm}}}.

Here’s the solution.

Given sequences x(n) and h(n) with length N = 4, the equation is:

y(m) = x(n) \circledast h(n) = \sum_{n = 0}^{3} x(n) h((m - n))_4

First, we draw two circles, one for x(n) and the other for h(n), plotting their elements in an anti-clockwise direction:

Two Input Circles

To calculate y(0), we substitute m = 0 into our circular convolution equation:

y(0) = x(n) \circledast h(n) = \sum_{n = 0}^{3} x(n) h((0 - n))_4

The sequence h((0 - n))_4 is the circular folding of h(n), obtained by plotting h(n) clockwise. To obtain y(0), we plot x(n) and h(-n) on two concentric circles, with x(n) on the inner circle and h(n) on the outer circle and then multiply and sum each point. The next figure shows this process:

First Concentric Circle

To obtain y(1), we shift h(-n) anti-clockwise by one sample. We then plot the new sequence clockwise and apply multiplication and summation, as shown in the next figure:

Second Concentric Circle

Here is the output y(2):

Third Concentric Circle

And here is for output y(3):

Fourth Concentric Circle

Here is the resultant output sequence y(m):

\mathbf{y(n)} = \{ 13, 19, 17, 11 \}\underset{\text{y(0) = } 13}{\xleftarrow{\hspace*{1cm}}}.

Now, let’s look at the matrix method.

4.3. Matrix Method

We convert sequences into circulant matrices using the matrix method to compute their circular convolution through matrix multiplication efficiently.

Table 6 provides the steps to follow:

Steps

Process

1

Define the sequence x(n) and h(n), each of length N

2

Construct a circulant matrix H using the sequence h(n)

3

Populate the circulant matrix H such that each row is a cyclic permutation of h(n)

4

Arrange the sequence x(n) as a column vector X

5

Perform the matrix multiplication Y = H . X

6

The resulting vector Y represents the circular convolution y(n)

7

The sequence y(n) is the output of the circular convolution

We illustrate this method with the same example as before.

\mathbf{x(k)} = \{ 0, 1, 2, 3 \}\underset{\text{x(0) = } 0}{\xleftarrow{\hspace*{1cm}}}, and,

\mathbf{h(k)} = \{ 2, 1, 3, 4 \}\underset{\text{h(0) = } 2}{\xleftarrow{\hspace*{1cm}}}.

The matrix will have this format:

Circulant Matrix

Let’s fill in the sequences into the matrix, we have:

The Filled Matrix

Now, let’s perform matrix multiplication and summation:

Matrix Computation Results

Here is the output sequence:

\mathbf{y(n)} = \{ 13, 19, 17, 11 \}\underset{\text{y(0) = } 13}{\xleftarrow{\hspace*{1cm}}},

It’s the same as the concentric circle method.

5. Interplay Between Linear and Circular Convolution.

In this section, we demonstrate how linear convolution extends sequences and how circular convolution handles periodic sequences efficiently, showcasing their interplay and equivalence under specific conditions.

Let’s perform the following operations on the sequences \mathbf{x(n)} = \{ 1, 2, 3, 1 \} and \mathbf{h(n)} = \{ 1, 1, 1 \} to illustrate the interplay between linear and circular convolution:

  • Linear convolution
  • Circular convolution
  • Converting linear convolution into circular convolution
  • Linear convolution using circular convolution

To perform linear convolution with x(n) = \{ 1, 2, 3, 1 \} and h(n) = \{ 1, 1, 1 \}, the lengths are L = 4 and M = 3, giving N_{\text{lin}} = L + M - 1 = 6. Using the matrix method, the output is \mathbf{y(n)} = \{ 1, 3, 6, 6, 4, 1 \}.

For circular convolution, both sequences must be the same length N_{\text{cir}}. Since x(n) has length 4 and h(n) has length 3, we append a zero to h(n), making \mathbf{h(n)} = \{ 1, 1, 1, 0 \}. Using the matrix method, the output is \mathbf{y(n)} = \{ 5, 4, 6, 6 \}.

To convert linear convolution to circular convolution, subtract N_{\text{cir}} from N_{\text{lin}}: 6 - 4 = 2. The value 2 means that we will add the last two elements of the linear convolution sequence to the first two elements as illustrated in the next figure:

Linear To Circular

To perform linear convolution using circular convolution, we append zeros to both sequences to make them equal in length. First, we determine the output sequence’s length, N_{\text{lin}} = L + M - 1 = 6. This means that we append M - 1 = 2  zeros to x(n) and L - 1 = 3 zeros to h(n) to make each sequence a length of 6.

Thus, \mathbf{x(n)} = \{ 1, 2, 3, 1, 0, 0 \} and \mathbf{h(n)} = \{ 1, 1, 1, 0, 0, 0 \}.

Using the circular matrix method, the output is \mathbf{y(n)} = \{ 1, 3, 6, 6, 4, 1 \}, matching the linear convolution result.

6. Differences Between Linear and Circular Convolution

Based on all our discussion in this article, here are the differences between linear and circular convolution:

Linear Convolution

Circular Convolution

It considers the full length of the resulting sequence

It warps around the sequence, assuming periodicity

Its output length is N + M -1 if x(n) and h(n) is of length N and M respectively

Its output has the same length as x(n) and h(n)

The result is not periodic

The result is periodic with period N

It requires zero padding to avoid edge effects

It doesn’t need zero padding as it is periodic

Convolution in the time domain corresponds to multiplication in the frequency domain with zero padding to the length of N + M – 1

Convolution in the time domain corresponds to multiplication in the frequency domain without zero padding

Direct computation is O(NM), but can use FFT for O((N + M)log(N + M))

Efficiently computed using FFT with complexity O(NlogN)

Requires zero padding before applying the DFT to avoid aliasing

Directly corresponds to the multiplication of DFTs of sequences

Exists if the sequences are invertible, but may not be straightforward

Circular deconvolution can be applied if the sequences are invertible in the circular sense

Typically used in applications where the length of the resulting sequence matters, like in analogue signal processing and filtering

Often used in digital signal processing, especially in FFT-based methods, where computational efficiency is crucial

7. Conclusion

In this article, we analyzed the intuition behind linear and circular convolution, their mathematical models, and various types. By focusing on short sequences, we provided examples to clarify these concepts. Finally, we listed the key differences between linear and circular convolution.