1. Introduction
In this tutorial, we’ll review binary, discrete and continuous inputs for neural networks: what they are, how they are represented, and in what ways they differ.
2. Definitions
Neural networks are algorithms created explicitly to simulate biological neural networks. Consequently, a neural network consists of interconnected neurons which process data through three or more layers. Neural networks are mostly used in pattern recognition tasks, predictive modeling tasks, computer vision, and many other applications.
The input layer is the initial layer in every neural network, comprising of neurons that accept input data. This layer accepts data and converts it to an output. In light of this, a neural network cannot function without data. Without data, the neural network cannot learn patterns or process anything.
The input data that is fed into neural networks come in different formats depending on the information that is represented in the data. We’ll discuss three of these formats in this tutorial:
3. Binary Inputs
When we talk of binary inputs, we’re talking of inputs in the format zeros and ones. In computer science, data in the form of zeros and ones are known as binary numbers. These numbers can only take on two values, which are 0 and 1. For neural networks, data containing these binary numbers are compiled in a matrix form and fed into the input layer.
Binary inputs for neural networks will usually have some information encoded in them. For example, let’s say we have a dataset consisting of answers to several questions. In this dataset, we have the answers as yes and no. Hence, the yes and no answers are encoded as binary values where yes is 1 and no is 0. We’d refer to this as binary input:
Variable 1
Variable 2
Variable 3
Variable 4
Variable 5
0
1
0
1
0
1
0
0
1
1
0
0
0
0
0
1
0
1
0
0
0
1
0
0
0
1
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
1
0
0
0
0
3.1. Characteristics
Alternatively, binary inputs can be used as a representation of categorical variables. For instance, the variable gender with values male and female can be represented as binary input. That is, a male is represented by 1 and a female with zero, or vice-versa. In light of this, binary inputs cannot be divided into sub-parts. For example, we cannot have half of zero as a value.
4. Discrete Inputs
Discrete inputs refer to numeric data that are finite and countable. These inputs can only take on certain values. For example, in our dataset of survey answers, let’s suppose we have the number of shoes a participant owns as part of the dataset. In this instance, the value associated with the number of shoes is a whole number. This means it can only take on values within the range of whole numbers. We’d say this is an example of a discrete input:
Variable: Number of shoes
36
60
19
21
60
45
24
18
44
72
4.1. Characteristics
Similar to binary input, discrete inputs can also be used to represent categorical data. Specifically, categorical data with a countable number of values. Most importantly, discrete inputs are usually non-negative numbers since they are countable. In addition, discrete inputs are immeasurable and can remain constant over time. For instance, in our example with the number of shoes, we cannot measure the number of shoes that a participant has. We can only count the number of shoes.
5. Continuous Inputs
On the contrary, continuous inputs refer to input data that has an infinite number of values. These inputs are usually measurable and subject to change over time. For example, suppose we have the number of hours spent on answering the questions as a variable in the dataset. This data can take on any value and is subject to change. We refer to this as continuous inputs:
Variable Hours Spent
1.3
5.0
10
2.3
6.5
7.9
4.0
2.1
5
4
5.1. Characteristics
Another key point with continuous input is that the values can be subdivided into parts. In particular, we’re referring to data that has additional parts in the form of decimals. This is made possible due to the fact that continuous inputs are usually measurable, hence precision is key. The subdivision of values adds precision to continuous inputs.
6. Differences and Similarities
The differences and similarities between binary, discrete and continuous inputs can be summed up this way:
Binary inputs
Discrete inputs
Continuous inputs
Constant over a time interval
Is constant over a time interval
Can change over a time interval
Data cannot be subdivided into parts
Cannot be subdivided into parts
Can be subdivided into parts
Data cannot be measured
Cannot be measured
Data is measurable
7. Conclusions
In this tutorial, we reviewed binary, discrete and continuous inputs of neural networks. Firstly, binary inputs can only take on two values: zeros and ones. Secondly, discrete inputs can take on a countable number of values, usually more than one. Lastly, continuous outputs have an infinite number of values. To sum up, binary, discrete and continuous inputs describe various representations of data that are usually fed into neural networks.