Neural Networks and Machine Learning

Have you ever heard about machine learning and neural networks? For some, creating a neural network seems like something very confusing, but in fact, they are not so difficult to create. How are they made? Let's try to independently create a feedforward neural network, which is also called a multilayer perceptron.

What is a neural network?

An artificial neural network (ANN) is understood as a mathematical model (including its software or hardware implementation), which is built and operates on the principle of functioning of biological neural networks - we are talking about neural networks of nerve cells of living organisms.

Simply put, ANN can be called a kind of 'black box' that turns input data into output data. If we look at this from the point of view of mathematics, then we are talking about how to map the space of input X-features into the space of output Y-features: X → Y. Thus, we need to find some F-function that can perform this transformation. At the first stage, this information is sufficient as a basis.

The classical definition tells us that a neural network is a certain sequence of neurons connected by synapses. If the program has the structure of a neural network, it becomes possible at the machine level to analyze the input data while storing the result.

That is, neural networks and data for neural networks are a simplified model of a biological analog. Some experts, talking about neural networks, remember the human brain. Yes, this is close to the truth, but the human brain is overly complex, so this is a very approximate comparison because we are not (yet) able to recreate its mechanisms in full even with the help of modern technologies. As a result, a neural network is better described as a program that is based on how the brain works.

A neural network is a bundle of neurons. Each of these neurons receives data, processes it, and then transmits it to another neuron. And each neuron processes signals the same way. But how, then, do we get different results? Synapses are responsible for this, connecting neurons to each other. Each neuron is capable of having multiple synapses that attenuate or amplify the signal. Neurons are able to change their characteristics over a period of time. By the way, having correctly chosen the parameters of the synapses, we will be able to obtain the correct results of converting the input information at the output.

Where are neural networks used?

Neural networks are used to solve many different problems. If we are talking about simple projects, then an ordinary computer program can cope with them, if we talk about complicated tasks that require solving equations and forecasting, a computer program that supports static processing methods is used. There are also quite complex tasks, the same pattern recognition. A different approach is needed here because in a person's head all these processes take place unconsciously (when recognizing and memorizing images, a person does this, so to speak, on his own, that is, he does not control the corresponding processes in the brain).

Types of the neural network:

Single Layer Neural Network

Multi-Layer Neural Network

What role does an artificial neuron play?

In our article, we will not go into the lyrics and talk about the structure of a biological neuron in the context of its connection with an artificial model. Better get down to business right away.

An artificial neuron is a weighted sum of the vector values ​​of the input elements. This sum is transferred to the non-linear activation function f:

Let’s see the formula of function f.

Z=f(y), y= w0.x0+w1x1 +...............+wm-1.xm-1

Neurolay

One neuron can turn the input vector into one point, but by condition, we want to get several points, since the output Y can have an arbitrary dimension, which is determined only by the situation (one output for XOR, ten outputs to determine whether it belongs to one of ten classes, and so on). How do you get n points? In fact, everything is simple: to get n output values, you need to use not one neuron, but n. As a result, n different weighted sums from X will be used for each element of the output Y.

As a result, we will come to the following ratio:

Let's take a close look at it. The above formula is nothing more than the definition of matrix-vector multiplication. An online matrix multiplication calculator will help you to solve problems in a very convenient and easy way, which is very time-saving. Indeed, if we take a matrix W of size n by m and multiply it by X of dimension m, then we get another vector value of n-dimension, that is, just what we need.

Thus, we can write a similar expression in a more convenient matrix form.

But the resulting vector is an inactivated state (intermediate, non-output) of all neurons, and in order to get an output value, each non-activated value must be fed to the input of the above activation function. The result of its application will be the output value of the layer.

A sequence of neural layers is often used to train a neural network deeper and more formalize the available data. That is why, in order to get the final output vector, you need to do the above operation a couple of times in a row in the direction from one layer to another. As a result, for the 1st layer, the input vector will be X, and for the subsequent ones, the input will be the output of the previous layer.

Activation function

This is a function that adds nonlinearity to a neural network. As a result, neurons will be able to mimic any function relatively accurately.

To write a code you’ll need to keep in the mind the following:

Vector

Matrix (each layer includes weights of the matrix)

Neural Network

Note that an online vector cross product calculator will help you to solve your calculations.