As part of the DAM learning activities I went thought the Neural Networks Machine Learning technique.
WHAT IS NEURAL NETWORK?
Neural Network is currently considered as one of the most advanced Machine Learning algorithms. It is used in a lot of Artificial Intelligence projects such as AlphaGo (the algorithm designed by Google who recently beat one of the world top player at the game Go).
Neural Network or Artificial Neural Network (ANN) is classified as a black-box algorithm as it is quite difficult to interpret the model it builds. It has been designed as a reflection of the human brain activity. Its objective is to create a network of interconnected neurons that will send signals to its neighbours if it receives an activation signal (based on a threshold).
There are 3 main characteristics for an ANN:
- An activation function that will trigger the broadcasting of a neuron to its neighbours. There are different types of activation functions depending on the type of the data: binary, logistic, linear, Gaussian etc.
- A network architecture that specifies the number of internal neurons and the number of internal layers. Adding more internal neurons and layers makes the model more complex. This can be necessary for tackling complex data sets but it increases the difficulty to interpret the model.
- The training algorithm which will specifies the weight to be applied to every neuron connection.
A Neural Network is also defined by its input and output neurons. An input neuron is assigned to each feature of the data set and an output neuron is assigned to every possible values of the outcome variable.
After going through the videos from the learning activities and after performing some researches about this algorithm I tried to run an ANN on the same data set as for the K Nearest Neighbours (credit scoring).
In Knime the node for running an ANN is called Multilayer Perceptron Predictor. This node requires 2 inputs: a training model and a normalized test set. The training model is defined through a node called RProp MLP Learner that takes the normalized training set as input.
I run several ANN with different number of internal layers and neurons and compared their model afterwards:
- 1 internal layer and 10 internal neurons
- 1 internal layer and 20 internal neurons
- 3 internal layers and 10 internal neurons
- 3 internal layers and 20 internal neurons
- 5 internal layers and 10 internal neurons
- 5 internal layers and 20 internal neurons
Before looking at the accuracy of these different models I thought that having more internal layers would have improved significantly the performance of the model. For this particular data set I was wrong. The performance for the model with 5 layers were less accurate than with the 1 layer one. From the different tests I did the best model is the one with only 1 internal layer and 20 internal neurons. Compared to the first model the accuracy has increased by 2% which is remarkable as the only thing I did was to change some settings.
For this topic the learning process was different from the KNN one. Here I had to learn by myself the Neural Network algorithm.
What did I do?
I had to do some research on this technique and find different type of materials. First I started to look for some introductory documents that can explain what it is about in simple ways. Then when I got a better understanding of it I started to look at some more advanced books that go into deeper details. I found one that was related to Machine Learning but in R. Even if I knew it isn’t the main tool recommended for this subject I decided to still read it as I am quite familiar with R. This has helped me to understand how to run an ANN step by step and I was able to create easily the same workflow in Knime afterwards.
What did I think and feel about what I did? Why?
I picked Neural Network as I knew it is currently one of the most advanced Machine Learning technique.
I was quite surprised by the simplicity and the easiness to run a K Nearest Neighbour model during the first class. As I said in my previous post I didn’t feel capable of running any Machine Learning algorithm before this class. So when I got the choice I picked one of the hardest to see if I will go through a similar experience even if it would be more challenging.
Even if the theory behind ANN is quite complex, the actual level of knowledge required to be able to run it is again much lower that I would have imagine at the beginning. It was again a big surprise for me.
What were the important elements of this experience and how do I know they are important? What have I learned?
I am really amazed by the fact that in such a short time I have been able to run some very complex Machine Learning algorithms. This experience has confirmed what I discovered during my first try with KNN: Machine Learning isn’t very complicated at a practical level. It requires some good understanding of the key concepts for each technique but you don’t need to understand all the theory behind. So my key learning for these past few weeks is the fact that I am actually already smart enough to start my journey on the Machine Learning field.
How does this learning relate to the literature and to industry standards?
Neural Network is one of the “hottest” technique at the moment. Every time I heard about an innovative Artificial Intelligence project I heard about Neural Network or Deep Learning. I think in the coming years there will be more and more application of this algorithm in Data Science projects.
Gorunescu, F., 2011, Data Mining Concepts, Models and Techniques, Springer
Lantz B., 2013, Machine Learning with R, Packt Publishing
- DAM Portfolio – Summative Reflection
- DAM Portfolio – K Nearest Neighbor (KNN)
- DAM Portfolio – Data preparation in Knime
- DAM Portfolio – CRISP-DM
- DAM Portfolio – Machine Learning Evaluation