Artificial neural networks for classification
Last updated
Was this helpful?
Last updated
Was this helpful?
The next step after the exploration of a custom hand-made classifier and various machine-learning techniques is to design an artifical neural network to do the job.
Neural networks are a powerful method for solving regression or classification problems.
Before we dive into creating a classifier using artificial neural networks, we need to get familiar with one of the big frameworks for training and evaluating artificial neural networks.
For this project we will work with PyTorch [Wikipedia], a framework developed by Facebook's Artificial Intelligence Research lab (FAIR). PyTorch is widely used and in direct competition with Google's TensorFlow. It offers tensor computing (similar as NumPy) with strong potential for acceleration exploiting GPUs (unlike Numpy). Crucially, it provides a type-based automatic differentiation system which is used for the back-propagation algorithm, the workhorse of every neural network training.
Task: develop a classifier (similar to those based on the k-nearest-neighbours algorithm or on Boosted Decision Trees) using a fully-connected feed-forward deep neural network. The network should process input features of a top quark (e.g. pt, eta, phi as a minimal set, can be extended later) and provide as the output a probability score whether it is a resonance top quark or a top quark from associated production. For the training, a cross-entropy loss function is the best choice.
Before you can tackle this task, let's get familiar with PyTorch. There is an excellent introduction by Alfredo Canciani (NYU).
Please have a look at the Concepts: Artificial neural networks section.
Now that you have a rudimentary understanding of the PyTorch language, take a look at the script
You can retrieve the script by updating your local git repository.
Please execute
to retrieve the latest updates. You will notice that also the requirements.txt file has changed. This is because we now will use PyTorch in our project and need to install it.
Here, we assume that you already worked through the section Creating a classifier and followed the setup instructions there.
As always, when starting in a new terminal, make sure that the virtual environment is active:
Now the virtual environment is activated. You notice that the command line now starts with "(venv)".
We need to do one more thing to install pytorch (make sure you do this in the active virtual environment).
Please execute
to update your python packages in the virtual environment, including the installation of PyTorch.
To run the script, first make sure that the virtual environment is active by sourcing the setup.sh script and execute
to learn about the several command line arguments you can use to run specific classifiers and test their performance.
For running the neural network, please execute
You will see some-print outs, showing the training of the neural network on the training dataset. The result is a trained model you can use as a classifier for identifying the resonance top quarks.
After the training is complete, inspect the plots directory to check out the ROC curve and the dependency of the loss and the accuracy versus the number of training epochs.
Now you can experiment. How does the performance change when you
increase or decrease the number of epochs in the training
modify batch size or learning rate
modify the architecture of the neural network (more or less nodes in the layers)
Have fun!
