1. Introduction

In this tutorial, we’re going to review some open-source neural network libraries.

Neural network libraries are usually used to implement neural networks in computer programs. Over the years, many of these libraries have been developed and enhanced to make it easier to implement and utilize the processing functionality of neural networks. In the subsequent sections, we’ll review some of these libraries.

2. TensorFlow

TensorFlow was first developed in 2015 by the team at Google Brain, a machine learning research group at Google. It was designed to facilitate research in building machine learning and neural network models. As such, TensorFlow offers easy model building and a robust environment for deploying machine learning models:

tensor

It also boasts of having a platform that can be used by both beginners and experts and can be used on Windows, Linux, Android, and macOS platforms. Some companies that use TensorFlow include Google, DeepMind, AirBnB, Coca-Cola, and Intel.

3. CNTK

CNTK is also known as Microsoft Cognitive Toolkit. First released in 2016 and now deprecated, CNTK is an open-source library for training neural networks for deep learning:

cntk

It allows users to create and combine commonly used neural networks such as Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN).

CNTK is supported on Linux, Windows, and macOS platforms.

4. PyTorch

Released in 2016, PyTorch was developed by Facebook’s AI Research Lab. PyTorch was built on top of Torch, which is a machine learning and scientific computing library. According to the developers of PyTorch, it is a machine learning framework that accelerates research prototyping to production deployment:

pytorch

PyTorch features distributed training of neural network models, cloud support, a robust ecosystem, and is supported on Windows, Linux, and macOS platforms. It is used by organizations such as Standford University and Udacity.

5. Theano

Theano is an open-source library released in 2007 by Montreal Institute for Learning Algorithms (MILA). Theano offers support for evaluating mathematical expressions commonly used for machine learning. It provides a Python interface that can be used together with Keras:

theano

Theano is also supported on operating systems such as Linux, Windows, and macOS.

6. Caffe

Caffe is a deep learning framework developed in 2013 by the Berkeley AI Research group. It was designed to offer speed and modularity in building neural networks. However, Caffe mostly supports neural networks for image classification and image segmentation:

caffe

It currently runs on Linux, macOS, and Windows operating systems.

7. Keras

Keras was designed by Francois Chollet, an engineer at Google in 2015. It provides a user-friendly interface for building neural networks in Python. Keras features modules such as activation functions and layers for implementing neural networks in a few steps for fast experimentation:

keras

Additionally, the library can run on top of other libraries such as TensorFlow and CNTK. Due to its simple yet powerful interface, it is used by organizations such as NASA and CERN.

8. DeepLearning4J

DeepLearning4J is a deep learning library written for Java and Scala and initially released in 2014. It features a distributed computing training environment that can accelerate performance. DeepLearning4J allows users the flexibility of composing and combining neural network models:

dl4j

DeepLearning4J is currently supported on Windows, Linux, and macOS platforms. In addition, it works well together with other neural network libraries such as TensorFlow and Keras.

9. Summary

Now let’s take a look at a summary of the pros and cons of the frameworks discussed above:

Software

Pros

Cons

TensorFlow

  • Provides data visualization through TensorBoard

  • Robust computational power

  • Well documented and has great community support

  • No commercial support

  • Slow compared to other frameworks like CNTK

  • Not many pre-trained models

CNTK

  • Has pre-trained neural network models

  • Very scalable on multiple machines

  • Efficient resource management

  • Lacks visualization functionality

  • Less community support as compared to TensorFlow

  • Not beginner-friendly

PyTorch

  • Provides modularity

  • Has many pre-trained neural network models

  • Provides useful debugging tools

  • No commercial support

  • Documentation available is not extensive

  • No interfaces for visualization

Theano

  • Has pre-trained neural network models

  • Can be used with Keras for simplification

  • Not very robust

  • Development of the library has been halted

Caffe

  • Has pre-trained neural network models

  • Excellent support for image processing

  • No commercial support

  • Slow development

Keras

  • Can function with other libraries such as TensorFlow, Theano, and DeepLearning4J

  • Beginner-friendly

  • Has pre-trained neural network models

  • Low-level API

  • Errors are not easily debugged

DeepLearning4J

  • Has pre-trained neural network models

  • Distributed computing framework

  • Allows users to compose and combine models

  • Limited to Java

  • Less popular compared to Keras, PyTorch, and TensorFlow

10. Conclusions

In this article, we have identified and reviewed some commonly used open-source libraries for neural networks.