PyTorch is an open source machine learning library used for developing and training neural network based deep learning models. It is primarily developed by Facebookâs AI research group. PyTorch can be used with Python as well as a C++. Naturally, the Python interface is more polished. PyTorch (backed by Facebook) is immensely popular in research labs. Its popularity is growing on the production servers as well.

Let's take a look at some of the interesting aspects of the PyTorch library.

- Getting Started
- PyTorch Tensors
- A Simple Neural Network
- Autograd - the heart of PyTorch
- Understanding the torch.nn module
- Object Detection with PyTorch
- Image Style Transfer with PyTorch
- GAN with Pytorch
- LSTM with PyTorch
- Audio Processing with torchaudio
- TorchText - NLP with PyTorch
- Implement a ChatBot with PyTorch
- Reinforcement Learning with PyTorch
- PyTorch 3D
- Crypten - PyTorch for encrypted data
- PyTorch in C++
- Working with Multi Core GPU's

Starting with Basics, we look into the installation and some core concepts of PyTorch. Then, we dive deeper by implementing some of the common usecases in PyTorch. That can give us an idea of how to start with the library. After that, we will look at the PyTorch 3D and PyTorch Audio extensions - for image and audio processing using PyTorch. That can provide a sufficient introduction to start working on a PyTorch project.Beyond this, you can look up the PyTorch documentation.

This meme aptly describes the growing popularity of PyTorch

Unlike TensorFlow and other such frameworks, Pytorch is a lot more intuitive and yet extremely powerful in its performance. It is designed for use on GPU's and hence provides a great performance. It is based on dynamic graphs, compared to the static graphs in TensorFlow. That causes a slight dip in the performance, but the gap is bridging pretty fast. It's syntax is a lot simpler than TensorFlow. Hence it is pretty useful for developing complex models. In most cases, the final deployment is on a C++ version of PyTorch - hence the performance is not a bottleneck anymore.

Let us now start with the basics of PyTorch. The foremost step is to install the library. If you are working on Google Colab, or other such platform, it is quite likely that PyTorch comes preinstalled. If you are working locally, you will have to install it.

Most DataScience/AI developers work with Anaconda, rather than a basic Python distribution. That is because Anaconda comes with a lot of features that we need. PyTorch can be installed on either.

```
$ conda install pytorch
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/vikas/dev/anaconda3
added / updated specs:
- pytorch
The following packages will be downloaded:
package | build
---------------------------|-----------------
_pytorch_select-0.2 | gpu_0 2 KB
conda-4.8.2 | py37_0 2.8 MB
cudatoolkit-10.0.130 | 0 261.2 MB
cudnn-7.6.5 | cuda10.0_0 165.0 MB
ninja-1.9.0 | py37hfd86e86_0 1.2 MB
pytorch-1.3.1 |cuda100py37h53c1284_0 169.0 MB
------------------------------------------------------------
Total: 599.2 MB
The following NEW packages will be INSTALLED:
_pytorch_select pkgs/main/linux-64::_pytorch_select-0.2-gpu_0
cudatoolkit pkgs/main/linux-64::cudatoolkit-10.0.130-0
cudnn pkgs/main/linux-64::cudnn-7.6.5-cuda10.0_0
ninja pkgs/main/linux-64::ninja-1.9.0-py37hfd86e86_0
pytorch pkgs/main/linux-64::pytorch-1.3.1-cuda100py37h53c1284_0
The following packages will be UPDATED:
conda 4.7.12-py37_0 --> 4.8.2-py37_0
Proceed ([y]/n)? y
Downloading and Extracting Packages
_pytorch_select-0.2 | 2 KB | ################################################################################################################################################################## | 100%
pytorch-1.3.1 | 169.0 MB | ################################################################################################################################################################## | 100%
cudatoolkit-10.0.130 | 261.2 MB | ################################################################################################################################################################## | 100%
conda-4.8.2 | 2.8 MB | ################################################################################################################################################################## | 100%
ninja-1.9.0 | 1.2 MB | ################################################################################################################################################################## | 100%
cudnn-7.6.5 | 165.0 MB | ################################################################################################################################################################## | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
$
```

The exact details of the output logs would be different, depending upon the status and version of your Anaconda installation. Now, we have a good installation of the PyTorch library. Let us now get back to some theory, to understand some of the core concepts of PyTorch.

PyTorch started as an alternative to NumPy - tailored for the GPU. It defines has a basic object - tensor. This is the counterpart of the NumPy multidimensional array. As we would expect, a PyTorch tensor provides a lot more than the NumPy array. We will check that later.

On the similarities, we can instantiate and initialize a PyTorch Tensor, just like a NumPy array.

We can create an empty (uninitialized tensor for a given shape), using:

```
import torch
x = torch.empty(2,3)
print(x)
.....
tensor([[4.2678e-17, 3.0852e-41, 4.2675e-17],
[3.0852e-41, 8.9683e-44, 0.0000e+00]])
```

Note that the values in the uninitialized tensor are not created by PyTorch. This is the junk data that was already present in the memory that is allocated to the newly created tensor object.

We can also create a tensor, initialized with random values.

```
import torch
x = torch.rand(2,3)
print(x)
.....
tensor([[0.3612, 0.3983, 0.9686],
[0.6017, 0.6697, 0.8142]])
```

These are random values generated by PyTorch, and assigned to the individual elements of the tensor.

PyTorch provides us to create a new tensor initialized with 0's using the torch.zeros() method

```
import torch
x = torch.zeros(2,3)
print(x)
.....
tensor([[0., 0., 0.],
[0., 0., 0.]])
```

Note that the datatype of each element is a float. We can insist on an int or a long by explicitly specifying the datatype as we create the tensor.

```
x = torch.zeros(2,3, dtype=torch.int)
print(x)
.....
tensor([[0, 0, 0],
[0, 0, 0]], dtype=torch.int32)
```

The above methods created tensors based on predefined template values. We can explicitly specify the data to be filled in - by instantiating the tensor based on the data.

```
import torch
x = torch.tensor([5, 3, -1])
print(x)
.....
tensor([ 5, 3, -1])
x = torch.tensor([.5, 3, -1])
print(x)
.....
tensor([ 0.5000, 3.0000, -1.0000])
```

Note that PyTorch automatically infers the tensor datatype, based on the data types in the input array.

If you are reading this blog, I guess you already know what is addition or multiplication. I won't waste time, disk space, network bandwidth and browser memory, in trying to explain that. But there are some queer aspects of PyTorch that we should discuss here. PyTorch provides for multiple types of operation syntax.

```
import torch
# Initialize two tensors with random values.
x = torch.rand(2,3)
y = torch.rand(2,3)
print(x)
print(y)
.....
tensor([[0.4933, 0.3527, 0.1636],
[0.9583, 0.3568, 0.8859]])
tensor([[0.7720, 0.1015, 0.2516],
[0.9815, 0.9426, 0.9421]])
```

Simple, traditional addition operation

```
print(x + y)
.....
tensor([[1.2652, 0.4541, 0.4152],
[1.9398, 1.2995, 1.8280]])
```

The torch version of addition

```
print(torch.add(x, y))
.....
tensor([[1.2652, 0.4541, 0.4152],
[1.9398, 1.2995, 1.8280]])
```

This is another

`z = torch.empty(2, 3) torch.add(x, y, out=z) print(z) ..... tensor([[1.2652, 0.4541, 0.4152], [1.9398, 1.2995, 1.8280]])`

Yet Another.

```
print(x.add(y))
print(x)
.....
tensor([[1.2652, 0.4541, 0.4152],
[1.9398, 1.2995, 1.8280]])
tensor([[0.4933, 0.3527, 0.1636],
[0.9583, 0.3568, 0.8859]])
```

Note that after all this, the value of x has not changed. But, if we add an _ to the function name, the tensor is modified in place.

```
print(x.add_(y))
print(x)
.....
tensor([[1.2652, 0.4541, 0.4152],
[1.9398, 1.2995, 1.8280]])
tensor([[1.2652, 0.4541, 0.4152],
[1.9398, 1.2995, 1.8280]])
```

PyTorch tensors support all the standard indexing operations of NumPy arrays.

```
print(x[:, 1])
.....
tensor([1.4541, 1.2995])
```

And also, reshaping tensors with the view() operator

```
import torch
x = torch.rand(2,3)
print(x)
.....
tensor([[0.5462, 0.5335, 0.3462],
[0.9560, 0.4732, 0.6421]])
```

We can reshape it to a single dimension tensor

```
print(x.view(6))
.....
tensor([0.5462, 0.5335, 0.3462, 0.9560, 0.4732, 0.6421])
```

Or we can reshape it to another two dimensional tensor. Note that the dimension -1 indicates that its value should be inferred from the rest of the dimensions. In this case, it has to be 3.

```
print(x.view(-1,2))
.....
tensor([[0.5462, 0.5335],
[0.3462, 0.9560],
[0.4732, 0.6421]])
```

If we have a one element tensor, we can use the item() method to extract the value.

```
import torch
x = torch.rand(1)
print(x)
print(x.item())
......
tensor([-0.4520])
-0.45202213525772095
```

PyTorch provides for many more of these operations that can simplify our code. For a detailed tutorial of these operators, check out the PyTorch Documentation.

Similarly, we can convert PyTorch tensors to and from NumPy arrays using the simple NumPy bridge.

```
import torch
import numpy as np
a = torch.rand(4)
print(a)
.....
tensor([0.3368, 0.6063, 0.4710, 0.3740])
```

We can now get the NumPy array using the numpy() method

```
b = a.numpy()
print(b)
.....
[0.33675885 0.60627323 0.4709654 0.37395465]
```

This NumPy array is based out of the same data as the original tensor. It points to the same memory location. Thus, modifying the tensor in place will alter the array as well. Here is how:

```
print(a.add_(1))
print(b)
.....
tensor([1.3368, 1.6063, 1.4710, 1.3740])
[1.3367589 1.6062732 1.4709654 1.3739547]
```

Changing the elements in a, causes the corresponding change in b as well - because both point to the same memory. NumPy does not support modification in place. So the problem of reverse modification does not arise.

Similarly, we can extract a PyTorch tensor from a NumPy array using from_numpy(). Here again, modifying the tensor modifies the NumPy array.

```
import torch
import numpy as np
x = np.ones(5)
y = torch.from_numpy(x)
y.add_(1)
print(x)
.....
tensor([2., 2., 2., 2., 2.], dtype=torch.float64)
[2. 2. 2. 2. 2.]
```