What is a Neural Network?
A neural network is a computational model inspired by the human brain. It consists of layers of interconnected nodes (neurons) that process information. Each connection has a weight that adjusts during training.
The Basic Unit: A Neuron
A single neuron takes inputs, multiplies them by weights, sums them up, adds a bias, and passes the result through an activation function.
python
class="text-purple-400 font-medium">import torch
class="text-purple-400 font-medium">import torch.nn as nn
class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># A single neuron
neuron = nn.Linear(in_features=class="text-amber-300">3, out_features=class="text-amber-300">1)
x = torch.tensor([class="text-amber-300">1.0, class="text-amber-300">2.0, class="text-amber-300">3.0])
output = neuron(x)
print(output) class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Single weighted sum + biasActivation Functions
Without activation functions, a neural network is just a linear model. Common ones:
- ●ReLU:
f(x) = max(0, x)— fast and widely used - ●Sigmoid:
f(x) = 1 / (1 + e^-x)— outputs between 0 and 1 - ●Softmax: Converts logits to probabilities across classes
python
relu = nn.ReLU()
sigmoid = nn.Sigmoid()
print(relu(torch.tensor([-class="text-amber-300">1.0, class="text-amber-300">0.0, class="text-amber-300">2.0]))) class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># [class="text-amber-300">0., class="text-amber-300">0., class="text-amber-300">2.]
print(sigmoid(torch.tensor([class="text-amber-300">0.0, class="text-amber-300">2.0]))) class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># [class="text-amber-300">0.5, class="text-amber-300">0.88]Building a Simple Network
python
class SimpleNN(nn.Module):
class="text-purple-400 font-medium">def __init__(self):
super().__init__()
self.layer1 = nn.Linear(class="text-amber-300">784, class="text-amber-300">128)
self.layer2 = nn.Linear(class="text-amber-300">128, class="text-amber-300">64)
self.output = nn.Linear(class="text-amber-300">64, class="text-amber-300">10)
self.relu = nn.ReLU()
class="text-purple-400 font-medium">def forward(self, x):
x = self.relu(self.layer1(x))
x = self.relu(self.layer2(x))
return self.output(x)
model = SimpleNN()How Learning Happens
- ●Forward pass — input flows through the network
- ●Loss calculation — compare output to expected answer
- ●Backward pass — compute gradients via backpropagation
- ●Update weights — optimizer adjusts weights to reduce loss
python
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=class="text-amber-300">0.001)
class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Training loop (one step)
outputs = model(x_train)
loss = criterion(outputs, y_train)
loss.backward() class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Compute gradients
optimizer.step() class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Update weights
optimizer.zero_grad() class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Reset gradientsKey Takeaways
- ●Neural networks learn by adjusting weights through backpropagation
- ●Activation functions add non-linearity, enabling complex pattern learning
- ●PyTorch makes building and training networks straightforward
- ●Start simple, then scale up as you understand the fundamentals