$ init portfolio
R
Back to Blog
PyTorchTensorsFundamentals

Tensors: The Data Structure Behind Deep Learning

Everything you need to know about tensors — what they are, how they work, and why they're the backbone of ML.

Raj Tiwari
Raj Tiwari
2026-07-134 min read
Tensors: The Data Structure Behind Deep Learning

What is a Tensor?

A tensor is a multi-dimensional array — the fundamental data structure in deep learning. Think of it as a generalization of scalars, vectors, and matrices.

  • ●Scalar (0D): A single number → torch.tensor(5)
  • ●Vector (1D): A list of numbers → torch.tensor([1, 2, 3])
  • ●Matrix (2D): A grid of numbers → torch.tensor([[1, 2], [3, 4]])
  • ●Tensor (3D+): A cube or higher-dimensional array

Creating Tensors

python
class="text-purple-400 font-medium">import torch

class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># From Python lists
t1 = torch.tensor([class="text-amber-300">1, class="text-amber-300">2, class="text-amber-300">3])              class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># 1D tensor
t2 = torch.tensor([[class="text-amber-300">1, class="text-amber-300">2], [class="text-amber-300">3, class="text-amber-300">4]])        class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># 2D tensor
t3 = torch.tensor([[[class="text-amber-300">1, class="text-amber-300">2], [class="text-amber-300">3, class="text-amber-300">4]]])      class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># 3D tensor

class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Common constructors
zeros = torch.zeros(class="text-amber-300">3, class="text-amber-300">4)       class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># 3x4 matrix of zeros
ones = torch.ones(class="text-amber-300">2, class="text-amber-300">3)         class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># 2x3 matrix of ones
rand = torch.randn(class="text-amber-300">3, class="text-amber-300">3)        class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># 3x3 random normal
eye = torch.eye(class="text-amber-300">3)              class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># 3x3 identity matrix

Tensor Operations

python
a = torch.tensor([[class="text-amber-300">1, class="text-amber-300">2], [class="text-amber-300">3, class="text-amber-300">4]], dtype=torch.float32)
b = torch.tensor([[class="text-amber-300">5, class="text-amber-300">6], [class="text-amber-300">7, class="text-amber-300">8]], dtype=torch.float32)

class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Element-wise
print(a + b)        class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># [[class="text-amber-300">6, class="text-amber-300">8], [class="text-amber-300">10, class="text-amber-300">12]]
print(a * b)        class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># [[class="text-amber-300">5, class="text-amber-300">12], [class="text-amber-300">21, class="text-amber-300">32]]

class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Matrix multiplication
print(a @ b)        class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># [[class="text-amber-300">19, class="text-amber-300">22], [class="text-amber-300">43, class="text-amber-300">50]]
print(torch.matmul(a, b))  class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Same thing

class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Reshaping
c = torch.arange(class="text-amber-300">12)
print(c.reshape(class="text-amber-300">3, class="text-amber-300">4))     class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># 3x4 matrix
print(c.view(class="text-amber-300">4, class="text-amber-300">3))        class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># 4x3 matrix

Broadcasting

Tensors with different shapes can be operated on if they're compatible:

python
class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Broadcasting example
a = torch.tensor([[class="text-amber-300">1, class="text-amber-300">2, class="text-amber-300">3]])    class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Shape: (class="text-amber-300">1, class="text-amber-300">3)
b = torch.tensor([[class="text-amber-300">1], [class="text-amber-300">2], [class="text-amber-300">3]]) class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Shape: (class="text-amber-300">3, class="text-amber-300">1)

class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Result shape: (class="text-amber-300">3, class="text-amber-300">3)
print(a + b)
class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># [[class="text-amber-300">2, class="text-amber-300">3, class="text-amber-300">4],
class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic">#  [class="text-amber-300">3, class="text-amber-300">4, class="text-amber-300">5],
class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic">#  [class="text-amber-300">4, class="text-amber-300">5, class="text-amber-300">6]]

GPU Acceleration

python
device = torch.device(class="text-emerald-class="text-amber-300">400">"cuda" if torch.cuda.is_available() else class="text-emerald-class="text-amber-300">400">"cpu")

t = torch.randn(class="text-amber-300">1000, class="text-amber-300">1000, device=device)
class=class="text-emerald-class="text-amber-300">400">"text-gray-class="text-amber-300">500 italic"># Operations on GPU are class="text-amber-300">10-100x faster

Key Takeaways

  • ●Tensors are multi-dimensional arrays that power all ML computations
  • ●PyTorch tensors support automatic differentiation for gradient computation
  • ●Use GPU tensors for significant speedup in training
  • ●Understanding tensor shapes is crucial for debugging model dimensions
0

More Posts

Neural Networks: The Building Blocks of AI

A beginner-friendly guide to neural networks — how they work, why they matter, and how to build one from scratch in PyTorch.

5 min read

Transformers: Attention Is All You Need

Understanding the transformer architecture that powers GPT, BERT, and every modern LLM.

6 min read