Complete Deep Learning Notes

From Basics to Advanced Architectures – Full Course Compilation

Course:Deep Learning Fundamentals

Date:July 2026

Version:1.0 Complete

Contents

1

2

MachineLearningisasubsetofAIthatenablessystemstolearnfromdatawithoutbeingex-plicitlyprogrammed.Itreliesheavilyontabularandstructuredfeaturesmanuallyisolatedby domain experts.

•Data Types:Works primarily with structured/tabular data (CSV, Excel).

•Feature Engineering:Requires extensive manual feature curation and selection.•Complexity:Relies on simpler, explainable mathematical algorithms.

•Hardware Constraints:Can easily run on standard modern CPUs.

1.2What is Deep Learning (DL)?

Deep Learning is a subset of Machine Learning that uses neural networks with multiple layers (hence“deep”)toautomaticallyextractandlearnhierarchicalrepresentationsfromcomplex, raw inputs.

•Data Types:Excels with raw, unstructured data (images, video, raw text, and audio).•Feature Engineering:Automatically discovers features during the optimization process.•Complexity:High density of parameterized connections (black box structures).

•Hardware Constraints:Requires parallel processing power of GPUs or TPUs.

ples)

weights)

3

Invented by Frank Rosenblatt, the Perceptron is the ancestral building block of modern neural network designs.It represents a single-layer neural framework processing inputs with a binary step activation.

early separable tasks (e.g., standard logic gates like AND, OR).It fundamentally fails on non-linear transformations like the XOR function, which requires multi-layer warping.

2.2Weights, Bias, and Parameters

•Weight (w):Represents the strength of the connection.It scales the incoming value, de-termining how much influence a given input has on the node.

•Bias (b):An offset value added to the weighted sum.It shifts the activation function curve horizontally, ensuring nodes can activate even when all input elements are zero (x = 0).

2.3Activation Functions

Activation functions introduce non-linear mapping boundaries,permitting networks to model sophisticated mathematical equations.

2.3.1Common Formulations

1.Sigmoid Function:Maps output to a probability interval (0, 1).

2.Hyperbolic Tangent (Tanh):Zero-centered scaling mapping to (−1, 1).

3.Rectified Linear Unit (ReLU): Extremely fast computation, avoids vanishing gradient lim- its on positive values.

f(x) = max(0, x) (5)

4.Leaky ReLU: Fixes the “dying ReLU” failure by preserving a small slope αon negative in- puts.