Menu

📰
0

Chapter 5: Linear Transformation and Softmax

DEV Community: csharp·Gary Jackson·about 1 month ago
#cdWkQRDN
#dev#class#code#value#highlight#article
Reading 0:00
15s threshold

What You'll Build Two helper functions that show up in nearly every layer of a neural network: Linear takes an input vector and a weight matrix, multiplies each row of weights element-by-element with the input, and sums each row into a single output value: input: [1, 2, 3] weights: [[0.1, 0.2, 0.3], row 0: 0.1*1 + 0.2*2 + 0.3*3 = 1.4 [0.4, 0.5, 0.6]] row 1: 0.4*1 + 0.5*2 + 0.6*3 = 3.2 output: [1.4, 3.2] Two rows of weights means two output values. This is how neural networks change the size of data as it flows through layers. Softmax takes a list of raw numbers and turns them into probabilities that add up to 1. For example, [2.0, 1.0, 0.1] becomes roughly [0.66, 0.24, 0.10] . The largest input gets the highest probability. They live in their own file because they're pure math utilities, independent of the model architecture. Depends On Chapters 1-2 (the Value class - our computation recorder). Code Linear needs a way to compute a dot product between two lists of Value objects.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More