Menu

Post image 1
Post image 2
1 / 2
0

Chapter 12: Inference - Generating New Text

DEV Community·Gary Jackson·about 1 month ago
#gSvku2lb
Reading 0:00
15s threshold

What You'll Build A sampling loop that generates new names from the trained model. Depends On Chapter 11 (the trained model). How Generation Works After training, the parameters are frozen. We start with the BOS token, feed it through the model, get a probability distribution over next tokens, sample one, feed it back in, and repeat until the model produces BOS again ("I'm done") or we hit the maximum length. Same generation loop from the bigram chapter. Only the source of the probabilities has changed. // --- FullTraining.cs (add below the training loop from Chapter 11) --- const double Temperature = 0.5 ; Console . WriteLine ( "\n--- inference (new, hallucinated names) ---" ); for ( int sampleIdx = 0 ; sampleIdx < 20 ; sampleIdx ++) { List < List < Value >>[] keys = model . CreateKvCache (); List < List < Value >>[] values = model . CreateKvCache (); int tokenId = tokenizer .…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More