samuiweb_gm

samuiweb_gm

Programming Machine Learning: Bias in linear regression

Hi there,
bought “Programming Machine Learning” (great book) from pragprog.

A question (from a newbie) for the author, or anyone can help:
At the end of the chapter 4 in the paragraph “Bye Bye, Bias” is described how to add the bias parameter to the multiple linear regression algorithm.
Why the bias is maintained constant at every iteration when in the “single” linear regression in chapter two it is variable?

Thank you!

Most Liked

nusco

nusco

Author of Metaprogramming Ruby & Programming Machine Learning

Here I am. Hello, @samuiweb_gm!

The trick with the bias can be confusing, so let me try to explain it here.

In Chapter 2, we use a line to approximate the data. here is its equation:

ŷ = x * w + b

So we calculate the output based on the value of the inputx. We do it with two variables, or “parameters”: w and b.

By contrast, in Chapter 4 we have multiple inputs: x1, x2, and so on. So we start by calculating the output based on those inputs, each given a weight… and a final bias, like we did before:

ŷ = x1 * w1 + x2 * w2 + x3 * w3 + b

The trick in “Bye, bye, bias” is all about turning that b into just another weight (let’s call it w0), by associating it with an artificial input:

ŷ = x1 * w1 + x2 * w2 + x3 * w3 + x0 * w0

The last two formulae are the same as long as we do two things:

  • We rename b to w0.
  • We add an artificial input x0 that has a value of 1, so that when we multiplicate it by w0, nothing changes.

So, to answer your question directly: b is still a variable, and it’s become a weight like any other. What we added is another input, and that one has a constant value of 1. By doing that, we can remove all the code that deals with the special case of b, and treat all the weights and the bias the same.

Does that make it clear?

dimitarvp

dimitarvp

You can probably mention them directly I reckon: @PragmaticBookshelf.

@AstonJ should it be enough that such questions are put in the PragProg Customers category, or are mentions desirable at all?

AstonJ

AstonJ

This section is fine - no mentions needed :smiley:

PragProg will be keeping an eye on the section (and their authors on threads related to their books) so they will see the thread at some point, though keep in mind they are busy so it might not be right away.

I am also currently working on our first iteration of book portals which will make it clearer where to post and find threads relating to specific books. Hoping to get the first version of this up early next week (to begin with we’ll just use the standard portal template, then customise this after reviewing how it’s used).

Where Next?

Popular Pragmatic Bookshelf topics Top

GilWright
Working through the steps (checking that the Info,plist matches exactly), run the demo game and what appears is grey but does not fill th...
New
jesse050717
Title: Web Development with Clojure, Third Edition, pg 116 Hi - I just started chapter 5 and I am stuck on page 116 while trying to star...
New
yulkin
your book suggests to use Image.toByteData() to convert image to bytes, however I get the following error: "the getter ‘toByteData’ isn’t...
New
Alexandr
Hi everyone! There is an error on the page 71 in the book “Programming machine learning from coding to depp learning” P. Perrotta. You c...
New
jdufour
Hello! On page xix of the preface, it says there is a community forum "… for help if your’re stuck on one of the exercises in this book… ...
New
swlaschin
The book has the same “Problem space/Solution space” diagram on page 18 as is on page 17. The correct Problem/Solution space diagrams ar...
New
jgchristopher
“The ProductLive.Index template calls a helper function, live_component/3, that in turn calls on the modal component. ” Excerpt From: Br...
New
New
SlowburnAZ
Getting an error when installing the dependencies at the start of this chapter: could not compile dependency :exla, "mix compile" failed...
New
dachristenson
@mfazio23 Android Studio will not accept anything I do when trying to use the Transformations class, as described on pp. 140-141. Googl...
New

Other popular topics Top

Devtalk
Reading something? Working on something? Planning something? Changing jobs even!? If you’re up for sharing, please let us know what you’...
1050 21151 394
New
New
PragmaticBookshelf
Rust is an exciting new programming language combining the power of C with memory safety, fearless concurrency, and productivity boosters...
New
AstonJ
This looks like a stunning keycap set :orange_heart: A LEGENDARY KEYBOARD LIVES ON When you bought an Apple Macintosh computer in the e...
New
Margaret
Hello everyone! This thread is to tell you about what authors from The Pragmatic Bookshelf are writing on Medium.
1147 29994 760
New
AstonJ
Saw this on TikTok of all places! :lol: Anyone heard of them before? Lite:
New
PragmaticBookshelf
Author Spotlight Mike Riley @mriley This month, we turn the spotlight on Mike Riley, author of Portable Python Projects. Mike’s book ...
New
CommunityNews
A Brief Review of the Minisforum V3 AMD Tablet. Update: I have created an awesome-minisforum-v3 GitHub repository to list information fo...
New
AstonJ
Curious what kind of results others are getting, I think actually prefer the 7B model to the 32B model, not only is it faster but the qua...
New
PragmaticBookshelf
A concise guide to MySQL 9 database administration, covering fundamental concepts, techniques, and best practices. Neil Smyth MySQL...
New

Sub Categories: