samuiweb_gm

samuiweb_gm

Programming Machine Learning: Bias in linear regression

Hi there,
bought “Programming Machine Learning” (great book) from pragprog.

A question (from a newbie) for the author, or anyone can help:
At the end of the chapter 4 in the paragraph “Bye Bye, Bias” is described how to add the bias parameter to the multiple linear regression algorithm.
Why the bias is maintained constant at every iteration when in the “single” linear regression in chapter two it is variable?

Thank you!

Most Liked

nusco

nusco

Author of Metaprogramming Ruby & Programming Machine Learning

Here I am. Hello, @samuiweb_gm!

The trick with the bias can be confusing, so let me try to explain it here.

In Chapter 2, we use a line to approximate the data. here is its equation:

ŷ = x * w + b

So we calculate the output based on the value of the inputx. We do it with two variables, or “parameters”: w and b.

By contrast, in Chapter 4 we have multiple inputs: x1, x2, and so on. So we start by calculating the output based on those inputs, each given a weight… and a final bias, like we did before:

ŷ = x1 * w1 + x2 * w2 + x3 * w3 + b

The trick in “Bye, bye, bias” is all about turning that b into just another weight (let’s call it w0), by associating it with an artificial input:

ŷ = x1 * w1 + x2 * w2 + x3 * w3 + x0 * w0

The last two formulae are the same as long as we do two things:

  • We rename b to w0.
  • We add an artificial input x0 that has a value of 1, so that when we multiplicate it by w0, nothing changes.

So, to answer your question directly: b is still a variable, and it’s become a weight like any other. What we added is another input, and that one has a constant value of 1. By doing that, we can remove all the code that deals with the special case of b, and treat all the weights and the bias the same.

Does that make it clear?

dimitarvp

dimitarvp

You can probably mention them directly I reckon: @PragmaticBookshelf.

@AstonJ should it be enough that such questions are put in the PragProg Customers category, or are mentions desirable at all?

AstonJ

AstonJ

This section is fine - no mentions needed :smiley:

PragProg will be keeping an eye on the section (and their authors on threads related to their books) so they will see the thread at some point, though keep in mind they are busy so it might not be right away.

I am also currently working on our first iteration of book portals which will make it clearer where to post and find threads relating to specific books. Hoping to get the first version of this up early next week (to begin with we’ll just use the standard portal template, then customise this after reviewing how it’s used).

Where Next?

Popular Pragmatic Bookshelf topics Top

jimschubert
In Chapter 3, the source for index introduces Config on page 31, followed by more code including tests; Config isn’t introduced until pag...
New
jon
Some minor things in the paper edition that says “3 2020” on the title page verso, not mentioned in the book’s errata online: p. 186 But...
New
iPaul
page 37 ANTLRInputStream input = new ANTLRInputStream(is); as of ANTLR 4 .8 should be: CharStream stream = CharStreams.fromStream(i...
New
jeffmcompsci
Title: Design and Build Great Web APIs - typo “https://company-atk.herokuapp.com/2258ie4t68jv” (page 19, third bullet in URL list) Typo:...
New
herminiotorres
Hi @Margaret , On page VII the book tells us the example and snippets will be all using Elixir version 1.11 But on page 3 almost the en...
New
adamwoolhether
When trying to generate the protobuf .go file, I receive this error: Unknown flag: --go_opt libprotoc 3.12.3 MacOS 11.3.1 Googling ...
New
jskubick
I found an issue in Chapter 7 regarding android:backgroundTint vs app:backgroundTint. How to replicate: load chapter-7 from zipfile i...
New
oaklandgit
Hi, I completed chapter 6 but am getting the following error when running: thread 'main' panicked at 'Failed to load texture: IoError(O...
New
creminology
Skimming ahead, much of the following is explained in Chapter 3, but new readers (like me!) will hit a roadblock in Chapter 2 with their ...
New
Keton
When running the program in chapter 8, “Implementing Combat”, the printout Health before attack was never printed so I assumed something ...
New

Other popular topics Top

Devtalk
Reading something? Working on something? Planning something? Changing jobs even!? If you’re up for sharing, please let us know what you’...
1045 20892 392
New
New
AstonJ
poll poll Be sure to check out @Dusty’s article posted here: An Introduction to Alternative Keyboard Layouts It’s one of the best write-...
New
New
PragmaticBookshelf
Build highly interactive applications without ever leaving Elixir, the way the experts do. Let LiveView take care of performance, scalabi...
New
PragmaticBookshelf
Create efficient, elegant software tests in pytest, Python's most powerful testing framework. Brian Okken @brianokken Edited by Kat...
New
mafinar
This is going to be a long an frequently posted thread. While talking to a friend of mine who has taken data structure and algorithm cou...
New
PragmaticBookshelf
Build efficient applications that exploit the unique benefits of a pure functional language, learning from an engineer who uses Haskell t...
New
AstonJ
Was just curious to see if any were around, found this one: I got 51/100: Not sure if it was meant to buy I am sure at times the b...
New
PragmaticBookshelf
Author Spotlight: VM Brasseur @vmbrasseur We have a treat for you today! We turn the spotlight onto Open Source as we sit down with V...
New

Sub Categories: