samuiweb_gm

samuiweb_gm

Programming Machine Learning: Bias in linear regression

Hi there,
bought “Programming Machine Learning” (great book) from pragprog.

A question (from a newbie) for the author, or anyone can help:
At the end of the chapter 4 in the paragraph “Bye Bye, Bias” is described how to add the bias parameter to the multiple linear regression algorithm.
Why the bias is maintained constant at every iteration when in the “single” linear regression in chapter two it is variable?

Thank you!

Most Liked

nusco

nusco

Author of Metaprogramming Ruby & Programming Machine Learning

Here I am. Hello, @samuiweb_gm!

The trick with the bias can be confusing, so let me try to explain it here.

In Chapter 2, we use a line to approximate the data. here is its equation:

ŷ = x * w + b

So we calculate the output ŷ based on the value of the inputx. We do it with two variables, or “parameters”: w and b.

By contrast, in Chapter 4 we have multiple inputs: x1, x2, and so on. So we start by calculating the output based on those inputs, each given a weight… and a final bias, like we did before:

ŷ = x1 * w1 + x2 * w2 + x3 * w3 + b

The trick in “Bye, bye, bias” is all about turning that b into just another weight (let’s call it w0), by associating it with an artificial input:

ŷ = x1 * w1 + x2 * w2 + x3 * w3 + x0 * w0

The last two formulae are the same as long as we do two things:

  • We rename b to w0.
  • We add an artificial input x0 that has a value of 1, so that when we multiplicate it by w0, nothing changes.

So, to answer your question directly: b is still a variable, and it’s become a weight like any other. What we added is another input, and that one has a constant value of 1. By doing that, we can remove all the code that deals with the special case of b, and treat all the weights and the bias the same.

Does that make it clear?

dimitarvp

dimitarvp

You can probably mention them directly I reckon: @PragmaticBookshelf.

@AstonJ should it be enough that such questions are put in the PragProg Customers category, or are mentions desirable at all?

AstonJ

AstonJ

This section is fine - no mentions needed :smiley:

PragProg will be keeping an eye on the section (and their authors on threads related to their books) so they will see the thread at some point, though keep in mind they are busy so it might not be right away.

I am also currently working on our first iteration of book portals which will make it clearer where to post and find threads relating to specific books. Hoping to get the first version of this up early next week (to begin with we’ll just use the standard portal template, then customise this after reviewing how it’s used).

Where Next?

Popular Pragmatic Bookshelf topics Top

abtin
page 20: … protoc command… I had to additionally run the following go get commands in order to be able to compile protobuf code using go...
New
Alexandr
Hi everyone! There is an error on the page 71 in the book “Programming machine learning from coding to depp learning” P. Perrotta. You c...
New
New
swlaschin
The book has the same “Problem space/Solution space” diagram on page 18 as is on page 17. The correct Problem/Solution space diagrams ar...
New
jskubick
I think I might have found a problem involving SwitchCompat, thumbTint, and trackTint. As entered, the SwitchCompat changes color to hol...
New
jgchristopher
“The ProductLive.Index template calls a helper function, live_component/3, that in turn calls on the modal component. ” Excerpt From: Br...
New
brunogirin
When I run the coverage example to report on missing lines, I get: pytest --cov=cards --report=term-missing ch7 ERROR: usage: pytest [op...
New
kolossal
Hi, I need some help, I’m new to rust and was learning through your book. but I got stuck at the last stage of distribution. Whenever I t...
New
rainforest
Hi, I’ve got a question about the implementation of PubSub when using a Phoenix.Socket.Transport behaviour rather than channels. Before ...
New
redconfetti
Docker-Machine became part of the Docker Toolbox, which was deprecated in 2020, long after Docker Desktop supported Docker Engine nativel...
New

Other popular topics Top

AstonJ
If it’s a mechanical keyboard, which switches do you have? Would you recommend it? Why? What will your next keyboard be? Pics always w...
New
PragmaticBookshelf
Machine learning can be intimidating, with its reliance on math and algorithms that most programmers don't encounter in their regular wor...
New
DevotionGeo
I know that -t flag is used along with -i flag for getting an interactive shell. But I cannot digest what the man page for docker run com...
New
PragmaticBookshelf
Learn different ways of writing concurrent code in Elixir and increase your application's performance, without sacrificing scalability or...
New
DevotionGeo
The V Programming Language Simple language for building maintainable programs V is already mentioned couple of times in the forum, but I...
New
PragmaticBookshelf
Create efficient, elegant software tests in pytest, Python's most powerful testing framework. Brian Okken @brianokken Edited by Kat...
New
Maartz
Hi folks, I don’t know if I saw this here but, here’s a new programming language, called Roc Reminds me a bit of Elm and thus Haskell. ...
New
foxtrottwist
A few weeks ago I started using Warp a terminal written in rust. Though in it’s current state of development there are a few caveats (tab...
New
AnfaengerAlex
Hello, I’m a beginner in Android development and I’m facing an issue with my project setup. In my build.gradle.kts file, I have the foll...
New
PragmaticBookshelf
Explore the power of Ash Framework by modeling and building the domain for a real-world web application. Rebecca Le @sevenseacat and ...
New

Sub Categories: