PragProg Customers

Next-Level A/B is Testing: 'inconclusive results' is ambiguous (p. 12)

dlindelof

dlindelof

Next-Level A/B is Testing: 'inconclusive results' is ambiguous (p. 12)

@leemay

The book refers to ‘inconclusive results’ starting on page 12. I would caution against introducing this term without carefully defining it. At my dayjob I frequently deal with product managers who believe that an A/B experiment with a neutral outcome is ‘inconclusive’, when in fact it just says that there is no statistically significant difference between the two arms.

2 comments
/book-next-level-a-b-testing
1 27 2

Most Liked

leemay

leemay

Author of Practical A/B Testing

That’s a really good point. I’ll do exactly as you suggest, the details matter. I’ll define and give caveats of what it looks like in practice/what that could mean. Luckily we’re still in beta so I can make these changes this week. Thank you @dlindelof!

1
Post #3

Where Next?

View thread on forum
View Next Level A B Testing's book portal
Post errata for this book
Post a suggestion for this book
Post a question about this book
Post a review about this book
Home PragProg Customers
/book-next-level-a-b-testing
1 27 2
Last post

Popular Pragmatic Bookshelf topics Top

yulkin
PragProg Customers

Programming Flutter: Image.toByteData() does not exist (page40)
your book suggests to use Image.toByteData() to convert image to bytes, however I get the following error: "the getter ‘toByteData’ isn’t...
/book-programming-flutter#question
1 4102 2
New
simonpeter
PragProg Customers

Web Development with Clojure, Third Edition: (create-migration "guestbook") not working (page 9)
When I try the command to create a pair of migration files I get an error. user=> (create-migration "guestbook") Execution error (Ill...
#errata/book-web-development-with-clojure-third-edition
5 1788 4
New
edruder
PragProg Customers

Docker for Rails Developers: Notes for Rails 6.1, Ruby 2.7.2
I thought that there might be interest in using the book with Rails 6.1 and Ruby 2.7.2. I’ll note what I needed to do differently here. ...
/book-docker-for-rails-developers
9 1177 1
New
raul
PragProg Customers

Distributed Services with Go: Chapter 3 suggestions
Page 28: It implements io.ReaderAt on the store type. Sorry if it’s a dumb question but was the io.ReaderAt supposed to be io.ReadAt? ...
#errata/book-distributed-services-with-go
0 1302 3
New
taguniversalmachine
PragProg Customers

Programming Phoenix LiveView: B6.0 pg 56 - current-user not set
It seems the second code snippet is missing the code to set the current_user: current_user: Accounts.get_user_by_session_token(session["...
#errata/book-programming-phoenix-liveview
0 1898 8
New
Keton
PragProg Customers

Hands-on Rust: How do I debug a call chain in Rust?
When running the program in chapter 8, “Implementing Combat”, the printout Health before attack was never printed so I assumed something ...
#question/book-hands-on-rust
2 1034 2
New
mcpierce
PragProg Customers

Kotlin and Android Development featuring Jetpack: Unique constraint error when running chapter 5 code changes
@mfazio23 I’ve applied the changes from Chapter 5 of the book and everything builds correctly and runs. But, when I try to start a game,...
/book-kotlin-and-android-development-featuring-jetpack
0 1180 10
New
davetron5000
PragProg Customers

Bug with dx/build - how to workaround until fix is up
Hello faithful readers! If you have tried to follow along in the book, you are asked to start up the dev environment via dx/build and ar...
#errata/book-ruby-on-rails-background-jobs-with-sidekiq
3 1178 19
New
dachristenson
PragProg Customers

Kotlin and Android Development featuring Jetpack: Transformations class doesn't seem to exist anymore (pp. 140-141
@mfazio23 Android Studio will not accept anything I do when trying to use the Transformations class, as described on pp. 140-141. Googl...
#errata/book-kotlin-and-android-development-featuring-jetpack
0 2155 5
New
dachristenson
PragProg Customers

Kotlin and Android Development featuring Jetpack: End Ch. 11 app runs but consistently crashes
I’ve got to the end of Ch. 11, and the app runs, with all tabs displaying what they should – at first. After switching around between St...
#question/book-kotlin-and-android-development-featuring-jetpack
0 936 8
New

Other popular topics Top

AstonJ
General Dev>Hardware

Which keyboard do you have?
If it’s a mechanical keyboard, which switches do you have? Would you recommend it? Why? What will your next keyboard be? Pics always w...
#hardware/keyboards#sticky#mechanical-keyboards
144 9115 50
New
AstonJ
General Dev>Hardware

Planck vs Preonic vs Subatomic (Keyboards)
I ended up cancelling my Moonlander order as I think it’s just going to be a bit too bulky for me. I think the Planck and the Preonic (o...
/keyboards#mechanical-keyboards#ortholinear#planck#preonic
105 17596 47
New
AstonJ
General Dev>Dev Chat

Languages Without Garbage Collection
Continuing the discussion from Thinking about learning Crystal, let’s discuss - I was wondering which languages don’t GC - maybe we can c...
#garbage-collection
21 5575 7
New
AstonJ
Frontend>Chat

Online Hand to eye coordination test
Was just curious to see if any were around, found this one: I got 51/100: Not sure if it was meant to buy I am sure at times the b...
#online-tools
4 4562 1
New
AstonJ
macOS>Chat

How to block any website on Mac using Little Snitch
If you want a quick and easy way to block any website on your Mac using Little Snitch simply… File > New Rule: And select Deny, O...
#macos#how-to#littlesnitch
5 11227 3
New
NewsBot
Backend>Official News

Node.js v22.14.0 released!
Node.js v22.14.0 has been released. Link: Release 2025-02-11, Version 22.14.0 'Jod' (LTS), @aduh95 · nodejs/node · GitHub
github.com
/nodejs#official-news
0 4251 0
New
RobertRichards
Game Dev>Chat

Hair Salon Games for Girls Fun
Hair Salon Games for Girls Fun Girls Hair Saloon game is mainly developed for kids. This game allows users to select virtual avatars to ...
#ios#android#unity#fun
2 3556 1
New
Fl4m3Ph03n1x
AI>Questions

What are the best text-to-speech ai generation tools that you can run locally?
Background Lately I am in a quest to find a good quality TTS ai generation tool to run locally in order to create audio for some videos I...
#ai#text-to-speech
6 12337 3
New
mindriot
General Dev>Reviews

Keyboard Review: UHK60V2 vs Defy vs Voyager vs Glove80 vs Svalboard
Ok, well here are some thoughts and opinions on some of the ergonomic keyboards I have, I guess like mini review of each that I use enoug...
/keyboards#uhk60v2#defy#voyager#glove80#svalboard
5 5681 7
New
xiji2646-netizen
AI>Chat

Claude Code's entire source just leaked (512K lines) - anyone else digging through it?
Woke up to this today: Claude Code’s complete source code exposed via npm source map. Not a snippet. All 512,000 lines. 1,900 TypeScript ...
#claude
6 8359 5
New
Back to top

Latest in Next-Level A/B Testing

Next-Level A/B Testing Portal ❯

PragProg Customers

Pragmatic Bookshelf PragProg Customers ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap