PragProg Customers

Next-Level A/B is Testing: 'inconclusive results' is ambiguous (p. 12)

dlindelof

dlindelof

Next-Level A/B is Testing: 'inconclusive results' is ambiguous (p. 12)

@leemay

The book refers to ‘inconclusive results’ starting on page 12. I would caution against introducing this term without carefully defining it. At my dayjob I frequently deal with product managers who believe that an A/B experiment with a neutral outcome is ‘inconclusive’, when in fact it just says that there is no statistically significant difference between the two arms.

2 comments
/book-next-level-a-b-testing
1 16 2

Most Liked

leemay

leemay

Author of Practical A/B Testing

That’s a really good point. I’ll do exactly as you suggest, the details matter. I’ll define and give caveats of what it looks like in practice/what that could mean. Luckily we’re still in beta so I can make these changes this week. Thank you @dlindelof!

1
Post #3

Where Next?

View thread on forum
View Next Level A B Testing's book portal
Post errata for this book
Post a suggestion for this book
Post a question about this book
Post a review about this book
Home PragProg Customers
/book-next-level-a-b-testing
1 16 2
Last post

Popular Pragmatic Bookshelf topics Top

jesse050717
PragProg Customers

Web Development with Clojure, Third Edition: Issue adding Shadow CLJS
Title: Web Development with Clojure, Third Edition, pg 116 Hi - I just started chapter 5 and I am stuck on page 116 while trying to star...
#question/book-web-development-with-clojure-third-edition
4 1191 4
New
raul
PragProg Customers

Distributed Services with Go: Chapter 1 Suggestions
Hi Travis! Thank you for the cool book! :slight_smile: I made a list of issues and thought I could post them chapter by chapter. I’m rev...
#errata/book-distributed-services-with-go
2 1179 3
New
alanq
PragProg Customers

Modern Front-End Development for Rails: Can't get TURBO_STREAM format in some cases
This isn’t directly about the book contents so maybe not the right forum…but in some of the code apps (e.g. turbo/06) it sends a TURBO_ST...
#question/book-modern-front-end-development-for-rails
1 1367 7
New
rmurray10127
PragProg Customers

Intuitive Python: docker run… denied error (page 2)
Title: Intuitive Python: docker run… denied error (page 2) Attempted to run the docker command in both CLI and Powershell PS C:\Users\r...
#errata/book-intuitive-python
9 1256 4
New
adamwoolhether
PragProg Customers

Distributed Services With Go: Unable to pass readiness/liveness checks. (page 210, 215)
I’m not quite sure what’s going on here, but I’m unable to have to containers successfully complete the Readiness/Liveness checks. I’m im...
/book-distributed-services-with-go
5 1406 5
New
jskubick
PragProg Customers

Kotlin and Android Development featuring Jetpack: android:tint vs app:tint (chapter 7, p186)
I found an issue in Chapter 7 regarding android:backgroundTint vs app:backgroundTint. How to replicate: load chapter-7 from zipfile i...
#errata/book-kotlin-and-android-development-featuring-jetpack
0 3500 3
New
mert
PragProg Customers

Agile Web Development with Rails 7: page 152, page 153:
AWDWR 7, page 152, page 153: Hello everyone, I’m a little bit lost on the hotwire part. I didn’t fully understand it. On page 152 @rub...
#question/book-agile-web-development-with-rails-7
0 1225 4
New
tkhobbes
PragProg Customers

Modern Front-End Development for Rails - application does not start after run bin/setup (page xviii)
After some hassle, I was able to finally run bin/setup, now I have started the rails server but I get this error message right when I vis...
#errata/book-modern-front-end-development-for-rails
0 1460 5
New
andreheijstek
PragProg Customers

Modern front-end development for Rails, second edition - Struggling to get the first chapter to work
After running /bin/setup, the first error was: The foreman' command exists in these Ruby versions: That was easy to fix: gem install fore...
#errata/book-modern-front-end-development-for-rails
1 1111 3
New
ggerico
PragProg Customers

Programming Machine Learning: some plot files are throwing error
I got this error when executing the plot files on macOS Ventura 13.0.1 with Python 3.10.8 and matplotlib 3.6.1: programming_ML/code/03_...
#question/book-programming-machine-learning
1 3040 5
New

Other popular topics Top

Devtalk
General Dev>Dev Chat

What dev-related stuff have you been up to?
Reading something? Working on something? Planning something? Changing jobs even!? If you’re up for sharing, please let us know what you’...
#community
1033 17470 383
New
malloryerik
Frontend>Chat

Svelte - what do you think about it?
Any thoughts on Svelte? Svelte is a radical new approach to building user interfaces. Whereas traditional frameworks like React and Vue...
#opinions/svelte
54 3555 17
New
axelson
Cross Platform>Libraries/Tools

Obsidian – a cross platform app to help you create a knowledge base with Markdown files
I’ve been really enjoying obsidian.md: It is very snappy (even though it is based on Electron). I love that it is all local by defaul...
#apps#macos#windows#linux/markdown#obsidian
38 4062 19
New
AstonJ
General Dev>Code Editors

Poll: Which code editor do you use?
You might be thinking we should just ask who’s not using VSCode :joy: however there are some new additions in the space that might give V...
#notepad
118 4871 60
New
Rainer
Community>Journals

Programming Erlang Book Club
My first contact with Erlang was about 2 years ago when I used RabbitMQ, which is written in Erlang, for my job. This made me curious and...
/erlang/book-programming-erlang-2nd-edition#book-club
195 6499 95
New
Exadra37
Linux>Chat

RancherOS is in end of life
Oh just spent so much time on this to discover now that RancherOS is in end of life but Rancher is refusing to mark the Github repo as su...
#linux#rancheros
10 5256 6
New
PragmaticBookshelf
Backend>Learning Resources

Python Testing with pytest, Second Edition
Create efficient, elegant software tests in pytest, Python's most powerful testing framework. Brian Okken @brianokken Edited by Kat...
pragprog.com
#pragprog/python#published-book/book-python-testing-with-pytest-second-edition
16 3580 4
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: David Bryant Copeland (Author) Interview and AMA!
Author Spotlight: David Bryant Copeland @davetron5000 We’re so happy to bring you another Author Spotlight, a series where we sit dow...
#author-spotlight/book-agile-web-development-with-rails-6/book-rails-angular-postgres-and-bootstrap-second-edition/book-sustainable-web-development-with-ruby-on-rails
19 3459 15
New
hilfordjames
Windows>Chat

Taskbar Overflow Menu (NOT System Tray Overflow)
There appears to have been an update that has changed the terminology for what has previously been known as the Taskbar Overflow - this h...
#taskbar-overflow-win-11
3 1664 2
New
CommunityNews
General Dev>In The News

X can’t stop spread of explicit, fake AI Taylor Swift images
Will Swifties’ war on AI fakes spark a deepfake porn reckoning?
arstechnica.com
/swift
0 5956 0
New
Back to top

Latest in Next-Level A/B Testing

Next-Level A/B Testing Portal ❯

PragProg Customers

Pragmatic Bookshelf PragProg Customers ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap