PragProg Customers

Next-Level A/B is Testing: 'inconclusive results' is ambiguous (p. 12)

dlindelof

dlindelof

Next-Level A/B is Testing: 'inconclusive results' is ambiguous (p. 12)

@leemay

The book refers to ‘inconclusive results’ starting on page 12. I would caution against introducing this term without carefully defining it. At my dayjob I frequently deal with product managers who believe that an A/B experiment with a neutral outcome is ‘inconclusive’, when in fact it just says that there is no statistically significant difference between the two arms.

2 comments
/book-next-level-a-b-testing
1 27 2

Most Liked

leemay

leemay

Author of Practical A/B Testing

That’s a really good point. I’ll do exactly as you suggest, the details matter. I’ll define and give caveats of what it looks like in practice/what that could mean. Luckily we’re still in beta so I can make these changes this week. Thank you @dlindelof!

1
Post #3

Where Next?

View thread on forum
View Next Level A B Testing's book portal
Post errata for this book
Post a suggestion for this book
Post a question about this book
Post a review about this book
Home PragProg Customers
/book-next-level-a-b-testing
1 27 2
Last post

Popular Pragmatic Bookshelf topics Top

iPaul
PragProg Customers

The Definitive ANTLR 4 Reference: Program deprecation warning
page 37 ANTLRInputStream input = new ANTLRInputStream(is); as of ANTLR 4 .8 should be: CharStream stream = CharStreams.fromStream(i...
#errata/book-the-definitive-antlr-4-reference
4 987 0
New
brianokken
PragProg Customers

Python Testing with pytest: pytest now requires markers to be declared (downloadable code)
Many tasks_proj/tests directories exist in chapters 2, 3, 5 that have tests that use the custom markers smoke and get, which are not decl...
#errata/book-python-testing-with-pytest
2 1383 2
New
yulkin
PragProg Customers

Programming Flutter: Image.toByteData() does not exist (page40)
your book suggests to use Image.toByteData() to convert image to bytes, however I get the following error: "the getter ‘toByteData’ isn’t...
/book-programming-flutter#question
1 4102 2
New
raul
PragProg Customers

Distributed Services with Go: Chapter 3 suggestions
Page 28: It implements io.ReaderAt on the store type. Sorry if it’s a dumb question but was the io.ReaderAt supposed to be io.ReadAt? ...
#errata/book-distributed-services-with-go
0 1302 3
New
brian-m-ops
PragProg Customers

Python Testing with pytest, Second Edition: cards pip install error (page 12)
#book-python-testing-with-pytest-second-edition Hi. Thanks for writing the book. I am just learning so this might just of been an issue ...
#errata/book-python-testing-with-pytest-second-edition
4 1191 3
New
jskubick
PragProg Customers

Kotlin and Android Development featuring Jetpack: Error in generated Game.java (page 136)
I’m under the impression that when the reader gets to page 136 (“View Data with the Database Inspector”), the code SHOULD be able to buil...
#question/book-kotlin-and-android-development-featuring-jetpack
2 1172 8
New
oaklandgit
PragProg Customers

Hands-on Rust: Failed to Load Texture error (page 121)
Hi, I completed chapter 6 but am getting the following error when running: thread 'main' panicked at 'Failed to load texture: IoError(O...
/book-hands-on-rust
2 1310 3
New
adamwoolhether
PragProg Customers

Powerful Command-Line Applications in Go: Exercise Solutions (many pages)
Is there any place where we can discuss the solutions to some of the exercises? I can figure most of them out, but am having trouble with...
#question/book-powerful-command-line-applications-in-go
17 1396 9
New
creminology
PragProg Customers

Real Time Phoenix Errata P17: Getting first sample working with Phoenix 1.6.x
Skimming ahead, much of the following is explained in Chapter 3, but new readers (like me!) will hit a roadblock in Chapter 2 with their ...
#errata/book-real-time-phoenix
6 1116 5
New
roadbike
PragProg Customers

Programming Machine Learning: Setting up your system (p 13)
From page 13: On Python 3.7, you can install the libraries with pip by running these commands inside a Python venv using Visual Studio ...
#question/book-programming-machine-learning
1 989 3
New

Other popular topics Top

AstonJ
General Dev>Dev Chat

What are you listening to?
A thread that every forum needs! Simply post a link to a track on YouTube (or SoundCloud or Vimeo amongst others!) on a separate line an...
#community#music
202 5126 102
New
ohm
Science/Tech>Tech Chat

Games! Which do you play?
Which, if any, games do you play? On what platform? I just bought (and completed) Minecraft Dungeons for my Nintendo Switch. Other than ...
#games
246 6097 101
New
Rainer
Community>Journals

Programming Erlang Book Club
My first contact with Erlang was about 2 years ago when I used RabbitMQ, which is written in Erlang, for my job. This made me curious and...
/erlang/book-programming-erlang-2nd-edition#book-club
195 6815 95
New
AstonJ
Data Science

Can AI/ML predict a lottery win?
Biggest jackpot ever apparently! :upside_down_face: I don’t (usually) gamble/play the lottery, but working on a program to predict the...
#ai#machine-learning
19 3939 10
New
Help
Game Dev>Questions

Can I use Java to program a game for Nintendo switch?
I am trying to crate a game for the Nintendo switch, I wanted to use Java as I am comfortable with that programming language. Can you use...
/java#nintendo
8 4771 3
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Mike Riley (Author) Interview and AMA!
Author Spotlight Mike Riley @mriley This month, we turn the spotlight on Mike Riley, author of Portable Python Projects. Mike’s book ...
#author-spotlight/python#iot/book-portable-python-projects#internet-of-things
62 7035 19
New
PragmaticBookshelf
General Dev>Learning Resources

A Common-Sense Guide to Data Structures and Algorithms in Python, Volume 1
Big O Notation can make your code faster by orders of magnitude. Get the hands-on info you need to master data structures and algorithms ...
pragprog.com
#pragprog/python#published-book/book-a-common-sense-guide-to-data-structures-and-algorithms-in-python-volume-1
24 5988 11
New
First poster: AstonJ
General Dev>In The News

Jan: An open source alternative to ChatGPT that runs on the desktop
Jan | Rethink the Computer. Jan turns your computer into an AI machine by running LLMs locally on your computer. It’s a privacy-focus, l...
jan.ai
#desktop#chatgpt
4 5652 4
New
CommunityNews
General Dev>In The News

Review of Linux on Minisforum V3 AMD Ryzen Tablet
A Brief Review of the Minisforum V3 AMD Tablet. Update: I have created an awesome-minisforum-v3 GitHub repository to list information fo...
mudkip.me
#linux#review#amd
0 4635 0
New
PragmaticBookshelf
Backend>Learning Resources

Simplicity
Fight complexity and reclaim the original spirit of agility by learning to simplify how you develop software. The result: a more humane a...
pragprog.com
#pragprog#published-book/book-simplicity
10 6553 8
New
Back to top

Latest in Next-Level A/B Testing

Next-Level A/B Testing Portal ❯

PragProg Customers

Pragmatic Bookshelf PragProg Customers ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap