PragProg Customers

Next-Level A/B is Testing: 'inconclusive results' is ambiguous (p. 12)

dlindelof

dlindelof

Next-Level A/B is Testing: 'inconclusive results' is ambiguous (p. 12)

@leemay

The book refers to ‘inconclusive results’ starting on page 12. I would caution against introducing this term without carefully defining it. At my dayjob I frequently deal with product managers who believe that an A/B experiment with a neutral outcome is ‘inconclusive’, when in fact it just says that there is no statistically significant difference between the two arms.

2 comments
/book-next-level-a-b-testing
1 27 2

Most Liked

leemay

leemay

Author of Practical A/B Testing

That’s a really good point. I’ll do exactly as you suggest, the details matter. I’ll define and give caveats of what it looks like in practice/what that could mean. Luckily we’re still in beta so I can make these changes this week. Thank you @dlindelof!

1
Post #3

Where Next?

View thread on forum
View Next Level A B Testing's book portal
Post errata for this book
Post a suggestion for this book
Post a question about this book
Post a review about this book
Home PragProg Customers
/book-next-level-a-b-testing
1 27 2
Last post

Popular Pragmatic Bookshelf topics Top

cro
PragProg Customers

Programming Phoenix LiveView: P27, live_path confusion
I am working on the “Your Turn” for chapter one and building out the restart button talked about on page 27. It recommends looking into ...
#question/book-programming-phoenix-liveview
7 3505 14
New
jskubick
PragProg Customers

Kotlin and Android Development featuring Jetpack: SwitchCompat's thumbTint and trackTint being ignored (page 76)
I think I might have found a problem involving SwitchCompat, thumbTint, and trackTint. As entered, the SwitchCompat changes color to hol...
#errata/book-kotlin-and-android-development-featuring-jetpack
0 2749 2
New
hgkjshegfskef
PragProg Customers

The Ray Tracer Challenge: cannot pass scaling test (page 69-70)
The test is as follows: Scenario: Intersecting a scaled sphere with a ray Given r ← ray(point(0, 0, -5), vector(0, 0, 1)) And s ← sphere...
#question/book-the-ray-tracer-challenge
1 1093 4
New
creminology
PragProg Customers

Real Time Phoenix Errata P17: Getting first sample working with Phoenix 1.6.x
Skimming ahead, much of the following is explained in Chapter 3, but new readers (like me!) will hit a roadblock in Chapter 2 with their ...
#errata/book-real-time-phoenix
6 1116 5
New
s2k
PragProg Customers

Agile Web Development with Rails 7: Reloading assets in development mode
Hi all, currently I wonder how the Tailwind colours work (or don’t work). For example, in app/views/layouts/application.html.erb I have...
#question/book-agile-web-development-with-rails-7
2 1380 4
New
jwandekoken
PragProg Customers

Programming Phoenix LiveView: Live Upload Match Error
Book: Programming Phoenix LiveView, page 142 (157/378), file lib/pento_web/live/product_live/form_component.ex, in the function below: d...
#errata/book-programming-phoenix-liveview
7 1158 10
New
tkhobbes
PragProg Customers

Modern Front-End Development for Rails - application does not start after run bin/setup (page xviii)
After some hassle, I was able to finally run bin/setup, now I have started the rails server but I get this error message right when I vis...
#errata/book-modern-front-end-development-for-rails
0 1461 5
New
Keton
PragProg Customers

Hands-on Rust: How do I debug a call chain in Rust?
When running the program in chapter 8, “Implementing Combat”, the printout Health before attack was never printed so I assumed something ...
#question/book-hands-on-rust
2 1034 2
New
NaplesDave
PragProg Customers

PragProg Customers Kotlin and Android Development: pg. 8 Build Gradle error: Could not get unknown property 'kotlin_version'
I am using Android Studio Chipmunk | 2021.2.1 Patch 2 Build #AI-212.5712.43.2112.8815526, built on July 10, 2022 Runtime version: 11.0....
#question/book-kotlin-and-android-development-featuring-jetpack
0 1456 2
New
ggerico
PragProg Customers

Programming Machine Learning: some plot files are throwing error
I got this error when executing the plot files on macOS Ventura 13.0.1 with Python 3.10.8 and matplotlib 3.6.1: programming_ML/code/03_...
#question/book-programming-machine-learning
1 3055 5
New

Other popular topics Top

DevotionGeo
Backend>Chat

What is the reason behind Rust’s web framework, Rocket, not performing as well as expected in the Techempower benchmarks?
I know that these benchmarks might not be the exact picture of real-world scenario, but still I expect a Rust web framework performing a ...
#web-frameworks/rust
36 7463 11
New
AstonJ
General Dev>Hardware

Poll: Which keyboard layout do you use?
poll poll Be sure to check out @Dusty’s article posted here: An Introduction to Alternative Keyboard Layouts It’s one of the best write-...
colemakmods.github.io
#polls/keyboards
10 6048 11
New
AstonJ
General Dev>Hardware

Custom keyboard keycaps
There’s a whole world of custom keycaps out there that I didn’t know existed! Check out all of our Keycaps threads here: https://forum....
#hardware/keyboards#keycaps#mechanical-keyboards
15 11086 19
New
AstonJ
General Dev>Hardware

BIIP MT3 Extended 2048 Custom Keycap Set (Drop)
This looks like a stunning keycap set :orange_heart: A LEGENDARY KEYBOARD LIVES ON When you bought an Apple Macintosh computer in the e...
/keyboards#apple#keycaps#mechanical-keyboards
14 6713 7
New
PragmaticBookshelf
Backend>Learning Resources

Programming Phoenix LiveView
Build highly interactive applications without ever leaving Elixir, the way the experts do. Let LiveView take care of performance, scalabi...
pragprog.com
#pragprog/elixir/phoenix#published-book/book-programming-phoenix-liveview
83 11955 26
New
PragmaticBookshelf
Backend>Learning Resources

Python Testing with pytest, Second Edition
Create efficient, elegant software tests in pytest, Python's most powerful testing framework. Brian Okken @brianokken Edited by Kat...
pragprog.com
#pragprog/python#published-book/book-python-testing-with-pytest-second-edition
16 7461 4
New
AstonJ
macOS>Chat

How to block any website on Mac using Little Snitch
If you want a quick and easy way to block any website on your Mac using Little Snitch simply… File > New Rule: And select Deny, O...
#macos#how-to#littlesnitch
5 11227 3
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: VM Brasseur (Author) Interview and AMA!
Author Spotlight: VM Brasseur @vmbrasseur We have a treat for you today! We turn the spotlight onto Open Source as we sit down with V...
#author-spotlight/book-forge-your-future-with-open-source
16 5051 11
New
CommunityNews
General Dev>In The News

Review of Linux on Minisforum V3 AMD Ryzen Tablet
A Brief Review of the Minisforum V3 AMD Tablet. Update: I have created an awesome-minisforum-v3 GitHub repository to list information fo...
mudkip.me
#linux#review#amd
0 4635 0
New
AstonJ
AI>Chat

How to: Run DeepSeek on Mac, Windows, and Linux!
This is a very quick guide, you just need to: Download LM Studio: https://lmstudio.ai/ Click on search Type DeepSeek, then select the o...
#macs/deepseek#guides#lm-studio
14 9328 10
New
Back to top

Latest in Next-Level A/B Testing

Next-Level A/B Testing Portal ❯

PragProg Customers

Pragmatic Bookshelf PragProg Customers ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap