PragProg Customers

Next-Level A/B is Testing: 'inconclusive results' is ambiguous (p. 12)

dlindelof

dlindelof

Next-Level A/B is Testing: 'inconclusive results' is ambiguous (p. 12)

@leemay

The book refers to ‘inconclusive results’ starting on page 12. I would caution against introducing this term without carefully defining it. At my dayjob I frequently deal with product managers who believe that an A/B experiment with a neutral outcome is ‘inconclusive’, when in fact it just says that there is no statistically significant difference between the two arms.

2 comments
/book-next-level-a-b-testing
1 27 2

Most Liked

leemay

leemay

Author of Practical A/B Testing

That’s a really good point. I’ll do exactly as you suggest, the details matter. I’ll define and give caveats of what it looks like in practice/what that could mean. Luckily we’re still in beta so I can make these changes this week. Thank you @dlindelof!

1
Post #3

Where Next?

View thread on forum
View Next Level A B Testing's book portal
Post errata for this book
Post a suggestion for this book
Post a question about this book
Post a review about this book
Home PragProg Customers
/book-next-level-a-b-testing
1 27 2
Last post

Popular Pragmatic Bookshelf topics Top

mikecargal
PragProg Customers

Hands-On Rust: Wrong multiplier for 8x8 console in main.rs
Title: Hands-On Rust (Chap 8 (Adding a Heads Up Display) It looks like ​.with_simple_console_no_bg​(SCREEN_WIDTH*2, SCREEN_HEIGHT*2...
#errata/book-hands-on-rust
9 2026 5
New
cro
PragProg Customers

Programming Phoenix LiveView: P27, live_path confusion
I am working on the “Your Turn” for chapter one and building out the restart button talked about on page 27. It recommends looking into ...
#question/book-programming-phoenix-liveview
7 3505 14
New
gilesdotcodes
PragProg Customers

Modern CSS with Tailwind: setting up Rails app
In case this helps anyone, I’ve had issues setting up the rails source code. Here were the solutions: In Gemfile, change gem 'rails' t...
#errata/book-modern-css-with-tailwind
3 1278 2
New
Chrichton
PragProg Customers

Programming Phoenix LiveView: B03 (page 56-57) auth exercise
Dear Sophie. I tried to do the “Authorization” exercise and have two questions: When trying to plug in an email-service, I found the ...
#question/book-programming-phoenix-liveview
1 1181 3
New
swlaschin
PragProg Customers

Domain Modeling Made Functional: Missing diagram on page 18
The book has the same “Problem space/Solution space” diagram on page 18 as is on page 17. The correct Problem/Solution space diagrams ar...
#errata/book-domain-modeling-made-functional
1 1078 1
New
leonW
PragProg Customers

Python Testing with pytest, Second Edition: File not Found Error (page 43)
I ran this command after installing the sample application: $ cards add do something --owner Brian And got a file not found error: Fil...
#errata/book-python-testing-with-pytest-second-edition
5 4391 7
New
curtosis
PragProg Customers

Build a Weather Station with Elixir and Nerves: 'mix deps.get' does not work if user has not created a keyfile (page 16)
Running mix deps.get in the sensor_hub directory fails with the following error: ** (Mix) No SSH public keys found in ~/.ssh. An ssh aut...
#errata/book-build-a-weather-station-with-elixir-and-nerves
2 1457 3
New
jgchristopher
PragProg Customers

Programming Phoenix LiveView: live_modal/2 instead of live_component/3 (210)
“The ProductLive.Index template calls a helper function, live_component/3, that in turn calls on the modal component. ” Excerpt From: Br...
#errata/book-programming-phoenix-liveview
3 1042 6
New
bjnord
PragProg Customers

Hands-on Rust: Chapter 3: Can't get first hello to run
Hello @herbert ! Trying to get the very first “Hello, Bracket Terminal!" example to run (p. 53). I develop on an Amazon EC2 instance runn...
#question/book-hands-on-rust
0 1147 2
New
gorkaio
PragProg Customers

Programming Phoenix LiveView: PentoWeb.LayoutView errata (56)
root_layout: {PentoWeb.LayoutView, :root}, This results in the following following error: no “root” html template defined for PentoWeb...
#errata/book-programming-phoenix-liveview
8 1094 4
New

Other popular topics Top

PragmaticBookshelf
Game Dev>Learning Resources

The Ray Tracer Challenge
Brace yourself for a fun challenge: build a photorealistic 3D renderer from scratch! In just a couple of weeks, build a ray tracer that r...
pragprog.com
#pragprog#published-book/book-the-ray-tracer-challenge#algorithms
3 6115 0
New
PragmaticBookshelf
Backend>Learning Resources

Seven Languages in Seven Weeks
Ruby, Io, Prolog, Scala, Erlang, Clojure, Haskell. With Seven Languages in Seven Weeks, by Bruce A. Tate, you’ll go beyond the syntax—and...
pragprog.com
#pragprog/clojure/erlang/haskell/prolog/ruby/scala#published-book/book-seven-languages-in-seven-weeks
5 5730 1
New
AstonJ
General Dev>Hardware

Custom keyboard keycaps
There’s a whole world of custom keycaps out there that I didn’t know existed! Check out all of our Keycaps threads here: https://forum....
#hardware/keyboards#keycaps#mechanical-keyboards
15 11086 19
New
dimitarvp
macOS>Chat

My thoughts on macOS vs Linux
Small essay with thoughts on macOS vs. Linux: I know @Exadra37 is just waiting around the corner to scream at me “I TOLD YOU SO!!!” but I...
#macos#linux
166 9730 69
New
PragmaticBookshelf
Frontend>Learning Resources

Modern CSS with Tailwind
Tailwind CSS is an exciting new CSS framework that allows you to design your site by composing simple utility classes to create complex e...
pragprog.com
#pragprog/tailwind#published-book/book-modern-css-with-tailwind
12 5813 4
New
DevotionGeo
General Dev>Dev Chat

The V Programming Language
The V Programming Language Simple language for building maintainable programs V is already mentioned couple of times in the forum, but I...
#programminguages/v
21 13874 7
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Rebecca Skinner (Author) Interview and AMA!
Author Spotlight Rebecca Skinner @RebeccaSkinner Welcome to our latest author spotlight, where we sit down with Rebecca Skinner, auth...
#author-spotlight/haskell/book-effective-haskell
106 11719 28
New
PragmaticBookshelf
Backend>Learning Resources

Ash Framework
Explore the power of Ash Framework by modeling and building the domain for a real-world web application. Rebecca Le @sevenseacat and ...
pragprog.com
#pragprog/elixir#published-book/ash/book-ash-framework
15 7555 9
New
RobertRichards
Game Dev>Chat

Hair Salon Games for Girls Fun
Hair Salon Games for Girls Fun Girls Hair Saloon game is mainly developed for kids. This game allows users to select virtual avatars to ...
#ios#android#unity#fun
2 3556 1
New
CommunityNews
Game Dev>In The News

Grand Theft Auto: Vice City | DOS games in browser
Open-source implementation of the classic GTA engine now running directly in your browser. Experience the reVC technology demo on DOS.Zon...
dos.zone
#games#browser
0 173 0
New
Back to top

Latest in Next-Level A/B Testing

Next-Level A/B Testing Portal ❯

PragProg Customers

Pragmatic Bookshelf PragProg Customers ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap