PragProg Customers

Next-Level A/B is Testing: 'inconclusive results' is ambiguous (p. 12)

dlindelof

dlindelof

Next-Level A/B is Testing: 'inconclusive results' is ambiguous (p. 12)

@leemay

The book refers to ‘inconclusive results’ starting on page 12. I would caution against introducing this term without carefully defining it. At my dayjob I frequently deal with product managers who believe that an A/B experiment with a neutral outcome is ‘inconclusive’, when in fact it just says that there is no statistically significant difference between the two arms.

2 comments
/book-next-level-a-b-testing
1 27 2

Most Liked

leemay

leemay

Author of Practical A/B Testing

That’s a really good point. I’ll do exactly as you suggest, the details matter. I’ll define and give caveats of what it looks like in practice/what that could mean. Luckily we’re still in beta so I can make these changes this week. Thank you @dlindelof!

1
Post #3

Where Next?

View thread on forum
View Next Level A B Testing's book portal
Post errata for this book
Post a suggestion for this book
Post a question about this book
Post a review about this book
Home PragProg Customers
/book-next-level-a-b-testing
1 27 2
Last post

Popular Pragmatic Bookshelf topics Top

jimmykiang
PragProg Customers

Distributed Services With Go - Agent_test.go error line 77
This test is broken right out of the box… — FAIL: TestAgent (7.82s) agent_test.go:77: Error Trace: agent_test.go:77 agent_test.go:...
#errata/book-distributed-services-with-go
4 1559 7
New
jon
PragProg Customers

The Pragmatic Programmer, 20th Anniversary Edition errata
Some minor things in the paper edition that says “3 2020” on the title page verso, not mentioned in the book’s errata online: p. 186 But...
#errata/book-the-pragmatic-programmer-20th-anniversary-edition
10 2636 9
New
iPaul
PragProg Customers

The Definitive ANTLR 4 Reference: Program deprecation warning
page 37 ANTLRInputStream input = new ANTLRInputStream(is); as of ANTLR 4 .8 should be: CharStream stream = CharStreams.fromStream(i...
#errata/book-the-definitive-antlr-4-reference
4 987 0
New
belgoros
PragProg Customers

Effective Testing with RSpec 3: Running Sequel migrations fails (page 84)
Following the steps described in Chapter 6 of the book, I’m stuck with running the migration as described on page 84: bundle exec sequel...
#errata/book-effective-testing-with-rspec-3
4 1152 1
New
patoncrispy
PragProg Customers

Hands-on Rust: Adequate docs for bracket_lib
I’m new to Rust and am using this book to learn more as well as to feed my interest in game dev. I’ve just finished the flappy dragon exa...
#question/book-hands-on-rust
5 1516 3
New
brunogirin
PragProg Customers

Python Testing with pytest, Second Edition: unrecognized arguments error (page 122)
When I run the coverage example to report on missing lines, I get: pytest --cov=cards --report=term-missing ch7 ERROR: usage: pytest [op...
#errata/book-python-testing-with-pytest-second-edition
0 4000 1
New
Charles
PragProg Customers

Programming Phoenix LiveView: book not updated to version 1.6 (p. 18)
In general, the book isn’t yet updated for Phoenix version 1.6. On page 18 of the book, the authors indicate that an auto generated of ro...
#errata/book-programming-phoenix-liveview
9 2051 12
New
brunogirin
PragProg Customers

Python Testing with pytest, Second Edition: error when installing editable package (page 178)
When installing Cards as an editable package, I get the following error: ERROR: File “setup.py” not found. Directory cannot be installe...
#errata/book-python-testing-with-pytest-second-edition
0 1739 1
New
akraut
PragProg Customers

Programming Phoenix LiveView: Display Image Uploads (pdf p142)
The markup used to display the uploaded image results in a Phoenix.LiveView.HTMLTokenizer.ParseError error. lib/pento_web/live/product_l...
#errata/book-programming-phoenix-liveview
2 2102 9
New
dachristenson
PragProg Customers

Kotlin and Android Development featuring Jetpack: No activity_main.xml anymore
I just bought this book to learn about Android development, and I’m already running into a major issue in Ch. 1, p. 20: “Update activity...
#errata/book-kotlin-and-android-development-featuring-jetpack
0 2172 6
New

Other popular topics Top

PragmaticBookshelf
General Dev>Learning Resources

The Pragmatic Programmer, 20th Anniversary Edition
Andy and Dave wrote this influential, classic book to help their clients create better software and rediscover the joy of coding. Almost ...
pragprog.com
#pragprog#published-book/book-the-pragmatic-programmer-20th-anniversary-edition
4 4782 0
New
PragmaticBookshelf
General Dev>Learning Resources

Forge Your Future with Open Source
Free and open source software is the default choice for the technologies that run our world, and it’s built and maintained by people like...
pragprog.com
#pragprog#published-book/book-forge-your-future-with-open-source
3 5654 0
New
Exadra37
Linux>Questions

AMD or Intel for Programming with Linux as the OS?
I am thinking in building or buy a desktop computer for programing, both professionally and on my free time, and my choice of OS is Linux...
#mobile#android#web-development#linux#desktop-computer#mobile-development
36 6006 10
New
PragmaticBookshelf
Game Dev>Learning Resources

Apple Game Frameworks and Technologies
Design and develop sophisticated 2D games that are as much fun to make as they are to play. From particle effects and pathfinding to soci...
pragprog.com
#pragprog#ios#game-dev#macos/swift#published-book#apple/book-apple-game-frameworks-and-technologies
30 7995 10
New
PragmaticBookshelf
Data Science

Genetic Algorithms in Elixir
From finance to artificial intelligence, genetic algorithms are a powerful tool with a wide array of applications. But you don't need an ...
#pragprog#ai/elixir#published-book/book-genetic-algorithms-in-elixir
25 5243 6
New
DevotionGeo
General Dev>Questions

Do you prefer regular mechanical keyboards or low profile mechanical keyboards and why?
I have always used antique keyboards like Cherry MX 1800 or Cherry MX 8100 and almost always have modified the switches in some way, like...
/keyboards#mechanical-keyboards
27 5496 10
New
First poster: bot
General Dev>In The News

Zig now has built-in HTTP server and client in std
zig/http.zig at 7cf2cbb33ef34c1d211135f56d30fe23b6cacd42 · ziglang/zig. General-purpose programming language and toolchain for maintaini...
github.com
/zig#http
0 5624 0
New
CommunityNews
General Dev>In The News

Review of Linux on Minisforum V3 AMD Ryzen Tablet
A Brief Review of the Minisforum V3 AMD Tablet. Update: I have created an awesome-minisforum-v3 GitHub repository to list information fo...
mudkip.me
#linux#review#amd
0 4635 0
New
PragmaticBookshelf
Backend>Learning Resources

Ash Framework
Explore the power of Ash Framework by modeling and building the domain for a real-world web application. Rebecca Le @sevenseacat and ...
pragprog.com
#pragprog/elixir#published-book/ash/book-ash-framework
15 7555 9
New
PragmaticBookshelf
Backend>Learning Resources

Advanced Functional Programming with Elixir
Use advanced functional programming principles, practical Domain-Driven Design techniques, and production-ready Elixir code to build scal...
joekoski.com
#pragprog/elixir#published-book#functional-programming/book-advanced-functional-programming-with-elixir
43 4989 22
New
Back to top

Latest in Next-Level A/B Testing

Next-Level A/B Testing Portal ❯

PragProg Customers

Pragmatic Bookshelf PragProg Customers ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap