Qqwy

Qqwy

PropCheck - Ruby library for property-based testing

Hi everyone!

Property-based testing is amazing: You specify what types of input values you expect and what kinds of properties are expected to hold true for all of those possible inputs, and the runtime is then able to automatically create and run thousands of test-cases for you. This often uncovers edge-cases that you might not have thought of if you were trying to come up with unit-test examples by hand.

What is even more cool is that when a test failure is encountered, the input can be shrunken back to the most simplest form that still results in an error, making the feedback very human-friendly.

Property-based testing can be considered a ‘pragmatic’ approach to ‘theorem-proving’. Mathematically proving that something holds for all cases is often very difficult, whilst stating that something should hold for all cases and then lobbing thousands of possible input sets at it is a lot easier to make. Of course this only allows to prove the presence (rather than the absence) of bugs, but because of the large number of input sets (and the fact that every time you run the test, yet more different input sets are tried), it converges to the same.

Besides working in a couple of language where property-testing is wide-spread I do quite a bit of consulting work in Ruby. Until now, Ruby did not have a mature library to do property-based testing.
So I wrote one :smiley:!

PropCheck

Gem Build Status Maintainability RubyDoc

It features:

  • Generators for common datatypes.
  • An easy DSL to define your own generators (by combining existing ones, or make completely custom ones).
  • Shrinking to a minimal counter-example on failure.

Usage Example

Here we check if naive_average indeed always returns an integer for any and all arrays of integers we can pass it:

# Somewhere you have this function definition:
def naive_average(array)
  array.sum / array.length
end

# And then in a test case:
include PropCheck::Generators
PropCheck.forall(numbers: array(integer)) do |numbers:|
  result = naive_average(numbers)
  unless result.is_a?(Integer) do
    raise "Expected the average to always return an integer!"
  end
end

When running this particular example PropCheck very quickly finds out that we have made a programming mistake:

ZeroDivisionError: 
(after 6 successful property test runs)
Failed on: 
`{
    :numbers => []
}`

Exception message:
---
divided by 0
---

(shrinking impossible)
---

Clearly we forgot to handle the case of an empty array being passed to the function.
This is a good example of the kind of conceptual bugs that PropCheck (and property-based testing in general) are able to check for.

(If we were e.g. using RSpec, we might have structured the test as follows:

describe "#naive_average" do
  include PropCheck
  include PropCheck::Generators

  it "returns an integer for any input" do
    forall(numbers: array(integer)) do |numbers:|
      result = naive_average(numbers)      
      expect(result).to be_a(Integer)
    end
  end
end

)

PropCheck comes with many built-in data-generators and it is easy to build your own on top of these.


Check it out now! I’m eager to hear your feedback :smiley:!

Most Liked

Qqwy

Qqwy

I very much recommend that book as well. Much of what I know about Property-based testing I learned from that book.

After finishing it I looked into the implementations of Elixir’s StreamData, Erlang’s PropEr and Haskell’s QuickCheck and Hedgehog to get some of the final details down that are necessary to actually implement your own property-checking library (like how shrinking actually works under the hood. It is amazing :exploding_head:).

AstonJ

AstonJ

Please consider writing reviews for books - they show up in the book’s portal and could really help others (as well as the authors) :blush:

AstonJ

AstonJ

Nice one Marten!

Popular Cross Platform topics Top

axelson
I’ve been really enjoying obsidian.md: It is very snappy (even though it is based on Electron). I love that it is all local by defaul...
New
AstonJ
Just discovered this: Wireshark is the world’s foremost and widely-used network protocol analyzer. It lets you see what’s happening on...
New
Qqwy
Hi everyone! Property-based testing is amazing: You specify what types of input values you expect and what kinds of properties are expec...
New
wolf4earth
I have to mention Notion.so, especially because they recently went completely free for personal usage (I had a subscription in before). ...
New
bot
FreeCAD/FreeCAD. This is the official source code of FreeCAD, a free and opensource multiplatform 3D parametric modeler. Issues are mana...
New
First poster: bot
Theseus is a new OS written from scratch in Rust to experiment with novel OS structure, better state management, and how to shift OS resp...
New
First poster: bot
WezTerm is a GPU-accelerated cross-platform terminal emulator and multiplexer written by @wez and implemented in Rust Features Runs on ...
New
CommunityNews
Finally, a command line shell for the 90s fish is a smart and user-friendly command line shell for Linux, macOS, and the rest of the fa...
New
First poster: bot
Recapping from the first post introducing this project, DataStation is an IDE for manipulating data (from HTTP servers, SQL databases, lo...
New

Other popular topics Top

AstonJ
A thread that every forum needs! Simply post a link to a track on YouTube (or SoundCloud or Vimeo amongst others!) on a separate line an...
New
AstonJ
SpaceVim seems to be gaining in features and popularity and I just wondered how it compares with SpaceMacs in 2020 - anyone have any thou...
New
AstonJ
Curious to know which languages and frameworks you’re all thinking about learning next :upside_down_face: Perhaps if there’s enough peop...
New
AstonJ
I have seen the keycaps I want - they are due for a group-buy this week but won’t be delivered until October next year!!! :rofl: The Ser...
New
AstonJ
Do the test and post your score :nerd_face: :keyboard: If possible, please add info such as the keyboard you’re using, the layout (Qw...
New
PragmaticBookshelf
“Finding the Boundaries” Hero’s Journey with Noel Rappin @noelrappin Even when you’re ultimately right about what the future ho...
New
Exadra37
Oh just spent so much time on this to discover now that RancherOS is in end of life but Rancher is refusing to mark the Github repo as su...
New
PragmaticBookshelf
Learn different ways of writing concurrent code in Elixir and increase your application's performance, without sacrificing scalability or...
New
PragmaticBookshelf
Author Spotlight: Karl Stolley @karlstolley Logic! Rhetoric! Prag! Wow, what a combination. In this spotlight, we sit down with Karl ...
New
PragmaticBookshelf
Author Spotlight: Tammy Coron @Paradox927 Gaming, and writing games in particular, is about passion, vision, experience, and immersio...
New