seblegall

seblegall

Distributed Services with Go - Method log.Read() fails reading Records

Hi,

I’m currently reading your book. By doing so, I like to rewrite the code by myself. It helps me to deeply understand how it works.

However, I found something strange in the log package, concerning the Read() func. Let’s see that with a test example :

In the log_test.go file, the original test is :

func testAppendRead(t *testing.T, log *log.Log) {
	append := &api.Record{
		Value: []byte("hello world"),
	}
	off, err := log.Append(append)
	require.NoError(t, err)
	require.Equal(t, uint64(0), off)

	read, err := log.Read(off)
	require.NoError(t, err)
	require.Equal(t, append, read)

}

In this test, we write 1 record and read it.
Let’s write 4 records and read them:

func testAppendRead(t *testing.T, log *log.Log) {

	for i := 0; i < 5; i++ {
		append := &api.Record{
			Value: []byte(fmt.Sprintf("Hello World %d !", i)),
		}
		off, err := log.Append(append)
		require.NoError(t, err)
		require.Equal(t, uint64(i), off)

		read, err := log.Read(off)
		require.NoError(t, err)
		require.Equal(t, append, read)
	}

}

Here is what I get by running go test -v log_test.go :

=== RUN   TestLog
=== RUN   TestLog/append_and_read_a_record_succeeds
    TestLog/append_and_read_a_record_succeeds: log_test.go:48: 
                Error Trace:    log_test.go:48
                                                        log_test.go:32
                Error:          Received unexpected error:
                                offset out of range: 2
                Test:           TestLog/append_and_read_a_record_succeeds

I’m not sure why it fails, but I have understood that, in a log, segments are ordered from the oldest to the newest. It means the oldest will have a base offset of 0 (for example), the second will have a base offset of 2.

But then, there is this code in the Read method :

    var s *segment
	for _, segment := range l.segments {
               //The first segment we read is the oldest one. That is to say, the one
               //with a base offset equals to 0, right?
 		if segment.baseOffset <= off {
			s = segment
			break
		}
	}
	if s == nil || s.nextOffset <= off {
		return nil, fmt.Errorf("offset out of range: %d", off)
	}

If we try to read offset 3, by reading the first (oldest) segment in the loop, we will have: segment.baseOffset = 0 <= 3 However, the Record with the offset 3 is in the segment which baseOffset is 2.

Did I miss something?

Most Liked

travisjeffery

travisjeffery

Author of Distributed Services with Go

Hey, yeah it should be implemented like this:

func (l *Log) Read(off uint64) (*api.Record, error) {
	l.mu.RLock()
	defer l.mu.RUnlock()
	var s *segment
	for _, segment := range l.segments {
		if segment.baseOffset <= off && off < segment.nextOffset {
			s = segment
			break
		}
	}
	// START: before
	if s == nil || s.nextOffset <= off {
		return nil, fmt.Errorf("offset out of range: %d", off)
	}
	// END: before
	return s.Read(off)
}

I fixed a ton of issues that will be in the next beta, which should be out either late this week or next week.

Where Next?

Popular Pragmatic Bookshelf topics Top

telemachus
Python Testing With Pytest - Chapter 2, warnings for “unregistered custom marks” While running the smoke tests in Chapter 2, I get these...
New
GilWright
Working through the steps (checking that the Info,plist matches exactly), run the demo game and what appears is grey but does not fill th...
New
mikecargal
Title: Hands-on Rust: question about get_component (page 295) (feel free to respond. “You dug you’re own hole… good luck”) I have somet...
New
swlaschin
The book has the same “Problem space/Solution space” diagram on page 18 as is on page 17. The correct Problem/Solution space diagrams ar...
New
brian-m-ops
#book-python-testing-with-pytest-second-edition Hi. Thanks for writing the book. I am just learning so this might just of been an issue ...
New
brunogirin
When I run the coverage example to report on missing lines, I get: pytest --cov=cards --report=term-missing ch7 ERROR: usage: pytest [op...
New
EdBorn
Title: Agile Web Development with Rails 7: (page 70) I am running windows 11 pro with rails 7.0.3 and ruby 3.1.2p20 (2022-04-12 revision...
New
Keton
When running the program in chapter 8, “Implementing Combat”, the printout Health before attack was never printed so I assumed something ...
New
ggerico
I got this error when executing the plot files on macOS Ventura 13.0.1 with Python 3.10.8 and matplotlib 3.6.1: programming_ML/code/03_...
New
redconfetti
Docker-Machine became part of the Docker Toolbox, which was deprecated in 2020, long after Docker Desktop supported Docker Engine nativel...
New

Other popular topics Top

AstonJ
There’s a whole world of custom keycaps out there that I didn’t know existed! Check out all of our Keycaps threads here: https://forum....
New
PragmaticBookshelf
Rust is an exciting new programming language combining the power of C with memory safety, fearless concurrency, and productivity boosters...
New
AstonJ
In case anyone else is wondering why Ruby 3 doesn’t show when you do asdf list-all ruby :man_facepalming: do this first: asdf plugin-upd...
New
Maartz
Hi folks, I don’t know if I saw this here but, here’s a new programming language, called Roc Reminds me a bit of Elm and thus Haskell. ...
New
First poster: bot
zig/http.zig at 7cf2cbb33ef34c1d211135f56d30fe23b6cacd42 · ziglang/zig. General-purpose programming language and toolchain for maintaini...
New
NewsBot
Node.js v22.14.0 has been released. Link: Release 2025-02-11, Version 22.14.0 'Jod' (LTS), @aduh95 · nodejs/node · GitHub
New
PragmaticBookshelf
A concise guide to MySQL 9 database administration, covering fundamental concepts, techniques, and best practices. Neil Smyth MySQL...
New
PragmaticBookshelf
This book forgoes the abstract and instead provides concrete examples to help you better leverage the unique properties of Elixir, Erlang...
New
PragmaticBookshelf
As digital systems increasingly run the world, mastery of the recurring patterns of software development risk is the key to fast and effe...
New
PragmaticBookshelf
Quickly turn complex problems into simple, working solutions using functional programming, safe concurrency, and the expressive tools of ...
New

Sub Categories: