seblegall

seblegall

Distributed Services with Go - Method log.Read() fails reading Records

Hi,

I’m currently reading your book. By doing so, I like to rewrite the code by myself. It helps me to deeply understand how it works.

However, I found something strange in the log package, concerning the Read() func. Let’s see that with a test example :

In the log_test.go file, the original test is :

func testAppendRead(t *testing.T, log *log.Log) {
	append := &api.Record{
		Value: []byte("hello world"),
	}
	off, err := log.Append(append)
	require.NoError(t, err)
	require.Equal(t, uint64(0), off)

	read, err := log.Read(off)
	require.NoError(t, err)
	require.Equal(t, append, read)

}

In this test, we write 1 record and read it.
Let’s write 4 records and read them:

func testAppendRead(t *testing.T, log *log.Log) {

	for i := 0; i < 5; i++ {
		append := &api.Record{
			Value: []byte(fmt.Sprintf("Hello World %d !", i)),
		}
		off, err := log.Append(append)
		require.NoError(t, err)
		require.Equal(t, uint64(i), off)

		read, err := log.Read(off)
		require.NoError(t, err)
		require.Equal(t, append, read)
	}

}

Here is what I get by running go test -v log_test.go :

=== RUN   TestLog
=== RUN   TestLog/append_and_read_a_record_succeeds
    TestLog/append_and_read_a_record_succeeds: log_test.go:48: 
                Error Trace:    log_test.go:48
                                                        log_test.go:32
                Error:          Received unexpected error:
                                offset out of range: 2
                Test:           TestLog/append_and_read_a_record_succeeds

I’m not sure why it fails, but I have understood that, in a log, segments are ordered from the oldest to the newest. It means the oldest will have a base offset of 0 (for example), the second will have a base offset of 2.

But then, there is this code in the Read method :

    var s *segment
	for _, segment := range l.segments {
               //The first segment we read is the oldest one. That is to say, the one
               //with a base offset equals to 0, right?
 		if segment.baseOffset <= off {
			s = segment
			break
		}
	}
	if s == nil || s.nextOffset <= off {
		return nil, fmt.Errorf("offset out of range: %d", off)
	}

If we try to read offset 3, by reading the first (oldest) segment in the loop, we will have: segment.baseOffset = 0 <= 3 However, the Record with the offset 3 is in the segment which baseOffset is 2.

Did I miss something?

Most Liked

travisjeffery

travisjeffery

Author of Distributed Services with Go

Hey, yeah it should be implemented like this:

func (l *Log) Read(off uint64) (*api.Record, error) {
	l.mu.RLock()
	defer l.mu.RUnlock()
	var s *segment
	for _, segment := range l.segments {
		if segment.baseOffset <= off && off < segment.nextOffset {
			s = segment
			break
		}
	}
	// START: before
	if s == nil || s.nextOffset <= off {
		return nil, fmt.Errorf("offset out of range: %d", off)
	}
	// END: before
	return s.Read(off)
}

I fixed a ton of issues that will be in the next beta, which should be out either late this week or next week.

Where Next?

Popular Pragmatic Bookshelf topics Top

jimmykiang
This test is broken right out of the box… — FAIL: TestAgent (7.82s) agent_test.go:77: Error Trace: agent_test.go:77 agent_test.go:...
New
jesse050717
Title: Web Development with Clojure, Third Edition, pg 116 Hi - I just started chapter 5 and I am stuck on page 116 while trying to star...
New
herminiotorres
Hi @Margaret , On page VII the book tells us the example and snippets will be all using Elixir version 1.11 But on page 3 almost the en...
New
New
New
tkhobbes
After some hassle, I was able to finally run bin/setup, now I have started the rails server but I get this error message right when I vis...
New
New
bjnord
Hello @herbert ! Trying to get the very first “Hello, Bracket Terminal!" example to run (p. 53). I develop on an Amazon EC2 instance runn...
New
dachristenson
@mfazio23 Android Studio will not accept anything I do when trying to use the Transformations class, as described on pp. 140-141. Googl...
New
New

Other popular topics Top

PragmaticBookshelf
Machine learning can be intimidating, with its reliance on math and algorithms that most programmers don't encounter in their regular wor...
New
AstonJ
Or looking forward to? :nerd_face:
498 14002 274
New
AstonJ
I ended up cancelling my Moonlander order as I think it’s just going to be a bit too bulky for me. I think the Planck and the Preonic (o...
New
AstonJ
This looks like a stunning keycap set :orange_heart: A LEGENDARY KEYBOARD LIVES ON When you bought an Apple Macintosh computer in the e...
New
New
AstonJ
Was just curious to see if any were around, found this one: I got 51/100: Not sure if it was meant to buy I am sure at times the b...
New
New
CommunityNews
A Brief Review of the Minisforum V3 AMD Tablet. Update: I have created an awesome-minisforum-v3 GitHub repository to list information fo...
New
AstonJ
If you’re getting errors like this: psql: error: connection to server on socket “/tmp/.s.PGSQL.5432” failed: No such file or directory ...
New
AnfaengerAlex
Hello, I’m a beginner in Android development and I’m facing an issue with my project setup. In my build.gradle.kts file, I have the foll...
New

Sub Categories: