seblegall

seblegall

Distributed Services with Go - Method log.Read() fails reading Records

Hi,

I’m currently reading your book. By doing so, I like to rewrite the code by myself. It helps me to deeply understand how it works.

However, I found something strange in the log package, concerning the Read() func. Let’s see that with a test example :

In the log_test.go file, the original test is :

func testAppendRead(t *testing.T, log *log.Log) {
	append := &api.Record{
		Value: []byte("hello world"),
	}
	off, err := log.Append(append)
	require.NoError(t, err)
	require.Equal(t, uint64(0), off)

	read, err := log.Read(off)
	require.NoError(t, err)
	require.Equal(t, append, read)

}

In this test, we write 1 record and read it.
Let’s write 4 records and read them:

func testAppendRead(t *testing.T, log *log.Log) {

	for i := 0; i < 5; i++ {
		append := &api.Record{
			Value: []byte(fmt.Sprintf("Hello World %d !", i)),
		}
		off, err := log.Append(append)
		require.NoError(t, err)
		require.Equal(t, uint64(i), off)

		read, err := log.Read(off)
		require.NoError(t, err)
		require.Equal(t, append, read)
	}

}

Here is what I get by running go test -v log_test.go :

=== RUN   TestLog
=== RUN   TestLog/append_and_read_a_record_succeeds
    TestLog/append_and_read_a_record_succeeds: log_test.go:48: 
                Error Trace:    log_test.go:48
                                                        log_test.go:32
                Error:          Received unexpected error:
                                offset out of range: 2
                Test:           TestLog/append_and_read_a_record_succeeds

I’m not sure why it fails, but I have understood that, in a log, segments are ordered from the oldest to the newest. It means the oldest will have a base offset of 0 (for example), the second will have a base offset of 2.

But then, there is this code in the Read method :

    var s *segment
	for _, segment := range l.segments {
               //The first segment we read is the oldest one. That is to say, the one
               //with a base offset equals to 0, right?
 		if segment.baseOffset <= off {
			s = segment
			break
		}
	}
	if s == nil || s.nextOffset <= off {
		return nil, fmt.Errorf("offset out of range: %d", off)
	}

If we try to read offset 3, by reading the first (oldest) segment in the loop, we will have: segment.baseOffset = 0 <= 3 However, the Record with the offset 3 is in the segment which baseOffset is 2.

Did I miss something?

Most Liked

travisjeffery

travisjeffery

Author of Distributed Services with Go

Hey, yeah it should be implemented like this:

func (l *Log) Read(off uint64) (*api.Record, error) {
	l.mu.RLock()
	defer l.mu.RUnlock()
	var s *segment
	for _, segment := range l.segments {
		if segment.baseOffset <= off && off < segment.nextOffset {
			s = segment
			break
		}
	}
	// START: before
	if s == nil || s.nextOffset <= off {
		return nil, fmt.Errorf("offset out of range: %d", off)
	}
	// END: before
	return s.Read(off)
}

I fixed a ton of issues that will be in the next beta, which should be out either late this week or next week.

Where Next?

Popular Pragmatic Bookshelf topics Top

yulkin
your book suggests to use Image.toByteData() to convert image to bytes, however I get the following error: "the getter ‘toByteData’ isn’t...
New
cro
I am working on the “Your Turn” for chapter one and building out the restart button talked about on page 27. It recommends looking into ...
New
leba0495
Hello! Thanks for the great book. I was attempting the Trie (chap 17) exercises and for number 4 the solution provided for the autocorre...
New
brian-m-ops
#book-python-testing-with-pytest-second-edition Hi. Thanks for writing the book. I am just learning so this might just of been an issue ...
New
nicoatridge
Hi, I have just acquired Michael Fazio’s “Kotlin and Android Development” to learn about game programming for Android. I have a game in p...
New
brunogirin
When I run the coverage example to report on missing lines, I get: pytest --cov=cards --report=term-missing ch7 ERROR: usage: pytest [op...
New
oaklandgit
Hi, I completed chapter 6 but am getting the following error when running: thread 'main' panicked at 'Failed to load texture: IoError(O...
New
adamwoolhether
Is there any place where we can discuss the solutions to some of the exercises? I can figure most of them out, but am having trouble with...
New
rainforest
Hi, I’ve got a question about the implementation of PubSub when using a Phoenix.Socket.Transport behaviour rather than channels. Before ...
New
Keton
When running the program in chapter 8, “Implementing Combat”, the printout Health before attack was never printed so I assumed something ...
New

Other popular topics Top

PragmaticBookshelf
Machine learning can be intimidating, with its reliance on math and algorithms that most programmers don't encounter in their regular wor...
New
PragmaticBookshelf
Brace yourself for a fun challenge: build a photorealistic 3D renderer from scratch! In just a couple of weeks, build a ray tracer that r...
New
Exadra37
I am thinking in building or buy a desktop computer for programing, both professionally and on my free time, and my choice of OS is Linux...
New
AstonJ
Thanks to @foxtrottwist’s and @Tomas’s posts in this thread: Poll: Which code editor do you use? I bought Onivim! :nerd_face: https://on...
New
New
Exadra37
Oh just spent so much time on this to discover now that RancherOS is in end of life but Rancher is refusing to mark the Github repo as su...
New
PragmaticBookshelf
Learn different ways of writing concurrent code in Elixir and increase your application's performance, without sacrificing scalability or...
New
AstonJ
Was just curious to see if any were around, found this one: I got 51/100: Not sure if it was meant to buy I am sure at times the b...
New
AnfaengerAlex
Hello, I’m a beginner in Android development and I’m facing an issue with my project setup. In my build.gradle.kts file, I have the foll...
New
PragmaticBookshelf
Fight complexity and reclaim the original spirit of agility by learning to simplify how you develop software. The result: a more humane a...
New

Sub Categories: