seblegall

seblegall

Distributed Services with Go - Method log.Read() fails reading Records

Hi,

I’m currently reading your book. By doing so, I like to rewrite the code by myself. It helps me to deeply understand how it works.

However, I found something strange in the log package, concerning the Read() func. Let’s see that with a test example :

In the log_test.go file, the original test is :

func testAppendRead(t *testing.T, log *log.Log) {
	append := &api.Record{
		Value: []byte("hello world"),
	}
	off, err := log.Append(append)
	require.NoError(t, err)
	require.Equal(t, uint64(0), off)

	read, err := log.Read(off)
	require.NoError(t, err)
	require.Equal(t, append, read)

}

In this test, we write 1 record and read it.
Let’s write 4 records and read them:

func testAppendRead(t *testing.T, log *log.Log) {

	for i := 0; i < 5; i++ {
		append := &api.Record{
			Value: []byte(fmt.Sprintf("Hello World %d !", i)),
		}
		off, err := log.Append(append)
		require.NoError(t, err)
		require.Equal(t, uint64(i), off)

		read, err := log.Read(off)
		require.NoError(t, err)
		require.Equal(t, append, read)
	}

}

Here is what I get by running go test -v log_test.go :

=== RUN   TestLog
=== RUN   TestLog/append_and_read_a_record_succeeds
    TestLog/append_and_read_a_record_succeeds: log_test.go:48: 
                Error Trace:    log_test.go:48
                                                        log_test.go:32
                Error:          Received unexpected error:
                                offset out of range: 2
                Test:           TestLog/append_and_read_a_record_succeeds

I’m not sure why it fails, but I have understood that, in a log, segments are ordered from the oldest to the newest. It means the oldest will have a base offset of 0 (for example), the second will have a base offset of 2.

But then, there is this code in the Read method :

    var s *segment
	for _, segment := range l.segments {
               //The first segment we read is the oldest one. That is to say, the one
               //with a base offset equals to 0, right?
 		if segment.baseOffset <= off {
			s = segment
			break
		}
	}
	if s == nil || s.nextOffset <= off {
		return nil, fmt.Errorf("offset out of range: %d", off)
	}

If we try to read offset 3, by reading the first (oldest) segment in the loop, we will have: segment.baseOffset = 0 <= 3 However, the Record with the offset 3 is in the segment which baseOffset is 2.

Did I miss something?

Most Liked

travisjeffery

travisjeffery

Author of Distributed Services with Go

Hey, yeah it should be implemented like this:

func (l *Log) Read(off uint64) (*api.Record, error) {
	l.mu.RLock()
	defer l.mu.RUnlock()
	var s *segment
	for _, segment := range l.segments {
		if segment.baseOffset <= off && off < segment.nextOffset {
			s = segment
			break
		}
	}
	// START: before
	if s == nil || s.nextOffset <= off {
		return nil, fmt.Errorf("offset out of range: %d", off)
	}
	// END: before
	return s.Read(off)
}

I fixed a ton of issues that will be in the next beta, which should be out either late this week or next week.

Where Next?

Popular Pragmatic Bookshelf topics Top

johnp
Running the examples in chapter 5 c under pytest 5.4.1 causes an AttributeError: ‘module’ object has no attribute ‘config’. In particula...
New
jesse050717
Title: Web Development with Clojure, Third Edition, pg 116 Hi - I just started chapter 5 and I am stuck on page 116 while trying to star...
New
gilesdotcodes
In case this helps anyone, I’ve had issues setting up the rails source code. Here were the solutions: In Gemfile, change gem 'rails' t...
New
AndyDavis3416
@noelrappin Running the webpack dev server, I receive the following warning: ERROR in tsconfig.json TS18003: No inputs were found in c...
New
hgkjshegfskef
The test is as follows: Scenario: Intersecting a scaled sphere with a ray Given r ← ray(point(0, 0, -5), vector(0, 0, 1)) And s ← sphere...
New
brunogirin
When trying to run tox in parallel as explained on page 151, I got the following error: tox: error: argument -p/–parallel: expected one...
New
kolossal
Hi, I need some help, I’m new to rust and was learning through your book. but I got stuck at the last stage of distribution. Whenever I t...
New
andreheijstek
After running /bin/setup, the first error was: The foreman' command exists in these Ruby versions: That was easy to fix: gem install fore...
New
bjnord
Hello @herbert ! Trying to get the very first “Hello, Bracket Terminal!" example to run (p. 53). I develop on an Amazon EC2 instance runn...
New
gorkaio
root_layout: {PentoWeb.LayoutView, :root}, This results in the following following error: no “root” html template defined for PentoWeb...
New

Other popular topics Top

PragmaticBookshelf
Write Elixir tests that you can be proud of. Dive into Elixir’s test philosophy and gain mastery over the terminology and concepts that u...
New
ohm
Which, if any, games do you play? On what platform? I just bought (and completed) Minecraft Dungeons for my Nintendo Switch. Other than ...
New
Exadra37
Please tell us what is your preferred monitor setup for programming(not gaming) and why you have chosen it. Does your monitor have eye p...
New
Rainer
My first contact with Erlang was about 2 years ago when I used RabbitMQ, which is written in Erlang, for my job. This made me curious and...
New
AstonJ
poll poll Be sure to check out @Dusty’s article posted here: An Introduction to Alternative Keyboard Layouts It’s one of the best write-...
New
AstonJ
I have seen the keycaps I want - they are due for a group-buy this week but won’t be delivered until October next year!!! :rofl: The Ser...
New
PragmaticBookshelf
Create efficient, elegant software tests in pytest, Python's most powerful testing framework. Brian Okken @brianokken Edited by Kat...
New
hilfordjames
There appears to have been an update that has changed the terminology for what has previously been known as the Taskbar Overflow - this h...
New
New
Fl4m3Ph03n1x
Background Lately I am in a quest to find a good quality TTS ai generation tool to run locally in order to create audio for some videos I...
New

Sub Categories: