Please introduce yourself.

My name is Dave Copeland. I write books under David Bryant Copeland because there’s another Dave Copeland out there that writes books. I’ve been a software engineer for twenty-five years. I’ve worked at a lot of startups and done a lot with Ruby on Rails for the last ten or so years.

I really enjoy working on small teams and with small companies, trying to make them bigger and better and work more effectively. Currently I’m a CTO at a mental health startup, which is still pretty small. I’m also the entire engineering team. Previous to that I was a very early engineer at Stitch Fix for around six years or so. I sort of grew with the company and ended my time there as a sort of chief software architect and technical project lead, more or less overseeing all the technical stuff that was going on at the time.

A long time ago I was inspired to write a book about Ruby command line applications. I submitted it to Pragmatic Programmers, and they wanted to publish it. So I did that, which was great. After that, I wrote another book with Pragmatic Programmers about writing a Rails app using angular JS, bootstrap, and Postgres—how to bring all those four things together to write an app. In addition, I contributed to a couple of versions of the Pragmatic Programmers’ Agile Web Development with Rails. I’ve also self-published a couple of books. One is called the Senior Software Engineer, which is about the noncoding aspects of being a tenured engineer.

The other is the most recent one, called Sustainable Web Development with Ruby on Rails. It’s basically an intermediate to advanced book about how to write Rails, in a way that allows you to sustain development on a growing team. Not just how to write Rails but how to maintain your Rails apps over time.

I knew Pragmatic distributed other authors’ books. I originally self-published my book because I thought it would just be faster for me to do it myself. I didn’t want to be tied down to a process but, of course, it took me forever to do it on my own. Since my book seemed to be something in line with what the Pragmatic Bookshelf published, I asked if they’d be willing to distribute it, and they said yes. That was great!

What is sustainability?

A lot of people talk about scalability. Can Rails scale? Can this software scale? And a lot of people interpret that word scalability differently. It’s sort of hard to have a conversation around it. I decided to use the word sustainability because it’s a little bit clearer what it means.

Can we write software that can be sustained? When we write software, it’s not just like we swoop in, write an app, and then leave. Usually we write the bare bones of an app and then we evolve it over time.

And then a bunch of people join the team that weren’t there when the app was written, and they have to add changes to it or fix bugs. Then you’ve got people who don’t have a lot of experience as programmers working on it or maybe people with lots of experience. So, my thought was, how can we sustain the development of this app, knowing how much change is going to happen over the life of that app over several years?

I think that makes it a little easier to know what I’m talking about when I say sustainability versus scalability.

What are some of the characteristics of sustainability?

A really simple way to put it is this: Will it be just as easy to make changes to the app five years down the road as it was when the app was brand new? Obviously that’s not always possible, but that’s something to strive for.

When you first make an app, there’s nothing there. It’s very easy to make features. It’s very fast. But can you write your code in a certain way so that five years later it’s just as easy to add new features and fix bugs and make changes? I think the easier it is to make adjustments, the more sustainable your development process and practices are.

How does a product’s purpose maintain its identity versus something that might have to evolve based on the needs that change over time? How do you create code that allows it to evolve as the new needs change?

Okay, great question. You want to make it as easy as possible to change the app. That is a hard thing to do.

The way that I would think to do it is to make the app do only what it needs to do at any given time and to make sure that you have a good way to know that the app is working, which is usually with tests. For example, if I need to make a change to the app and the app only does what it needed to do yesterday, today’s change should be relatively simple, since I don’t have to navigate a bunch of unneeded complexity.

There should be tests so I can make my new change, run all the old tests, make sure I didn’t break anything that it did yesterday, and be confident that my change will work.

Now, in contrast, another option—that I do not think works—is to make your app with added flexibility. I build a feature that I need today, but I make that feature more flexible than it needs to be. And then in my mind, it’s like, “Well, in the future, that flexibility will pay off because then I can use that flexibility to add new features to my app that I wasn’t anticipating, and it’ll be easier than having to code it completely.” But, it really never pays off.

It always makes things more complicated because you don’t know how the app is going to change, so you never need that flexibility you added. Given that you don’t know, the simplest thing to do is make the app do what you know it needs to do now. You make sure it has tests, and then whatever change comes that you did or didn’t imagine, you’re in a pretty good place to make that change. And you repeat that forever. Then you’ve got sustainability.

When building sustainable apps in Rails, what are the features that sort access your data or that help you model the data?

In my view, the data is the most important thing—for example, if you have an application with data and the application source code just goes away from all of existence, you still have the data. You can rebuild the app since you have the data and you can access the data to do things. But if the source code stayed, and all the data vanished out of existence, that would be catastrophic. That could end a company if that happened.

S, since the data’s really important, the way to manage that is you use a database, of course, and the database gives you various facilities for ensuring the correctness of the data. If I know that a customer is always going to have an email, I can tell the database, “Do not store a customer record unless there’s an email.”

So if I tell the database that, that will happen, and then I can know everywhere in my code, everywhere in the world, if I access a customer, I know they have an email. That simplifies my life and it ensures that the data in there is correct. And when we want to report on that data, we can rely on those sorts of things.

You can get very sophisticated with what you tell the database is and is not allowed. You can say that if we have a product catalog, the prices must be greater than zero dollars. The database can enforce that if you want. You can say, “The price has to be more than zero dollars but less than a thousand dollars.” You can get as sophisticated as you want. And so there is a bit of a balance there to make sure that you are sort of baking in the right constraints when you’re modeling the data.

To do that, you have to have a conversation with who is using this data. What is this data for? What are we trying to achieve with it? And from there, you need to make a judgment about how firm the requirements are. We may feel really strongly that customers have to have an email, because that’s the way that we’re interacting. We could make the database check.

But we may also say, “Customers should have a first and a last name.” Well, not everybody has a first and a last name, and maybe our customers do now, but maybe in the future they don’t. And so maybe we don’t necessarily want to have the database enforce a first and last name. Maybe I want to let that be a little loose because I’m not confident that’s a true requirement that’s going to stand the test of time.

That subtlety can be hard to manage. You definitely need to try to figure out “What do I want the database to enforce?” And “What do I want to let be looser?” Once you have that in place then, somewhat mechanically, you create the database tables with the Rails migrations. Rails gives you all the features that you need to set up those constraints and just model your database however you decide that you need it.

But it always starts from that conversation. What is this stuff for? And what are we trying to achieve? Once you have that, then in Rails, you have a bunch of model objects that allow you to access the database, and those can have validations that are like constraints, but you don’t necessarily need them unless a user is going to be interacting with that model.

For example, If we’re making it so a user is allowed to create a customer record, well, we might validate that the email is there in the model because we want to tell the user that the email is required. But, if users are not creating these records, if, for example, we’re importing them from a CSV, we don’t need to have a validation because the database will catch that and there’s no user to interact with. There’s some subtlety there as well, but you ultimately cultivate a bunch of models that give you access to the database. And then from there any feature that you want that needs to access the database can use those models as needed.

Because of the modeling and constraints, you can be relatively certain that even if you have bugs in your application, they’re not going to write invalid data into the database. You could have someone who doesn’t know Rails at all just come in and upload a CSV into your database. And that’s fine because the database just won’t allow any rows that contain invalid data. You can continue to be really sure that your data is good.

What is a controller, and what is its role in Rails development?

Controller in Rails is named after this UI pattern called model-view-controller or MVC. And in that pattern, the controller would take input and deliver views based on data that has been modeled. The way that works in Rails is essentially when you request a URL of your Rails app, Rails will figure out what controller that corresponds to, and it will figure out what method of that controller your request corresponds to. And then Rails will call that method of your controller.

For example, if I request the /widgets URL of my app, Rails is going to find a controller named WidgetsController, and it’s going to find a method called index and it’s going to call that method. Now, there are a lot of expectations about how those methods are implemented. They’re not just any random method, they don’t take any parameters, and you don’t really return anything.

Basically what you do is you can examine a bunch of stuff from the request. Like what were the HTTP request parameters? What were the headers? What were the cookies that were given? You can have access to all that HTTP stuff to do whatever you want to do. What Rails intends you to do in the controller is figure out what request is being made, specifically to trigger some sort of downstream logic—for example, updating the email address of a customer, or ordering a a new widget: figure out what that is and trigger whatever the logic is that implements that.

And then based on that logic, what do we need to do in terms of giving the user a response? Is the update to the customer invalid, so we need to tell the user, “Hey, here’s the errors that you created”? Or, did they do it in the right way? Then we tell the user, “Great, we updated it.” The controller is sort of brokering that sort of thing. It’s not intended to have all of the sort of business logic as you might think in it. It’s just sort of the layer between raw HTTP and the rest of your app.

What can you tell me about jobs and specifically about something called Sidekiq?

Sidekiq is a very popular Ruby gem that allows you to manage background jobs.

Background jobs is, well, just a term used to indicate, “Hey, I want to run some code, but I don’t want to make the user wait for this code.” I want to run the code that’s sort of in the background somewhere. And so a user may say, “Hey, I want to check out. I want to pay for this product.” The actual payment processing is very slow, so we might tell the user, “Okay, cool. I got your payment. You’re good.” And then meanwhile, in the background, this payment processing is happening. And so we do that in a job. Rails for the longest time did not provide any particular way to run jobs. And there are lots of different ways to do that with third-party gems, like Sidekiq.

With Sidekiq you say, “Here’s the name of a class and here are some parameters. And I want to run that in the background.” And Sidekiq will then, in the background, create an instance of that class, call the method called perform, and pass those parameters in.

So Sidekiq is an easy way to manage background jobs. And it provides a whole bunch of infrastructure there to do that. In my example of a payment processor, maybe the payment processor is down. In that case, your job will fail, but you might want to retry it later.

Sidekiq gives you the tools to decide how and when to manage those situations. It even gives you a web UI so you can look at the jobs that are running. So if you’re an engineer on call, you can see what’s going on with all the background jobs that are happening in the system.

Rails, I think in version five, provides a thing called ActiveJob, which is a layer on top of a job system like Sidekiq. In theory, you would use ActiveJob and say, “Hey, ActiveJob. Your backend is Sidekiq, so when I ask you to run a job, use Sidekiq for me.” So you’d use the ActiveJob API to start up a background job, but really under the covers, it uses Sidekiq.

I don’t find a lot of benefit in doing that because you have to understand Sidekiq either way. Background jobs are a great way to make your app perform well because anything that’s slow or flaky or that the user doesn’t need to wait on can go into a background job. And then the user experience is as fast as you can make it. The user doesn’t necessarily have to sit there for three or four seconds while you talk to the payment processor; they can see that their payment has been initiated and go on about their business.

Earlier, you mentioned that when you’re coding, you should code for the problems that are in front of you rather than things that might pop up in the future. How do you design APIs, because they often are about creating tools to solve potential problems rather than specific and immediate requirements.

Yeah, that’s a good question because it definitely illustrates a little bit of subtlety with the phrase, “only build for what you have.” You can take that to the extreme. You can make decisions that you will regret. And so APIs…it’s good that you’re bringing that up because you can make an API that other people are going to use, and they’re going to rely on those URLs like /api/v1/widgets. It makes it a lot harder to change.

If you realize, “Oh, we don’t call them widgets anymore. We call them products.” Well, you can’t just change your API to be /api/v1/products, because everyone out there that’s using widgets is going to all of a sudden be broken, and that’s probably not what you want.

And so what that implies is that you need to think it through a little bit before you make decisions, because some of the decisions can’t be easily undone. And that sort of goes against the whole notion of build what you have in front of you. And so you have to figure out ”How much thinking ahead do I want to do?”

It’s similar to what we were talking about with the data modeling so when you’re thinking about your API or any decision, that’s going to be hard to undo later. You need to really understand, “What are we trying to solve here? And how confident are we? And what is this a part of?” Like, if I’m making an API, it’s probably, I’m part of some business. Literally what is the business’s growth expectation and how does the API fit in?

If the business is like, “We’re going to sell widgets or we’re going to go out of business.” Okay. I would be pretty confident we’re calling them widgets. But if, on the other hand, they say, “Well we think widgets might be a new source of revenue, but we’re not totally sure.” That may change my thinking because it doesn’t sound like we’re confident. You have to really balance that and then try to leave yourself flexibility if you think that you’re creating a situation that you can’t undo later.

So when it comes to APIs, the reality is that you make a URL that someone uses… And if you don’t think through a way to get them off of it when you need to change it or to get them to use a new version when you need them to, then you’re going to have to maintain that original version forever.

Maybe that’s fine, but maybe that’s not. If it’s not, then you start thinking “Okay, well how would we get people to switch versions?” Which is weird because we don’t know that we need to switch versions, but we’ve made this decision to not allow people to use an old version forever so you get into this sort of cycle of having to think ahead and it can be really tricky. I think you really need to understand as much context as you can about what we are trying to achieve and how hard it might be to change it later.

And if it’s going to be hard, then you might want to think it through before commiting. The same exact thing with the database, changing the database fields and columns and constraints like that. There’s a point at which that can be very difficult to do. And thus it implies, you need to think it through a little bit better. You can’t just go forward and hope for the best like other stuff in your app. You can usually just change internal logic. You can change it all you want. No, one’s going to know really. But the API is external.

From a provider point of view and then as a consumer view, what’s your philosophy about security, passwords, API’s key secret, so forth. What should developers be thinking of and taking into account in their design?

Yeah, I think with security, authentication, things like that, you want to find a reliable implementation from someone you trust because they’re going to be better at it than the average developer.

It’s incredibly difficult to think through all of the potential attack vectors that someone might have with an authentication system and make sure you’ve handled them. In the book Web Development with Ruby on Rails I talk about names and passwords, and there’s two ways in Rails to do it.

One is this library called devise, which has been around for years and is pretty reliable. And the people that have contributed to it understand security, and it has all the right defaults. The second would be to use something like Google or GitHub to authenticate. Google and Github’s security teams are awesome; authenticating through them should be relatively safe.

For APIs, it’s a little bit different. It can be a little bit simpler. You start with an API key. But, you need to allow consumers to have multiple keys and to rotate those keys and tell you that a key has been compromised, and then you need to turn that key off. There’s still a lot to think through. And unfortunately there is no magic Rails gem that implements API keys in the right way like there is for login.

There are services like Auth0 that you can defer all of your authentication to. Setting stuff up like that requires a pretty big investment in your runtime infrastructure. And that can also be incredibly complicated to get right. For API level stuff, you really need to sort of look at what others are doing and try to replicate that because there just isn’t an out-of-the-box thing. To come back to what you were asking before, this is a thing where you really need to think it through before you do it; you really need to think through what are you trying to achieve and try to set up as many good defaults as possible and avail yourself of any expert.

If you’re at a big company, there’s a security team; ask them what they think. And accept the sad reality that you can’t make it perfect. So you need to figure out how you can know if there’s been a problem with your authentication or authorization so that if something happens that you weren’t able to predict, you at least know that it happened and can remedy it. Probably not doing that topic much justice, but it’s kind of a big one.

Would you speak a little about the trade-offs between monoliths and microservices?

Yeah. Rails is very much designed to be the one app that runs everything at your company. It’s basically the best way to make a system like that. A monolithic system. Rails’ opinions are, “We’ll tell you what to do.” And there are a lot of advantages to that because everything is in one place.

You have one database, one set of code. If you need to reuse code in another part of the app, it’s just right there; you just call it like you would any other code. It makes stuff like that simple. It makes it easy to sort of react to unforeseen requirements or off-the-wall requirements because you’re basically just writing code to access the database and queue some background jobs.

Even though that can be complicated, it’s much less complicated than if you have microservices where, for example, if I need to access shipping labels from a service, well, I’ve got to call into some shipping labels microservice, and that’s a network call. Now I manage that network call. What if the service is down? What do I do? Who’s in charge of that service? How does it work? I’ve got to understand its documentation because I can’t necessarily just go look at the code, and it’s not just a bit of code anymore. It’s this entire other app that has this entire other database.

It just becomes a lot harder to sort of be agile and just bring things together. So why do it at all, right? Well, If you have a team of two hundred developers working in a single monolithic application, that can be rough, and you need to be very diligent and disciplined about how you all work in that same large application. And even if you do a great job, you still end up with situations where two senior developers have a different way of doing things.

And so then you have two different ways of doing things in the same app, which can be confusing. You also have a situation where your public website needs to be up all the time, but maybe your customer service UI doesn’t, and in a monolith they’re all wrapped up in the same thing. You end up having to keep your customer service stuff running all the time just because it’s part of the public website.

If you had microservices, then you can divide and conquer this problem. You can have a smaller team manage a much simpler app. If I’m the team that works on the shipping labels service, well, that’s a much more constrained problem set, and it’s much easier to understand how that works, get that right, make sure it’s running. And you don’t have to think about customer signups because you’re just focused on shipping labels.

It can have some advantages there. To make it work with a team, you need an investment in infrastructure and tooling and monitoring, and usually—depending on the size of your team— a whole team dedicated to the platform on which all of these services run.

I think like having gone through both of these things in my career, I would not rush into microservices too quickly, but I would probably want to know when I needed to start doing it because it can take a while to go from a monolithic architecture to one based on microservices.

This falls into that category of “you need to think it through a little bit” because it’s going to be a long slog to build, but you’re going to wish you did at a certain size.

What got you originally into Ruby?

My very first programming job, I was writing Perl. Well, I was writing C++, and then I did a thing in Perl and I was like, “Wow, this is amazing. It’s like really easy to do stuff.” I don’t have to manage memory or compile anything. It’s just really nice. And so I did Perl for a while, and then I actually heard about Rails when I was doing Java at work. I bought the agile web development Rails book at the time. And it made no sense to me because I just couldn’t understand Ruby from the book. I was completely lost. I said, "Okay, I’ll just learn Ruby. And all the stuff I would do in Perl, I’ll do that in Ruby because it’s kind of similar. And then once I do that, then I’ll come back to this Rails thing.”

I was still writing a lot of Perl to automate stuff on my computer or write little admin tasks and things like that. So I just switched to doing it in Ruby and actually liked it a lot better than Perl. I haven’t done Perl in years. Ruby was very fun and nice and easy and not a lot of ceremony. And, it was what I needed to do to learn Rails, which was something that I wanted to learn.

Looking at where things are now, what are some of the other entry points that didn’t exist? You were learning Ruby on Rails in terms of languages and technologies that intrigued you. What are the cool kids on the block now?

Hmm. I mean, I don’t know. When you get to be my age, you get a little jaded about this sort of thing, but I think there’s a lot of…here’s always interest in functional programming. I can’t tell if it’s just a sort of moderate level that’s just never going to get bigger and never going to get smaller or if it really is the future, but I’ve been hearing about it for a long time, and it doesn’t seem like it’s really taking off.

Although, there are a lot of interesting things that use it. Like Elixir is a much more functional language. People really like that. My guess would be that the appeal there is that it’s not typed and it’s very similar to Ruby. It’s very fun to write. And the way you deploy and manage Elixir apps has some good advantages for performance, though it is very odd compared to most other popular systems.

I don’t know, the JavaScript ecosystem, it’s just constantly changing. I would love to see that settle down, but the reality is that people write more and more JavaScript, and when something comes out, its APIs tend to be, "Here’s the JavaScript version. If you want something else you’re kind of on your own.” That’s definitely a thing that I wouldn’t want to lapse on just because it’s going to be a fact of life and it’s going to be bigger and bigger. I don’t know. It’s funny. I don’t have a lot of enthusiasm for things like I did for Rails. Because I feel like Rails has sort of gotten me mentally to like, “All right, this is the tool I use to solve problems.”

And while I personally like programming, I don’t like to program just to program. I like to program to solve for something and to achieve some result. And so Rails feels like a very easy way to do that. And I haven’t seen anything recently that seems like it’s a sea change or a real step up from that.

How do people stay in touch with you? Where can they follow you and find out what you’re doing?

Probably the easiest way is on Twitter. I’m DaveTron5000 on Twitter. I have a website that I should update more than I do, but I really haven’t updated it in quite a while. I think Twitter is probably the best place.

Thank you. This has been really insightful!

What do you think of the current direction of RoR?
Currently Rails is still heavily on Monolith and Server Side Rendering.
This is the approach we are using for 20 years, while most of the stacks are going to be distributed and are splitting data layer vs UI layer.

Regards,
Adam

It will be interesting to see how it plays out. I think monolithic apps with server-side rendering are exceedingly easy to build, test, and maintain. Having a split front-end/back-end adds so much complexity and teams rarely take the time and care needed to make them work as well as server-rendered. Think about all the apps you use that are built this way - they are always more flaky, slower to render, and harder to use than if they were built conventionally. You have to actually take steps to explicitly make the back button work! and a million more things. You also see newer stacks return to server-side rendering as well.

That said, at a certain size, a team might well need to be organized this way, and I think Rails makes a great back-end API-only solution. My previously company had tons of microservices using Rails and I think Rails is great for that!

Where Rails is not great is as a so-called monorepo where your front-end and back-end exist in the same place with the front-end managed by the Javascript bundler of the day. Rails point-of-view is definitely not to do that so it doesn’t make it easy. Rails’ answer to dynamic front-ends is Turbo and while it might be great tech, it’s certainly not where the rest of the industry is headed, so time will tell how much adoption that gets.

If I could wave a magic wand, I would probably rather see Rails figure out how to use the industry-adopted front-end tools better and not diverge from them. Webpacker was a good try, but it is very painful to use and doesn’t work well enough to be as slick as the rest of Rails. Is there a better way to leverage React (server-rendered, client-active) in some sort of “rails way”? I dont’ know, but that seems more promising to me.