The blog ofJonathan Pepin

Move fast and break things?...

2018-09-11

...or move slow and be safe?

There are two main schools of thoughts.

First, "move fast and break things". I'm using an adage that means something else, but at the same time I feel like this represents well this first school of thoughts.

The idea here is that as an early stage startup, you don't have much resources so you need to use those resources efficiently. You need to ship to hear what users have to say and iterate quickly.

At that stage, the one thing you should be concerned about is people not giving a shit for your product. You need to get it out there, build features to attract users, etc.

At that stage, preparing for scalability is a fool's errand. Having scalability issues will be a good problem to have, if we ever have this problem. "We'll rebuild it when we get traction".

The second school of thought is about how tech debt being the one thing to avoid. An evil staying put, hiding until it decides to start coming back, crippling your product and making your engineering team slower and slower by the day.

Each new feature ends up taking weeks to build and bringing a very high risk of bugs because side effects are everywhere.

Here, we believe that you don't get the chance to rebuild things once you scale.

Both sides are very far appart, each on the far end of the spectrum. It is interesting to see, because while each side is very extreme, both make sense to me. I've seen both sides in success and in failure, and in all cases, all arguments are still valid. Let me explain a bit;

I've seen engineering first startups. Those startups have engineering talent and understand the importance of it. They invest in it and take the time to build robust systems, taking into account what they guess will be their future needs.

When the startup starts getting traction, it's a magical thing to observe. The code is clean, well documented, well structured. As they add people on the team, people onboard quickly and the team keeps a great velocity as the team grows. It's a rare thing, and it's amazing. I think Stripe might be an example like this.
At the same time, when a company uses this approach and fails, it's also very easy to blame the strategy. The product should have been shipped faster, we should have talked more to our customers, we built features we didn't need and built infrastructure we didn't need.

Same stories on the other side. A product first startup, hacking everything to ship as soon as possible. Founders who are pushing for everything to be built asap, for yesterday.

The reason? Product strategy, being first to a competitive market. I mean, this makes sense right? A great software is useless if nobody uses it. The software is just a means to an end - the product. If the product works, the user doesn't care if the code is cleaned and well tested under the hood.
The biggest successes are like that.

Twitter was built on Rails and at some point they had to rebuild everything Java or Scala. Facebook was built on PHP, and still is, but afaik they had to build a lot of tooling and rebuild their own flavor of php. Uber was in a Python monolith and we had to rebuilt everything in micro services in Go and Java.
I don't know about Twitter and Facebook, but having been at Uber, I know part of our success is the hustle, and we made it despite loosing millions due to technical debt (outages, rewrites, lost productivity, etc).

See, in both situation, the arguments make sense.

Build strong engineering to keep up with growth and keep efficiencies while growing, or focus on shipping product.

Whatever you say, you can't have both. Resources, especially engineering resources are extremely scarce. C-suite executives believe access to developer talent is a bigger threat to the success of their business than running out of money.

What is bad code anyway?

I've been told "I'm concerned you'll write bad code". Actually, all clients have told me that.
I totally get it. I'd be concerned too in their position.

This is my main concern when I work with freelancers. Which is why I usually only hire people I've worked with and know they are the best.

But this is irrationnal. The bad or malicious engineer that writes bad code is rare. 99% of the time, code is not perfect because... it's the real world, and the real world is not perfect.
If you are running a business, you are not doing research. You can't spend days or weeks on finding a perfect solution. And what is a perfect solution? It doesn't even exist. You'll find a perfect solution for today's problem, but you don't know how tomorrow's requirements will evolve.
I can't tell you what the perfect solution is, but I can tell you an infinite amount of worst solution. You have to move forward and find an ok solution. So by nature, anyone coming after you, with less context, will think the solution is not great.

We've all been through that; You start a new job and think "Oh my gosh, this is the worst code I've seen. I'll rewrite everything, it's gonna be awesome!".

This is normal reaction to legacy code, but take a step back. Maybe two. Look around. Do you think all this code was written by the worst engineers in the world? Do you think people around you might be that bad? Probably not. So think a bit more. Take another step back.

Why do you think the code is this way? Yes, this is a hard question, and the answer might be that this code is actually not the worst you've seen. It actually might not be that bad, if you take into account all the whys.

What is bad code anyway? Bad compared to what? Is it bad when it uses tabs instead of spaces? Is it bad because it's C++? Is it bad if it works and allows the company to be successful?

Yup, another bunch of hard questions.

The answer? I don't have it. But here's what I have to say to my clients when they tell me they are concerned I'll write bad code.

Do I write bad code?

I want a long term relationship

I want to be working with you in years, and that means that I'll have to deal with the code I write. So I'll make sure it's as easy to work with it as possible.

I want to help you build your team

Since I'm building the software, I'm a great position to train your future hires on the team. I'm also pretty good at it, since I've interviewed over 300 engineers at Uber, hired dozens and built multiple teams, of multiple scales.

I'll be looking at them in the eyes when they'll join and think "Oh boy, this is the worst code I've seen".

I'm not too bad as an engineer

I've worked at small YC startups, I've had a great 4 years at Uber, working on multiple systems at multiple scales, from iterating fast on a 20 people growth team to tech leading the rebuild of the drivers payment system. At scale. Doing thousands of idempotent, concurrent RPS, through multiple regions (all of them) and servers.

I'll find a great balance between speed and quality, one that fits well with your situation.

I enjoy this

Software engineers are often compared to artists or artisans. I feel like this fits with how I see myself and the people I work with. I really enjoy writing software, refactoring and having a clean code that I'm proud of.

This would be my answer to anyone asking what is good code, or if they should hack it and move fast, or taking the time to make it solid. Both ways are pretty mutually exclusive. But you can find a balance that allows you to move fast and consciously encure tech debt, and being slower and taking the time to build something robust when it matters.