Argument isn't the right word. No, we weren't arguing. But we sure were having a spirited discussion.
We had a super talented team to do a quick project. The chief programmer/engineer and I were having - well, we weren't arguing, no, we weren't doing that. But we were spiritedly discussing. He had realized that the general technology stack we were going to use had some limitations, and he wanted to go back do it in Ruby on Rails from scratch.
"Timetable?" I ask.
"Three weeks," he says.
"Unacceptable. Let's go forward with what we've got. I'm aware of the flaws."
"I don't think you understand Sebastian."
"No, I do understand. We're pushing problems down the road. They're big problems. I get it. It's no good. But we need to move fast here. We have lots of team members, lots of moving pieces."
He goes on to explain that a few weeks now means we'll have much less work later in maintenance.
I got it. He's an extremely smart guy, one of the top technical people I've ever met. But I have this strong intuition, this vague not-quite-explainable thing... how do I explain it?
I try: "Okay man, you know when casinos get their license to open up in Vegas? Every day they're not open that they could be, they lose something like a million dollars. So when they build, they don't care about causing problems down the road, because a five day delay means they're out five million. Just do it as fast as you can, and use all the resources you've got later to fix it if the project is successful. And if it's not, you don't have a problem anyways."
That halfway explains it.
He reiterates though, that we're building up some huge technical debt by doing it this way.
"I know. Okay. Let's say we need human intervention on all the accounts for a number of actions, every month. Let's say one hour per customer per month of customer support guaranteed. Totally destroys scalability. Got that mental picture? Okay, that would acceptable and preferable to getting the coding done three weeks behind schedule."
"That's insane. That's just... that's insane. Don't you want to make any profit?"
"Well, this is relatively low level customer support type stuff. Maybe it whacks 25% of their initial margins to have someone do it all. That's really bad when we could build out something elegantly automated. But we're going onto an entire new stack, there's some percent chance - maybe low, but not zero - that we run into a roadblock over there too. And so three weeks have passed, and now we've got another problem..."
I'm still trying to understand this time thing. I'm not explaining it well. I don't even really understand it well yet, except for getting the impression that it's really really important.
I try explaining like this - "How many projects have you done? I've done... I don't know how many, but enough that were six months late, eight months late, a year late, or even canceled. That happens a lot, it's normal actually. See, projects don't die because they're launched half-complete or with a problem down the road. You're saying, our shit totally falls apart at 10,000 users. Whatever man, if we've got 10,000 users we've got a hell of a lot of money to rebuild from scratch, even with all the damage that entails. Projects don't really die from launching too early as long as the customer-facing stuff all looks good and does its thing, projects do die when they slip forwards. Momentum is lost, people get demoralized... we've got two people waiting on this next step as a dependency, they can't move until it's done. They're going to wait two weeks? And then, with no guarantee of being ready at that point? No, we need to move fast. Yeah, we're pushing the problems down the road, but we can solve them down the road. Problems down the road - that only come up if we're successful, mind you - they won't kill us. Going too slow will kill us. It kills projects."
I'm still not satisfied with that explanation. But there's something there. Perhaps it's a culture of things going "just a bit" into the future that's so dangerous? I still can't quite explain it right, but I'm starting to see things happening in faster feedback loops as crucial for success.
If it 's a prototype Minimum viable product, yes you are correct in the theory you can fix things later on If you reach market traction.
If it's something that could down the entire business if everything fails in the future... Well.. it depends.
Focus on the priority goals of the system you're building, and if those goals are met, yo're safe.
If things will go your way and you have bugs down the road, I suggest not focusing on the technology stack but thinking out of the box, and trying to find ways around it. Look at facebook. They're still using that database but they haven't really shut down. What happened? They did several work-arounds like Memcache and inventing new technology to cope with the demand.
Here's Rands' take on the/a similar issue:
I love this website. That guy is clearly very smart.
I don't know, I certainly don't have the whole picture here but technical debts the sooner are fixed the better. You think it's easy to translate an app to another language/framework but the reality is you'll always have something more important to do, and at that point you have to hope your programmers will stay with you despite the big technical debt (they warned you after all). And it's actually not that simple to do, and will cause more problems than benefits.
Also three weeks to me don't sound like a lot of time, but again I don't have the complete picture here so I can't judge properly. But if you want my 2 cents, listen to the people who are building your product, if they're saying this to you now they're probably being conservative, and the reality may be much worse.
I agree, Sebastian, and I add that as that team becomes more successful at launching projects, building them right will be more viable and more important because the problems down the road are real, even if they aren't primary.
To use a baseball analogy, hitting is more important than running to base until the batter has it down, because it comes first. Then running well becomes more important as the number of tagouts becomes the new problem. Then stealing becomes important once getting to base becomes easy...it goes on.
I took your post as kind of a thinking exercise, I wrote below what I thought about it. Please don't take any of it seriously. I don't have enough real-world project experience.
If I were in that situation I'd like to say that I would start by asking questions, but it probably wouldn't occur to me because I'd probably be too emotionally involved.
How fast do you expect to grow? Or better yet, what probability do you estimate for growth at different rates?
If you expect to grow really fast then that 1 hour of support /per customer /per month can quickly become a really big liability. Not just in possible upset customers, but in the necessity of bringing in more staff. That means more managers, more communication, and more physical resources. That also means that the quality of staff you bring on board will likely have less qualifications because of the time pressures involved in hiring.
With more moderate growth then your CTO can probably pay off the technical debt after launch. Take care of your CTO during that timeframe, that situation can quickly become highly stressful. Make sure that your CTO gets a really long, good vacation after things have settled down. Burnout is unproductive for everyone involved.
With slower growth, you can easily handle the customer support time. Then your CTO will probably have plenty of time to handle a redesign and elimination of technical debt.
These all assume that you customer support time estimate are correct and that your CTO will still want to do a redesign later on.
How much ongoing programming maintenance do you estimate after the launch?
How much of that maintenance could a new hire/contractor perform?
Taking some of the maintenance work off of the CTO's plate will give them plenty of time to focus on the redesign/reimplementation. This approach has a couple of well known risks associated with it. Malicious individuals, getting the new person up to speed on the code base, etc.
How critical do you consider those three weeks? Do you have a lot of resources waiting on this? Potential competitors that may launch?
If its critical you'll want to launch if you won't have a broken project.
Since it sounds like you have people waiting for this finish (at least 2) then launching it becomes more critical.
Which parts of the project do the next stage people need? All of it? A subset? Can you launch a working subset of the project so that they can get started, and in the mean time let you CTO rebuild the system?
If you can handle the disruption, see if you can cut anything out that will let you do the redesign and lose only a week or two. That way you get a more robust and hopefully better designed system, you don't lose 3 weeks, and you can go ahead with a more limited, but more scalable launch only slightly behind schedule.
Of course both of those approaches really depend on the malleability of the project in question.
What probability do have that the customer support issue will take an hour/month? What happens if it takes a lot more? Or a lot less?
If it takes a lot less then it's no longer a critical problem. You can fix it after the launch.
If it takes a lot more then the redesign may become a lot more useful.
How much future development do you plan? Will this 3 week cost turn into a large savings later on from a more flexible platform?
Final Thoughts: Given that I have no information other than what you've told us. I think that if you have decent risk estimates on everything, you'll want to push forward and launch. The other advantage you get from launching, you find some of the other sticky points in your design.
As a developer I've been in a similar situations that went both ways. One where it worked out, and another where launching ended disastrously. I consider both really good learning experiences, the second one just hurt a lot more.
Good Luck, take the above with a grain of salt, I just found it a fun thought experiment.
who cares, as long as the customer doesn't notice and isn't negatively affected... that should be the primary consideration; will the customer experience suffer in any way and/or will it negatively impact our prospects for growth and expansion down the road...
don't let a technicians desire for perfection and flawlessness get in the way of winning!
This'll probably be pretty disorganized, because I'm more or less just dumping my thoughts to see what comes out.
It seems like you've got a few up front costs by waiting three weeks to release: #1. You lose three weeks of profit. #2. You use three extra weeks of the programmer's time. #3. You keep people waiting on a fairly long timeline without prior warning -- they didn't see this coming, and now they might worry that there will be future holdups that they won't see coming. They might start to lose faith that the project's actually going to get released, and so be less willing to dedicate resources to it.
But if you've already got it out there, being tested by customers, and making money for you, then you don't have to worry about #3 -- you've got momentum, you've got some more capital to confront problems which may come up, and you'll have a much better idea of what you need to do to improve it, so your programmer doesn't potentially end up wasting 3 weeks doing something that didn't have as much benefit as you predicted.
Sounds to me like I just rephrased what you wrote, but I thought I'd take a jab at it.
Relate it to one of those carnival shooting games. You have X amount of ammo. But if you don't hit this target, you don't get to go on to the next round. You've missed twice already. But the target is moving steadily to the right and is almost to where you can't shoot at it anymore. What do you do? If you don't shoot, you don't waste ammo. But you don't make it to the next round for a bigger prize.
That's how new ideas are formed: there is already something there, an intuition, but it needs articulation. I'm sure you're about to find out what's really behind the intuition so it's also clear how to think about momentum in these cases.
For some reason or another I think your blog posts are really starting to improve. Posts like "Oppositional Identities" are really insightful, like the inevitable future blog post exploring this intuition. Maybe you have been doing this all along, but for this individual reader they seem to get more profound.
By the way, about faster feedback loops: are you familiar with the military strategist John Boyd? His essay "Destruction and Creation" deals on a very high conceptual level with the importance of fast feedback loops.
The followup to: "If you want to get rich, stop being a fucking joker."
Phone is ringing.
I'm not sure what to say if he answers.
There's a race going on, but it's not an ordinary marathon. First of all, it's not a mere twenty six point two miles long. We don't know how long it is because we can't see that far; no one has been to the end of the course yet. And, really, it's more of a relay race than a marathon. The entire history of mankind has been running it, passing the baton to future generations. And now the baton has been passed to us, and we're on the course.
Even though we're all entered into the race-- every single one of us-- not all of us are running. Some people are sitting on the side of the road. As you run by, you can see them, staring at the sky with a glazed over look, completely oblivious of the honor that has been passed down to them.
Other people are on the road, but they're walking. They're zigzagging all over the road, but going in the right direction. They'll never make it to the end of the road. Neither will anyone else, but the irony is that the walkers are the only group who DOES think that they'll make it to the end. They have no idea how long the road actually is, so they figure they've got all the time in the world. Walk a bit this way, walk a bit that way, sit down with the sitters for a few minutes, start walking again.
Along the road, also, are joggers. They're mostly running in packs. Whenever one of them starts to get ahead of the group, the rest of them yell at him and he slows down and rejoins the pack. The packs of joggers are like soldiers, plodding away as a big unit. They won't make it to the end either. They know it, but they've accepted it. Just keep jogging, stick with your pack, make it as far as you can.