Knowing Where to Look by using "Not Invented Here" (and a bit of NoSQL propaganda)

Ted Dziuba wrote an interesting article yesterday on things he wished he had learned earlier about programming in a start-up. He's spot on since start-up programming requires the full breadth of the computing field. He gives the best advice you can give, but it is often the hardest to actually execute on.

"You just need to know where to look."

In today's world with Google, this is becoming its own kind of skill. Not only that, you need to know the words related to your problem domain. Unfortunately, until you understand your problem and the language around it, Google isn't going to be of much help. Especially with all these damn kittens everywhere that are ... JUST SO CUTE!!!

Fortunately, our field suffers from a massive influx of "Not Invented Here" which means that for any known idea, there are probably ten good solutions available and already open source. Unfortunately, you don't know about any of them and you are about to invent the eleventh.

In its own way, this is pure awesomeness.

As your eleventh solution matures, you run into smaller problems that are easily understood and can be googled. When you get answers, your concepts get related to existing problems with existing solutions that may also solve your original problem. In the end, you will either create something new that is unique and solves a new class of problems or you will find an existing solution that solves your problem very well (or enables you to solve it very well).

The hard part is recognizing when your "not invented here" derived solution is inferior to an existing product and ditching your solution; it takes a surprising amount of discipline and humility to look at your creation (or child) and say "this sucks" and move on to make your product better.

For instance, I wrote my own programming language called Kira. For me, it solved many of my problems at the time very well. It was the perfect tool for the set of problems I was working on at that time, and then I discovered new problems for which it didn't work out that well, and beauty yielded to monstrosity. I killed Kira because C# was better.

When I created Kira, I didn't know that C# had closures (because it didn't when I initially learned C#). My ignorance combined with technical ability means that I took "Not Invented Here" to the next step and "Invented Here"; this can be dangerous for businesses, but it is a great learning tool. Looking back, Kira sucked. However, the knowledge I gained is priceless as long as I can keep an open mind and not become blinded by my own inventions.

Unlearning the old ways
While learning about existing products can be nice, it also may not be applicable to the next generation of developers. The computing field evolves and changes based on many factors. Ted took time to bash NoSQL again, and while I can agree with him on many levels. His points are going to be irrelevant because NoSQL is focusing on a deeper problem.

If everyone learned proper database design, then there wouldn't be a need for NoSQL. That is the problem, no one can agree on what is proper database design. It is more of an art than a science. Business is unpredictable, and your designs need to handle that. It takes expert designers to work with existing databases to make them do new things.

Core to the NoSQL movement is storing data in a way that is easily accessible. It must also be understood in a context more similar to how we worked with data in Computer Science 101. Remember those days? All you had was a file, and you could deliver neat little programs. Why do we add all the database complexity? What do we gain, especially with the hardware we have?

On the other hand, there are huge amounts of fail just lurking around NoSQL solutions. I see the shadow of fail lurking, and I wager many of the hot solutions now are going to epic fail in many businesses. I also see some that are going to do very well. For instance: I think CouchDB is great. I know this because I've invented my own version in PHP. Does mine suck? Yes. Did it enable me to find CouchDB? Yes!. Is there enough commercial support being raised for CouchDB to be a viable business solution? Maybe. Is it worth me investing in it since it enables me to solve some of my problems perfectly? Definitely.

On getting old
I'm getting old, and I'm feeling old. It seems that all these young people want to do is make mistakes when "all they have to do is learn `old person technology X` and master it!!!" This is the monologue in my brain when a green developer is talking about the latest and greatest technology, but it isn't the right mode of thought. I have learned to fight the urge to act on these "old people" feelings.

Change is great for the field in so far as the we the old people take a step back from the holy wars and mentor the new programmers in how to be successful professionals and not become zealots in some war against each other.