This is part of my comment on HN.
At the right level of abstraction, DRY is the best advice possible. If you are building a data layer or a system, then you will be best off if you keep things DRY. However, once you get into product land. Then, I advise people to just get the job done quickly rather than worry about engineering principles.
The reason is very clear, in product land, engineering principles are third to usability and marketing. Avoiding DRY enables two things.
Polish
If you have five sections of code that are the same now, then there is good chance that they will diverge as the product matures. This is polish, and it is a good thing. While it is true that you have more work to do, it isn't rocket science. Trying to use DRY for polish is going to create even more cumbersome code with a lot of branches for all the special cases, and it tries to create an artificial problem of intelligence. Please avoid.
Hiring Scrubs
If you realize that the polish needed to make a product isn't exactly rocket science, then you can hire many scrubs. I define a scrub as someone new to computing, but capable enough to work within a developer environment to find and polish simple code. I like to hire scrubs as it provides a great first job in programming for many people. I was a scrub once, and it wasn't that bad as a 16 year-old.
Once you enable these two things, you have enabled marketing and the usability folks to iterate.
Update: Don't just hire scrubs
It takes a balancing act to get products out using a combination of awesome engineers, "good enough" engineers, and scrubs. A company ultimately needs all tiers to be able to push and iterate products quickly, and it is the company's job to ensure that the people process can nurture engineers from scrub to awesomeness.
Sunday, November 21, 2010
Wednesday, November 17, 2010
Escaping Mr. 20% and losing the ego
This is a fairly long response to a comment on HN.
I've written many lines of code and thought many ideas in my life, but I have not shipped nearly as many. Why? I love coding for coding's sake more than shipping. Shipping requires product development, non-longbeard design, marketing, sales, and more important: customers.
Except for the game engine, everything I did was the special 20%. All the 20% projects were just a neat little idea implemented as if the core tech behind a masters thesis or a senior thesis. However, they didn't have the polish to become a product, and I get easily distracted by more difficult and more sexy ideas. At best, most of them make me look like Captain Hindsight.
I'm perfect for academics, but I found myself being too impractical for my students. This was one of reasons I dropped out of graduate school as I had become an irrelevant preacher. I did not practice what they were going to do, so how could I teach them? It felt dirty teaching them fairly useless things. I was working with tools (such as OCaml and LISP) that enabled me to program better, but programming better doesn't mean shipping better. Most programmers just need to learn how to solve basic problems and have the discipline needed to ship products; telling them that they can avoid memory leaks by using OCaml is like telling them to learn french to avoid pissing off Mexican drug lords. Instead, they just need to get in the discipline of being careful with alloc and free. Any aspect of a technical decision could be looked at as either a technological or a management+discipline issue.
So, looking back on the pile of code I've written. They are all shit because they didn't ship. Shitty code that ships is better than perfect code that doesn't. This is one of the lessons in life that make people like me depressed, but I've gotten over that. I've helped shipped three products and working on two more. One of them is my November start-up sprint (which I'm doing OK on, but I could be doing better).
Problematically, building many 20% things builds an ego. Ego is fascinating. On one hand, you need Ego (or to utilize a customer's ego) to sell a product. On the other hand, you need to lose your ego to build, ship, and support a product. This is why you need to have someone else to do your sales for you. When you get some one to sell you, then you don't need ego anymore. You just have a bunch of work to do.
I've written many lines of code and thought many ideas in my life, but I have not shipped nearly as many. Why? I love coding for coding's sake more than shipping. Shipping requires product development, non-longbeard design, marketing, sales, and more important: customers.
Except for the game engine, everything I did was the special 20%. All the 20% projects were just a neat little idea implemented as if the core tech behind a masters thesis or a senior thesis. However, they didn't have the polish to become a product, and I get easily distracted by more difficult and more sexy ideas. At best, most of them make me look like Captain Hindsight.
I'm perfect for academics, but I found myself being too impractical for my students. This was one of reasons I dropped out of graduate school as I had become an irrelevant preacher. I did not practice what they were going to do, so how could I teach them? It felt dirty teaching them fairly useless things. I was working with tools (such as OCaml and LISP) that enabled me to program better, but programming better doesn't mean shipping better. Most programmers just need to learn how to solve basic problems and have the discipline needed to ship products; telling them that they can avoid memory leaks by using OCaml is like telling them to learn french to avoid pissing off Mexican drug lords. Instead, they just need to get in the discipline of being careful with alloc and free. Any aspect of a technical decision could be looked at as either a technological or a management+discipline issue.
So, looking back on the pile of code I've written. They are all shit because they didn't ship. Shitty code that ships is better than perfect code that doesn't. This is one of the lessons in life that make people like me depressed, but I've gotten over that. I've helped shipped three products and working on two more. One of them is my November start-up sprint (which I'm doing OK on, but I could be doing better).
Problematically, building many 20% things builds an ego. Ego is fascinating. On one hand, you need Ego (or to utilize a customer's ego) to sell a product. On the other hand, you need to lose your ego to build, ship, and support a product. This is why you need to have someone else to do your sales for you. When you get some one to sell you, then you don't need ego anymore. You just have a bunch of work to do.
Monday, November 15, 2010
How to use CouchDB? like this
CouchDB is a very interesting persistence package, and it solves 90% of the problems you find when you build a back-end for a web application. The 90% that CouchDB includes get/put/b-tree indexing/reliability; all this is good standard-stuff in the database world. I want to talk about the other 10% since crud is boring.
The last 10% is usually something like search; it is the novel algorithm that takes all your data and provides it in a meaningful way that makes your product awesome. CouchDB rarely solves this (neither do other packages). The more special the algorithm is, the more painful it will be to try to solve with CouchDB's MapReduce framework alone.
Fortunately, CouchDB has replication built in. I use the replication to push data from CouchDB to a custom server where I aggregate it into a meaningful service. The library is called Otto (short for ottoman).
The biggest problem you are going to have is what happens when your custom server crashes?
This can be solved by
From a complexity standpoint, you can make your life easier by enabling your custom software to merge in bulk sets. This enables you to lazily run your algorithm as you collect a lot of data at once. I have found that this style of bulk insertions makes the third option feasible in many domains.
The nice thing about using CouchDB is that you don't need a schema. You don't need to "plan". If you need store data, then you just store it. Just give it a namespace and insert.
Please comment if you think I should write a book on CouchDB? I've done relational databases for years, and I've lurked in the CouchDB for a while. I'm currently building a web framework around node.js and CouchDB called WIN.
The last 10% is usually something like search; it is the novel algorithm that takes all your data and provides it in a meaningful way that makes your product awesome. CouchDB rarely solves this (neither do other packages). The more special the algorithm is, the more painful it will be to try to solve with CouchDB's MapReduce framework alone.
Fortunately, CouchDB has replication built in. I use the replication to push data from CouchDB to a custom server where I aggregate it into a meaningful service. The library is called Otto (short for ottoman).
The biggest problem you are going to have is what happens when your custom server crashes?
This can be solved by
- providing your own persistence, and deal with reliability
- not worrying about it and launch 3 servers with a custom HTTP server that replicates three ways; spend more money.
- don't care and re-replicate the entire data set, and have potentially non-trivial down-time.
From a complexity standpoint, you can make your life easier by enabling your custom software to merge in bulk sets. This enables you to lazily run your algorithm as you collect a lot of data at once. I have found that this style of bulk insertions makes the third option feasible in many domains.
The nice thing about using CouchDB is that you don't need a schema. You don't need to "plan". If you need store data, then you just store it. Just give it a namespace and insert.
Please comment if you think I should write a book on CouchDB? I've done relational databases for years, and I've lurked in the CouchDB for a while. I'm currently building a web framework around node.js and CouchDB called WIN.
Cloud Coding
I just want to say that Twilio is an awesome company. About a year ago, I was seriously considering dealing with asterisk. Asterisk is perfectly good software, but it didn't make sense for the problem I had. I took a risk and quickly developed the VOIP feature for a product.
Developing and supporting a product with Twilio over the past year has made me think about the need for developing in the cloud. I'm at the base level of cloud development: terminal. I can log into a VM and get things done. I don't need scp, sshfs, or anything else to support my development capability. My brain, fingers, and ssh are all that I need.
Why do I need a VM to develop with twilio? Well, the monitor and way to debug a twilio application is to use the phone. The phone isn't easy to work with, so I need a public IP. I also need a way to watch the HTTP traffic, so I need the ability to grep logs.
Basically, fundamental Unix skills let you work with new technologies faster.
Developing and supporting a product with Twilio over the past year has made me think about the need for developing in the cloud. I'm at the base level of cloud development: terminal. I can log into a VM and get things done. I don't need scp, sshfs, or anything else to support my development capability. My brain, fingers, and ssh are all that I need.
Why do I need a VM to develop with twilio? Well, the monitor and way to debug a twilio application is to use the phone. The phone isn't easy to work with, so I need a public IP. I also need a way to watch the HTTP traffic, so I need the ability to grep logs.
Basically, fundamental Unix skills let you work with new technologies faster.
Saturday, November 13, 2010
A programming language is not just a tool
A programming language is tool to let you transform representations and solve problems.
A programming language is a contract that enables groups of people to solve problems together.
A programming language is a contract that enables frameworks to be built
A programming language is a way to expression thoughts
A programming language is a contract that enables groups of people to solve problems together.
A programming language is a contract that enables frameworks to be built
A programming language is a way to expression thoughts
Wednesday, November 10, 2010
Now, having said that
Having given praise to my polyglot nature, I must condemn it.
One of the reasons I study programming languages is so that I can write my own. When I find a neat feature in a programming language, I think about how I would solve the computational problem to translate it into C.
Having aged, I'ved decided against writing yet another language. The world needs less languages than more. We ultimately only need 3.
We need one language to experiment and think about complexity.
We need one language to ship products.
We need one language to train beginners.
Will we ever agree which language fits these roles best? No. Now, I do my best not to amplify the problem.
Programming Language research will be over when all three questions can be answered with one language. In my opinion, the closest languages now to do it are either C# or JavaScript.
I'm betting on JavaScript.
One of the reasons I study programming languages is so that I can write my own. When I find a neat feature in a programming language, I think about how I would solve the computational problem to translate it into C.
Having aged, I'ved decided against writing yet another language. The world needs less languages than more. We ultimately only need 3.
We need one language to experiment and think about complexity.
We need one language to ship products.
We need one language to train beginners.
Will we ever agree which language fits these roles best? No. Now, I do my best not to amplify the problem.
Programming Language research will be over when all three questions can be answered with one language. In my opinion, the closest languages now to do it are either C# or JavaScript.
I'm betting on JavaScript.
The 3 Programming Languages you need to Know
Every good programmer needs to know at least 3 languages. Of course, I'm probably wrong.
I can quickly understand a programmer using the biases and stereotypes that I've built up over the years by knowing their favorite programming languages. When I read a resume, I try to classify the "why the programmer used the programming language" with these arch types and how I stereotype and use my biases to find what I want from a stack of resumes.
Happiness Language
This language is what you think in. This is the language that you wish you could use all the time. This is the language that you write your projects in. For me, this is OCaml (and now JavaScript although I'm integrating CoffeeScript into my universe). For many, this is LISP or Haskell. When I find out someone's happiness language, it tells me a lot about them.
If the language is esoteric or new, then they are passionate about computing.
If the language is mainstream, then they may be more sensible or practical about computing.
Hack-it-out / GTD Language
This is the language that contains everything including a kitchen sink. It is very mature and has a massive library base. With this language, you enable yourself to build quick services and command line utilities to help you out in a pinch. Anything that has already been done is at your finger tips.
If the programmer lists many languages, then they may be able to utilize all of them by building RESTful services.
If I don't detect a hack-it-out language or too few languages, then I suspect they are either inexperienced or too specialized.
Bread and Butter
This is the language that you can use to keep yourself alive when life hands you lemons. This is the language that you know just in case you need to hustle yourself to provide for yourself and your family.
If they don't have a bread and butter language, then they probably need some education on how to work in a team effectively.
I can quickly understand a programmer using the biases and stereotypes that I've built up over the years by knowing their favorite programming languages. When I read a resume, I try to classify the "why the programmer used the programming language" with these arch types and how I stereotype and use my biases to find what I want from a stack of resumes.
Happiness Language
This language is what you think in. This is the language that you wish you could use all the time. This is the language that you write your projects in. For me, this is OCaml (and now JavaScript although I'm integrating CoffeeScript into my universe). For many, this is LISP or Haskell. When I find out someone's happiness language, it tells me a lot about them.
If the language is esoteric or new, then they are passionate about computing.
If the language is mainstream, then they may be more sensible or practical about computing.
Hack-it-out / GTD Language
This is the language that contains everything including a kitchen sink. It is very mature and has a massive library base. With this language, you enable yourself to build quick services and command line utilities to help you out in a pinch. Anything that has already been done is at your finger tips.
If the programmer lists many languages, then they may be able to utilize all of them by building RESTful services.
If I don't detect a hack-it-out language or too few languages, then I suspect they are either inexperienced or too specialized.
Bread and Butter
This is the language that you can use to keep yourself alive when life hands you lemons. This is the language that you know just in case you need to hustle yourself to provide for yourself and your family.
If they don't have a bread and butter language, then they probably need some education on how to work in a team effectively.
Subscribe to:
Posts (Atom)