Disclosure

I'm an amazon employee. Nothing I say or do on this blog is an official amazon thing. I don't speak for the company at all.

I am a Red Nova Labs shareholder.

I'm awesome.

The Developer/Service Singularity, Saturation, and Good Ideas

It takes very little capital these days to turn an idea into a functional prototype that scales to a decent size. This seems crazy... crazy AWESOME.

I have lots of ideas, and the sirens always call me. But, like Ulysses, I stay the maiden voyage of my plan. I also like to tell people my ideas. If you want some of them, then please send me an email at ideas@mathgladiator.com

Now, onwards to the meat of this article.

We have reached an interesting point in technology. It used to be, back in the old days, that if you had an idea, then you had to go hustle up some capital, invest in some engineers, and hope that it got built. Then you had to market it and sell it.

Now, if you are a developer and find a problem in your life, then you can solve it by combining pieces. For instance, I was thinking about automatically texting a friend to remind him that he should go the gym. Well, I can just do that with twilio and cron job. Given that I already have a fleet of servers, it's literally an hour away due to Twilio.

If you are a developer, then it takes practically no time to just solve pains; the number of pieces available to developers these days is rather insane.

So, what makes an idea good. Well, if it solves a pain, then it is good. What makes an idea a business? It has to solve a pain that is too painful for the garden variety developer to solve. Those ideas are the ones that you can build a company, and then provide pieces to developers.

I rather like this because this means that more and more pieces are going to be going to the cloud.

As more pieces of the puzzle are available, we will reach a point when all things are possible connecting the real world to the Internet. It's just a matter of time.

What about real problems? There was a post back on hackers news about how facebook was saddening due to the brain drain. In a way, I agree. I think I can say the same thing about wall street and math majors. It's very easy for smart people to go make serious money in wall street. Money is the same siren that pulls computer scientists and engineers to silicon valley and entrepreneurship. However, there are the faithful that stay back in academic to fight the good fight.

I think there will always be the faithful, and I think it is important for people in industry to understand the problems of the faithful. I understood the problems when I was a graduate student, but then I discovered EC2. EC2 enabled me to get more compute quickly than I could muster up in a math or CS department. It enabled me to iterate on serious problems and fail quickly.

So, I look now at the valley and the crazy economics of engineers as a good thing. The infrastructure and technology is being built by Facebook, Google, Amazon, Apple, and Microsoft. This cloud will be a commodity to enable future faithful researchers to get a new level. Researchers will iterate quickly, have more data, process more data, and solve the real problems.

Listen to old people

In the entrepreneurial age, there is a lot of ageism.

I think the reason is two fold: one, young people are ignorant and thus more likely to re-invent a wheel in a different way that may work in the market. Second, they have less discretionary funds and are thus cheaper to fund. That being said, I would bet on old people building a better business to survive any bubble since they've been around longer.

I think worshiping youth is foolish.

When I was a kid, I actually listened to the adults around me. I actually didn't have a rebellious time in my life as I realized that it would most likely have negative consequences on my future. Every old person is basically a data point mapping a personality and life experiences into a final outcome, and you can learn from every single one of them. Listening to old people gives you a wealth of data if you are willing to listen, archive, and keep an open mind. Aside: I love quora because it brings out so much condensed wisdom on a variety of topics from everyone.

Now, I'm becoming an old fuck. I'll never become an old man as that is a state of mind (reminder, find George Carlin quote here, if you read this, then please help me find it). Now, the difficulty of becoming an old fuck is picking out the desired outcome and then back tracking to solve the right personality and right life experiences given the current life experiences.

The funny thing about life is that it is mostly under your control (except certain diseases and natural disasters) if you are willing to take control. It is very easy to let others take control of your life if you let them, and I would recommend not doing so unless you benefit in a fair exchange.

Optimizing your personality

You have a choice, you can work through your shit with a therapist, become an alcoholic like most of america, or just come to realize that you are 99.999% wrong on everything. I'm serious. The sooner you realize that you are just plain stupid and wrong is the moment you gain the power to change your mind and optimize your life. Now, if you want to be right, then you can be, but you have to learn to determine which battles are worth fighting; this is the trick. Use the battles you are willing to fight to determine your personality.

Optimizing your life experiences

There are a ton of opportunities to have interesting experiences in your life. You just have to put yourself out there to have them. You need to know what is possible, and you need to do things crazy. Problematically, people lack creativity these days and think spending time in Disney land is fun. Try going homeless and then running naked in Cardwell hall using the chemical shower to bath... That's exciting. Listening to old people about the stupid shit they have done is way more interesting than pre-packaged vacations.

Making all this work

Listen to old people. Old people can tell you what is possible and what is survivable. For instance, I'm proof that you can live on campus without housing. Let the old people that succeeded or fucked up serve as beacons on your path of what is possible and what to avoid. If you don't listen to old people, then you risk falling into the traps that they fell into.

Everyone on the planet will tell you they know how to live, but they are lying. They only know how their exact circumstances map into their current state with an uncertain future. Listening to old people give you data to predict what may be next, and that is a whole lot better than trying to reinvent the wheel. As a caveat, it may tune you into things that have been done rather than trying something crazy and innovative. This is the fundamental problem of doing something innovate: it looks crazy stupid to old people. The perception of a crazy stupid idea and a crazy brilliant idea are fundamentally the same until it has been proven one way or another.

Now, I'm writing this because I'm getting older, and I'm starting to the understand more about life. I'm priming the internet with content on old age since we are going to see bloggers get old. I was thinking about this when I realized that George Carlin passed away. George Carlin was my favorite comedian, and he was one of my heroes. It's interesting to watch his HBO specials and see him age and realize that's going to happen to me. That's going to happen to everyone blogging. What is going to happen when this internet thing matures? Holy shit, these bloggers are all going to die! The internet is going to become an odd graveyard of content. Holy shit, I'm going to die. Shit! Well, I might as well get started on the content about that so I get #1 on google in 20 years. *sigh*

My Research Problems for 2011 and Beyond

I think about a set of problems, and apparently I'm not the only one. So, I'd like to share the problems I think about here and there.

1. How do I make a programming language not Turing complete yet useful enough and have that programming language satisfy the halting problem?
2. Why do graphical models feel powerful until I need to do something with them?
3. How can I utilize a Bayesian Network in a compiled form (for embedded systems)? Is this even useful? Can compiled Bayesian Networks adapt over time? Can I produce chips that get better over time?
4. How can I make Bayesian network inference super fast?
5. How can I use computer graphics to aid in teaching programming and software design?
6. Can I map out the entire infrastructure with a modelling language and have a compiler manage an infrastructure?
7. Do I even need a schema? What are the managerial aspects of going schema-free?
8. How can I make using a database suck less? Why is relational so sell-able yet so damn silly? Is relational out of date? How do I design with key-value pairs?
9. How do I deal with replication between non-heterogeneous environments? How do I deal with the CAP theorem? How do I replicate from the earth to the moon?
10. Is it feasible to put a mini data center on the moon?
11. How do I balance expressive graphics with high performances?
12. How can I make extracting good data from HTML suck way less? How can I query a tree with a beautiful language?
13. How do I utilize map reduce to produce good relevancy findings (recommendation engine, duplicate content)? How can I improve the quality given free-text?
14. How do I build a Voronoi diagram on non-euclidean spaces?
15. How does building a state machine compiler work for making synchronous looking code asynchronous without threads? Would Narrative Javascript + node.js be full of win?
16. How can I make sure node.js is going to win?
17. How can a game/physics engine be built around non-euclidean geometry?
18. How do I make sure CouchDB doesn't suck for some of my future projects?
19. How do I make jobs?

Some of these are old, and I'm fairly sure that some have open source solutions now. Some of them have answers, and I just need to read the current literature. I've got other questions, but they are more business related.

The most fundamental question I have is how can we improve education?

I think about education more than I should because I care deeply about it. My next project is going to related to education.

Do 1x Programmers keep 10x Programmers down?

A question on quora got me thinking about programmers pay.

Is it fair for a 10x programmer to get 10x pay of a 1x programmer? The short answer is no, and while this does depress the hell out of me, I get it.

Being a 10x programmer is a crock of shit to begin with. I believe it in principle that people can be 10x and even 100x more effective, but it is a signal that the 10x programmer needs to move up either into management or work on more difficult problems. I guarantee that management will slow someone down, and I guarantee that there are problems that can slow anyone down.

Now, for various reasons, a 10x programmer may be stuck with a 1x job. The solution is for management to be results-orientated, or for the employee to figure out how to work at home. This enables the results to speak for you. I'm a huge fan of the four hour work-week, and I wish I wrote about it before Tim Ferriss.

The nice thing about being results-orientated is that it turns the tables and rewards effectiveness. I value my time, so I'll be effective with it however I want.

You can be so effective that you can wear multiple hats, and if you can do this, then you must do a start-up. A start-up is hard work, and it requires everything you've got to pull it off.

Generally thou, the four hour work-week doesn't work in start-up land unless either (a) you are the boss, (b) another part of the business is slow [and you can't change it], or (c) management is ineffective [and you are not a part of it].

I'll be honest, if you are a 10x programmer, then you have to try a start-up especially if you have nothing binding you to an area. Think of it like college, and if you succeed, then you make more than 10x pay. If you fail, then you are hopefully networked with people who know you are worth 10x pay.

How to always be right.

Never take an opinionated stance on anything.

It's just that simple, but the problem is that you are not going to make money with such an approach.

Imagine if the news was just facts, then it wouldn't sell. Celebrity gossip sells like hotcakes, but a balanced emotion-less discussion of pros and cons of economic theory wouldn't sell at all. Push an agenda, then you'll get fanatics and make some money. This is why I'm not a politician/reporter, nor do I have aspiration to be one.

I was going to write a book on computer science called "It depends" and just write about all the stupid shit we do in the industry. Problematically, if we all suddenly understood and cared about the pros and cons, then we would work together and cut the work force by 90% and make 10% of our salaries. That would suck. Fortunately, the world is insane, and the pros and cons turn into a competition.

That being said, being right is surprisingly easy. The key is knowing as many pros and cons as possible beforehand and simply judging based on relevancy to the people it affects. I hinted at the philosophy that I've constructed for myself in an answer on quora.

If you apply this methodically and try to put emotions on the back burner, then you will only find yourself wrong in two situations.

When new knowledge is presented, and a new basis vector is discovered. This is when you have to seriously think and incorporate it into your relevancy equation or concede. This is a good way to learn new perspectives in the world.

Now, the other situation is purely political. Some people hate being wrong and hate changing their mind, and they will fight. I recommend just avoiding these people in life as life is too short to deal will closed-minded people. Now, this isn't you being wrong, this is the other person making you wrong especially if they can (i.e. your client, your boss, your co-workers, your spouse).

That being said, you can surprise people using new points of view and take people out of an one-sided debate and make it a discussion. For instance, when I was back on campus, some people came up to me as they wanted to share their faith with me. My friends know that I have no faith as I am a robot (actually, I invented my own religion, but that's an aside I'll share when I try to sell it... it's going to make super-super rich).

Here is how the conversation went.

Them: "We wanted to discuss your faith, what do you believe in?"

Me: "Money"

Them: "..."

/* note: I doubt anyone on a campus has ever given that answer */

Me: "I believe money is a side-effect from wealth, and building wealth creates jobs and makes life better for everyone."

Them: "Well, what happens when you die?"

Me: "Well, I hope the businesses I build continue on without me or someone puts them out of business with something better."

Them: "What happens to you?"

Me: "I don't know, but I don't really care anymore since what will happen will happen."

Them: "So, have you heard of Jesus?"

Me: "Yes, he was an awesome salesman. I know an awesome salesman too named Marc."

Them: "But, "

Me: "Let me stop you right there, who is paying you to sell me on your religion?"

Them: "No one"

Me: "That's awesome, I wish I could get college kids to work for me for free as I got lots of stuff to sell"

Them: "But we are not salesmen"

Me: "Sure you are"

Them: "But, salesmen sell a product, we are trying to help you."

Me: "If I have a lot of dirt on my floor, then a salesmen will help me with that problem by selling my a vacuum cleaner. You just don't understand your product yet. Your product is an answer to the question 'what happens why I die?' that you tend to ask yourself sometimes before you go to bed."

Them: "..."

Me: "See, the way I look at it, Islam and Christianity are just like Pepsi and Coke. They are products. Now, I don't have consumers digest to help me pick, and I've heard lots of bug reports from both sides. Personally, I'd rather just avoid the issue and do something else with my time"

Them: "... umm"

At this point, the conversation became small talk about my shoes and nothing important. They walked away visibly confused. The point I'm trying to make is that I brought up a point they have never considered before, and that basically confused them.

I don't really have a point to this article, but I wanted to share the story and talk about my philosophy. While I do enjoy being right, I enjoy learning more.

What are the most in-demand programming skills for 2011 and beyond, & why?

Being good at code is always in demand, and it is far more important to be effective at reading code. To practice, you can take a library that does something you understand (say convert a csv into a data-structure) then strip out the comments and trace it. Extra points goes to the one that can do this under some kind of source obfuscation. I've met too many programmers who can't read code at a serious level, and I've met programmers that rely on superfluous comment; however, there is nothing except hope and trust that ensure the comment and the code are in parity.

No matter what, JavaScript is going to be the next C for the web. Everyone is going to know it. Due to its predominance, I wager there is going to be a boom in the programming language market searching for nicer languages to compile to JavaScript. See: http://en.wikipedia.org/wiki/CoffeeScript

I further wager that the whole difference between asynchronous programming and multi-threaded programming is going to be wrapped into a neat present with a nice bow-tie. Working with node.js (or twisted or event machine) will give you a leg up on new design skills as more businesses rely on node.js. Learn Asynchronous design.

NoSQL is just getting started, and I wager there is going to be evolution in who wins the developer market. I wager CouchDB (http://blog.mathgladiator.com/2010/11/how-i-use-couchdb.html), because I'm a betting man. I also wager someone is going to write (I've thought about it) a SQL for NoSQL. Once this is invented, I wager Oracle is going to buy that company or sue it into oblivion. Learn MapReduce.

Java/.NET wll be good for a long time, and it is the new COBOL. There are a couple of problems that may unseat them: Multiple cores with good parallelization, good asynchronous primitives, and stronger type systems (well, for the markets that need that type of stuff anyway). At least know either Java or .NET.

Java/. NET are always going to be ugly to someone, and that someone may invent something better (i.e. Ruby). It's important to jump in some bandwagons, so find something esoteric and learn it as a hobby, It could be big some day. You want to help make it big? Then build a business using it, and you will put your ass in the fire when things go wrong.

UX+Dev+Marketing is going to be key in the coming years, and I would keep abreast with those markets as you either may need it someday or you may want to sell them something.

Databases as we know them today are going to die... over the next ten years, so I would pick a NoSQL platform and start playing with the Idoms now. My picks are: CouchDB, Riak, Redis, and Cassandra.

Now, you could say that all code should be properly documented. Ok, well, tell that to the guy that left and you are a consultant coming in to add a feature. You are now screwed. If you can get effective at reading and understanding code, then you can easily charge $250/hr to fix really simple things in five to ten minutes (do lump sum billing and you are rewarded for being effective rather than get-it-done on a schedule, see http://en.wikipedia.org/wiki/Parkinson's_Law).

These are purely technical skills. You could amp up and start to learn where the industry is going by getting in touch with it. I recommend lurking on http://news.ycombinator.com/news. Once you see where business is kind of going, you can guess where the technology needs to go for businesses to get there. Predict business needs and you can predict technology.