Saturday, August 21, 2010

NoSQL and You, what is the daily deal for databases

To promoters of the relational world, I wish to speak to you as a concerned human being. Your days are numbered just like the days were numbered for COBOL developers. I wager I'm in the middle group between innovators and early majority, and I'm working towards figuring out what NoSQL solutions mean for businesses that rely on persistance.

This isn't a rant about scalability.

Scalability is just one facet that the NoSQL movement aims to provide, and it is the big facet that will get important players involved to fund the polish that makes NoSQL solutions awesome. Scalability isn't the biggest argument that promotes NoSQL solutions. The biggest and best argument is flexibility and how we can design software around NoSQL and solve problems quickly.

Flexibility always Wins

Unless your art (or job) demands you work in assembly, then you don't work in assembly. The struggle to produce high level languages has yielded a multitude of languages that are in-arguably superior to older languages for many domains. The quest for a better way to express how we compute continues on. and on. and on. and on.

The NoSQL movement is no different, and it is core to our quest for a better way to represent and share data. We seek better ways to store data. We seek better ways to share and transact data with other processes over a network. We want need our queries to be fast, optimal, and beautiful.

I believe the current generation of NoSQL solutions provide a platform for making our lives better, our work more productive, and enable us the flexibility to refine our services.

Momentum

Due to the companies that sponsor the work behind NoSQL solutions because they have scalability problems, we have enough momentum and interest to overcome the technical debt that causes typical databases to remain entrenched in small and medium sized sectors. NoSQL solutions are untested in many sectors and domains, but the current generation of start-ups are going to test them and polish them to make them work for a larger class of problems. Due to the open source nature of the multitude of NoSQL solutions, the knowledge to solve problems and the tools will be available either for free or offered by a vendor. This is great!

What is wrong with a Database?

Nothing! But, there is nothing wrong with assembly either. Sure, you can build any NoSQL solution offering with a Database. But then you are building on top of a crappy NoSQL offering. What matters in NoSQL isn't where or how you store data, but how you design and build services. Being competitive in this day and age means being flexible to adapt and iterate. How does one enable flexibility with NoSQL? How does one execute and deliver change to make services better? These are open questions that the programming community is facing now, and we will be answering them soon.

The Complement Set

The relational theory set forth back in the 70s is an amazing mathematical model. However, it is just one model. Is it complete? What is the set of all solutions that span the offering of a database package? Is it everything? No. The problem that set me on the path towards NoSQL solutions is that of graph traversals. Relational databases are horrible at graphs. It is no surprise that due to the rise of social networks, the interest in graph based databases has gone up as well. While this is a single problem, it is sufficient to demonstrate that relational databases are not the everything panacea that some claim it to be.

Better Database Education?

"Just learn everything your database offers!" While this is an argument that I would have given years ago, it is not an appropriate response for entrepreneurial computer science (the name of my future book). It is far more important to create a product that generates revenue, so if I run into a business problem that depends on code then I need to solve it quickly to test viability and get market feedback.

Using a database may be the wrong way to persist data, which means that what you initially learned about databases is probably bunk. Most medium sized problems can be solved by using insane amounts of memory as an intermediate cache to some Key Value Pair System (for backups and failure persistence). If I go back in time, I would unlearn what I know about databases and just make a product like I was doing when I was 16. All the knowledge of how seek time affects hard drive performance, network latency, and all the other tier data bottlenecks is bullshit (given that in today's world I can get commodity servers with insane memory). I can use that memory to solve problems at both the small and medium scale to get revenue, and that revenue will fund the larger scale version which may be easier to build due to the choice of NoSQL designs in the beginning.

I believe NoSQL techniques are the path from transient memory to a persisted disk, and I feel that designing and learning how to make NoSQL work will be more valuable than knowing how to configure and consume a database package.

In the End

What matters most is providing a service to the world.

If you argue that a relational database is good enough, then it is good enough... for you. If that is the case, then don't tell me what is good enough because you don't know the crazy shit I'm working on that could change the world for the better!

1 comment:

  1. "The biggest and best argument is flexibility and how we can design software around NoSQL and solve problems quickly."

    Couldn't agree more!

    ReplyDelete