NoSQL is getting a lot of traction and hype these days but in reality it's not that new of a thing. I thought I'd trace the roots of NoSQL and see what I'd find. The name "NoSQL" was in fact first used by Carlo Strozzi in 1998 as the name of file-based database he was developing. Ironically it's relational database just one without a SQL interface. As such it is not actually a part of the whole NoSQL movement we see today. The term re-surfaced in 2009 when Eric Evans used it to name the current surge in non-relational databases. It seems like the name has stuck for better or for worse. Note that not all projects are included in this post. See the post on analyzing the NoSQL landscape for a more complete listing.
1960s
- MultiValue (aka PICK) databases are developed at TRW in 1965.
- According to comment from Scott Jones M[umps] is developed at Mass General Hospital in 1966. It is a programming language that incorporates a hierarchical database with B+ tree storage.
- IBM IMS, a hierarchical database, is developed with Rockwell and Caterpillar for the Apollo space program in 1966.
1970s
- InterSystems develops the ISM product family succeeded by the Open M product, all M[umps] implementations. See comment from Scott Jones below.
- M[umps] is approved as a ANSI standard language in 1977.
- in 1979 Ken Thompson creates DBM which is released by AT&T. At it's core it is a file-based hash.
1980's
Several successors to DBM spring into life.
- TDBM supporting atomic transactions
- NDBM was the Berkeley version of DBM supporting having multiple databases open at the same time.
- SDBM - another clone of DBM mainly for licensing reasons.
- GT.M is the first version of a key-value store with focus on high performance transaction processing. It is open sourced in 2000.
- BerkeleyDB is created at Berkeley in the transition from 4.3BSD to 4.4BSD. Sleepycat software is started as a company in 1996 when Netscape needed new features for BerkeleyDB. Later acquired by Oracle which still sell and maintain BerkeleyDB.
- Lotus Notes or rather the server part, Lotus Domino, which really is a document database has it's initial release in 1989, now sold by IBM. It has evolved a lot from the early versions and is now a full office and collaboration suite.
1990's
- GDBM is the Gnu project clone of DBM
- Mnesia is developed by Ericsson as a soft real-time database to be used in telecom. It is relational in nature but does not use SQL as query language but rather Erlang itself.
- InterSystems Caché launched in 1997 and is a hybrid so-called post-relational database. It has object interfaces, SQL, PICK/MultiValue and direct manipulation of data structures. It is a M[umps] implementation. See Scott Jones comment below for more on the history of InterSystems
- Metakit is started in 1997 and is probably the first document oriented database. Supports smaller datasets than the ones in vogue nowadays.
2000-2005
This is were the NoSQL train really picks up some momentum and a lot is starting to happen.
- Graph database Neo4j is started in 2000.
- db4o an object database for java and .net is started in 2000
- QDBM is a re-implementation of DBM with better performance by Mikio Hirabayashi.
- Memcached is started in 2003 by Danga to power Livejournal. Memcached isn't really a database since it's memory-only but there is soon a version with file storage called memcachedb.
- Infogrid graph database is started as closed source in 2005, open sourced in 2008
- CouchDB is started in 2005 and provides a document database inspired by Lotus Notes. The project moves to the Apache Foundation in 2008.
- Google BigTable is started in 2004 and the research paper is released in 2006.
2006-2010
- JackRabbit is started in 2006 as an implementation of JSR 170 and 283.
- Tokyo Cabinet is a successor to QDBM by (Mikio Hirabayashi) started in 2006
- The research paper on Amazon Dynamo is released in 2007.
- The document database MongoDB is started in 2007 as a part of a open source cloud computing stack and first standalone release in 2009.
- Facebooks open sources the Cassandra project in 2008
- Project Voldemort is a replicated database with no single point-of-failure. Started in 2008.
- Dynomite is a Dynamo clone written in Erlang.
- Terrastore is a scalable elastic document store started in 2009
- Redis is persistent key-value store started in 2009
- Riak Another dynamo-inspired database started in 2009.
- HBase is a BigTable clone for the Hadoop project while Hypertable is another BigTable type database also from 2009.
- Vertexdb another graph database is started in 2009
- Eric Evans of Rackspace, a committer on the Cassandra project, introduces the term "NoSQL" often used in the sense of "Not only SQL" to describe the surge of new projects and products.
(Some of these dates need to be taken with a small pinch of salt as finding out exactly when the projects started can be a bit difficult. Also not all projects started in last few years have been included)
In 2009 and 2010 we also saw the coming of NoSQL conferences like NoSQL live in Boston in 2010, the upcoming NoSQL eu in London in April 2010. Last year we also saw the NoSQL east conference in Atlanta.
Great overview Knut. CouchDB is inspired by Lotus Notes, which definitely has a place in this history.
You're absolutely right. Lotus Notes was in my notes for the complete(?) list of NoSQL projects but somehow managed to sneak out of the history notes.
I think you have a typo in the link to InfoGrid, should be http://infogrid.org/ . Also, it's been called InfoGrid since 2005 when it still was closed source.
This is a helpful list BTW.
Hey Johannes
Thanks for the info. Corrected now.
MongoDB started in 2007 as part of an open source cloud computing stack. First standalone release 2009.
CouchDB was started in 2005, and moved to the Apache Software Foundation in 2008.
Thanks for the history!
@dm and J. Chris Anderson: Thanks for your input. Article corrected and completed.
Hi Knut,
Your also missing MarkLogic which is a document database created in 2001 (http://en.wikipedia.org/wiki/Mark_Logic)
You mention GT.M, but seem to forget that GT.M was just an implementation of an ANSI standard language, MUMPS.
InterSystems' Caché product supports ANSI M[umps], but also is a much more complete product, with CacheObjectScript (object oriented language, based on M), CacheBasic (based on VBscript), and CacheMultiValueBasic (based on MultiValue/Pick Basics), as well as supporting accessing data as multidimensional arrays, as objects, as Pick style multivalue key/value stores, or as a relational database.
M[umps] started at Mass General Hospital back in 1966, and Pick/MultiValue back at TRW in 1965... way before the databases mentioned above.
What you are now calling NoSQL databases have been around for ages...
Nuno, as far as I know MarkLogic is classified as a XML database. Does it have similar capabilities to document databases too?
Your history is getting closer, but M[umps] was based on a B+ tree database, not a document oriented database.
It was the 3rd ANSI standard language, after Basic and Fortran(long before C and C++!)... the standard came in 1977, with updates in 1984, 1990, and 1995.
InterSystems Caché didn't just "surface" in 2000, it was launched in 1997, and was the successor of the Open M product, which in turn was the successor of the ISM-11 (PDP-11), ISM-VX (on Digital VAX/VMS), ISM-PC (DOS), ISM-DG (on Data General's proprietary operating system), ISM-VM (on IBM mainframes running VM/CMS), and ISM-UX (on many different Unix implementations).
InterSystems was founded in 1978, and its products have been (and are being) used at many very large, critical sites.
InterSystems over time also bought out other M[umps] implementors, such as DataTree, the DSM group at Digital, and MicroNetics, consolidating to better compete against the relational database vendors such as Oracle, Sybase, etc.
Things like DBM, TDBM, BerkeleyDB were very late, and lacked many features, compared to what was already available and used widely years earlier in the M[umps] world.
Caché can definitely be considered a "NoSQL" database (at least under the "Not only SQL" definition [InterSystems has had SQL support since the 1980s]), as long as you don't ignore databases that are not open source.
Scott, I bow to your superior knowledge in this matter ;-)
The "surface" comment was due to the fact that I couldn't find the start date of the project/product on the InterSystems web page and the first mention of the product in the wayback machine was in 2000. I see now that the wikipedia page for InterSystems the company mentions this but I somehow missed first time around.
I have no bias against commercial players but a lot of the NoSQL databases in vogue today are open source. And as mentioned in the post (now) the history was never meant to be complete, but rather the major developments. And thank you very much for completing the picture!
Your documenting this history is interesting, and can be useful for people learning about where NoSQL databases have come from (and also show that they've really been around for a long time, and can provide very scalable, reliable solutions).
I think it is good that people are finally breaking away from the assumption that a row oriented RDBMS is the only game in town.
Hi Knut,
I love History and programming, so I love your article. I was always wondering where the "NoSQL" came from.
Another to add to the 1960's ...
IBM Information Management System (IMS)
http://en.wikipedia.org/wiki/IBM_Information_Management_System
This is a hierarchical database system that has been the core of most legacy Fortune 1000 company data processing applications (engineering and accounting). More than likely, when you are doing online banking at a large bank, there is a legacy IBM IMS based application running on a cluster of mainframes servicing that request.
Thanks. Info added to the 60s section.
Metakit (http://en.wikipedia.org/wiki/Metakit) was the first column oriented database I heard about. It was created in 1997, and it used extensively by, at least, the TCL community.