The end of SQL and relational databases? (part 2 of 3)
- TAGS:cloud computing, enterprise, Graph, Key-Value, NoSQL, open source, SQL
- IT TOPICS:Applications, Desktop Apps, Development, Emerging Technology, Enterprise Apps, Open Source, Web Apps
In "The end of SQL and relational databases? (part 1 of 3)" I covered some background on the SQL language and relational databases, the current and future for relational databases, the rise of frameworks that hide some of the complexities of database programming and the rise of the NoSQL databases. In this second installment I will take a walk through (some of) the currently available open and closed source NoSQL databases. Then, in part 3, I will point you to NoSQL Internet resources, past/upcoming events and offer some guidance for developers.
It's amazing to see how many open and closed source alternative data stores have appeared. At the same time more are appearing every day. If I have left off one or more of your favorite NoSQL databases post a comment. Below you will find many different types of NoSQL databases: Document-Oriented, Collection-Oriented, Column-Oriented, Object-Oriented, Graph-Oriented, Set-Oriented, Row-Oriented and more.
AllegroGraph |
|
| Company/Org: | Franz Inc. |
| Type: | Graph |
| Description: | Modern, high performance, persistent graph database. |
| Storage: | Disk based, meta-data and data triples. |
| API(s): | SPARQL, Prolog |
BerkleyDB |
|
| Company/Org: | Oracle |
| Type: | Key/Value |
| Description: | C language embeddable library for enterprise-grade, concurrent, transactional storage services. Thread safe to avoid data corruption or loss |
| Storage: | B-tree, hash table, persistent queue |
| API(s): | C, C++ and Java |
| Notes: | Use BerkleyDB XML layer on top of BerkleyDB for XML based applications. Comparison of BerkleyDB and relational databases |
BigTable |
|
| Company/Org: | |
| Type: | Sparse, distributed, persistent multidimensional sorted map. |
| Description: | Distributed storage system for structured data. Data model provides dynamic control over data layout and format. Data can live in memory or on disk. |
| Storage: | Data is stored as an uninterpreted array of bytes. Client applications can create structured and semi-structured data inside the byte arrays. |
| API(s): | Python, GQL, Sawzall API, REST, various. |
| Notes: | Overview: Bigtable: A Distributed Storage System for Structured Data (PDF format) |
Cassandra |
|
| Company/Org: | Apache |
| Type: | Dimensional hash table |
| Description: | Highly scalable distributed database. Combines Dynamo's distributed design and Bigtable's column family data model. |
| Storage: | Clusters of multiple keyspaces. The keyspace is a name space for column families. Columns are comprised of a name, value and timestamp. |
| API(s): | Java, Ruby, perl, Python, C#, Thrift framework. |
| Notes: | Open sourced by Facebook in 2008. Wiki, FAQ, Examples |
CouchDB |
|
| Company/Org: | Apache |
| Type: | Document |
| Description: | Distributed database with incremental replication, bi-directional conflict detection and management. |
| Storage: | Ad-hoc and schema-free with a flat address space. |
| API(s): | RESTful JSON API. JavaScript query language. |
| Notes: | CouchDB Introduction, Technical Overview |
db4o |
|
| Company/Org: | Versant |
| Type: | Object |
| Description: | Java and .NET dual license (commercial and open source) object database. |
| Storage: | Data objects are stored in the way they are defined in the application. |
| API(s): | Java, .NET languages. |
| Notes: | db4o db4o database runtime engine, about db4o |
Dovetaildb |
|
| Company/Org: | Millstone Creative Works |
| Type: | JSON-based |
| Description: | Schemaless database similar to Amazon's SimpleDB. Open source, standalone Java application server. |
| Storage: | JSON data format, "bags" (similar to tables). |
| API(s): | HTTP and Javascript APIs |
| Notes: | Dovetaildb JavaScript API reference manual |
Dynomite |
|
| Company/Org: | Cliff Moon |
| Type: | Key/Value |
| Description: | Open source Amazon Dynamo clone written in Erlang. |
| Storage: | Distributed key/valve store, Pluggable storage engines. |
| API(s): | Thrift API |
| Notes: | Dynomite Wiki |
eXtreme Scale |
|
| Company/Org: | IBM |
| Type: | In-memory grid/cache |
| Description: | Distributed cache processes, partitions, replicates and manages data across servers. |
| Storage: | Data and database cache, "near cache" for local subset of data. Java persistent cache. Map reduce support. |
| API(s): | Java APIs, REST data service |
| Notes: | eXtreme Scale Document library web site |
GT.M |
|
| Company/Org: | FIS |
| Type: | Hierarchical, multi-dimensional sparse arrays, content associative memory |
| Description: | Small footprint, multi-dimensional array with fill support for ACID transactions, optimistic concurrency and software transactional memory. |
| Storage: | Unstructured array of bytes. Can be Key/Value, document oriented, schema-less, dictionary or any other data model. |
| API(s): | Mumps, C/C++, SQL |
| Notes: | GT.M FAQ |
hamsterDB |
|
| Company/Org: | Christoph Rupp |
| Type: | Embedded storage library |
| Description: | Lightweight embedded database engine. Supports on disk and in memory databases. |
| Storage: | B+tree with variable length keys. |
| API(s): | C++, Python, .NET and Java |
| Notes: | hamsterdb FAQ, examples, tutorial |
HBase |
|
| Company/Org: | Apache |
| Type: | Sparse, distributed, persistent multidimensional sorted map. |
| Description: | Open source, distributed, column-oriented, "Bigtable like" store |
| Storage: | Data row has a sortable row key and an arbitrary number of columns, each containing arrays of bytes. |
| API(s): | Java API, Thrift API, RESTful API |
| Notes: | Part of Apache Hadoop project. HBase Wiki, FAQ |
Hypertable |
|
| Company/Org: | Zvents Inc. |
| Type: | Sparse, distributed, persistent multidimensional sorted map. |
| Description: | High performance distributed data storage system designed to run on distributed filesystems (but can run on local filesystems). Modeled after Google Bigtable. |
| Storage: | Row key (primary key), column family, column qualifier, time stamp. |
| API(s): | C++, Thrift API, HQL |
| Notes: | Hypertable Architectural overview, FAQ |
Infinispan |
|
| Company/Org: | JBoss Community |
| Type: | Grid/Cache |
| Description: | Scalable, highly available, peer to peer, data grid platform. |
| Storage: | Key/Value pair with optional expiration lifespan. |
| API(s): | Java, PHP, Python, Ruby, C |
| Notes: | Infinispan FAQ, Wiki |
InfoGrid |
|
| Company/Org: | Â |
| Type: | Graph |
| Description: | Internet graph database made up on nodes and edges. Supports in-memory and persistent storage alternatives including RDBMS, file system, file grid, and custom storage. |
| Storage: | Nodes (meshobjects) and edges (relationships). Meshobjects can have entity types, properties and participage in relationships. MeshObjects raise events. |
| API(s): | RESTful web services. |
| Notes: | InfoGrid Overview, FAQ |
Keyspace |
|
| Company/Org: | Scalien |
| Type: | Key/Value |
| Description: | Distributed (master/slave) key-value data store delivering strong consistency, fault-tolerance and high availability. |
| Storage: | Uses BErkeleyDB library for For local storage. Key/Value pairs and their state are replicated to multiple servers. |
| API(s): | C/C++, Python, PHP, HTTP |
| Notes: | Keyspace Overview, FAQ |
MemcachedDB |
|
| Company/Org: | Â |
| Type: | Key/Value |
| Description: | High performance, high realiability persistent storage engine for key/value object storage. |
| Storage: | Uses BerkeleyDB as storage library/backend. |
| API(s): | Memcache protocol, C, Python, Java, perl |
| Notes: | MemcacheDB complete guide (PDF format) |
Mnesia |
|
| Company/Org: | Ericsson |
| Type: | Key/Value |
| Description: | Multiuser distributed database including support for replication and dynamic reconfiguration. |
| Storage: | Organized as a set of tables made up of Erlang records. Tables also have properties including type location, persistence, etc. |
| API(s): | Erlang |
| Notes: | Mnesia Reference manual |
MongoDB |
|
| Company/Org: | 10gen |
| Type: | Document |
| Description: | Scalable, high-performance, open source, schema-free, document-oriented database |
| Storage: | JSON-like data schemas, Dynamic queries, Indexing, replication, MapReduc |
| API(s): | C,C++, Java, JavaScript, perl, PHP, Python, Ruby, C#, Erlang, Go, Groovy, Haskell, Scala, F# |
| Notes: | MongoDB Documentation Index |
Neo4J |
|
| Company/Org: | Neo Technology |
| Type: | Graph |
| Description: | Embedded, small footprint, disk based, transactional graph database written in Java. Dual license - free and commercial. |
| Storage: | Graph-oriented data model with nodes, relationships and properties. |
| API(s): | Java, Python, Ruby, Scala, Groovy, PHP, RESTful API. |
| Notes: | Neo4J Wiki, API, FAQ |
Redis |
|
| Company/Org: | Â |
| Type: | Key/Value |
| Description: | Key/Value store with the dataset kept in memory and saved to disk asynchronously. "not just another key-value DB" |
| Storage: | Values can be strings, lists sets and sorted sets. |
| API(s): | Python, Ruby, PHP, Erlang, Lua, C, C#, Java, Scala, perl |
| Notes: | Redis Wiki |
SimpleDB |
|
| Company/Org: | Amazon |
| Type: | Item/Attribute/Value |
| Description: | Scalable Web Service providing data storage, query and indexing in Amazon's cloud. |
| Storage: | Items (like rows of data), Attributes (like column headers), and Values (can be multiple values) |
| API(s): | SOAP, REST |
| Notes: | SimpleDB FAQ, Getting Started Guide, Developer Guide, API |
Tokyo Cabinet |
|
| Company/Org: | Mikio Hirabayashi |
| Type: | Key/Value |
| Description: | Library (written in C) of functions for managing files of key/value pairs. Multi-thread support. |
| Storage: | Keys and Values can have variable byte length. Binary data and strings can be used as a key and a value. |
| API(s): | C, perl, Ruby, Java, Lua. |
| Notes: | Tokyo Cabinet Specifications, presentation (PDF format). Also available: Tokyo Tyrant (remote service), Tokyo Distopia (full text search), Tokyo Promenade (content management). |
Voldemort |
|
| Company/Org: | |
| Type: | Hash Table |
| Description: | "It is basically just a big, distributed, persistent, fault-tolerant hash table." High performance and availability. |
| Storage: | Each key is unique to a store. Each key can have at most one value. Supported types: JSON, string, identity, protobuf, java-serialization. |
| API(s): | Java, C++, custom clients |
| Notes: | Project Voldemort Wiki, Client how-to |
It's one thing to have lots of choices for non-relational databases. Building up a NoSQL knowledge and experience base will definitely help managers, architects and developers compare and contrast what they already know about relational databases. Relational databases and the SQL language are still the architecture and lingua franca for the design, development and management of database applications. While we are still at the beginning of the use of databases in cloud infrastructures, we can move forward faster because of all of the work and collaboration that is taking place. Depending on the user and business requirements, we can choose between existing relational database technologies or the NoSQL alternatives.
Stay tuned to the conclusion (part three) of this blog series where I'll give you links to additional NoSQL Internet resources, past/upcoming events, offer some guidance for developers and highlight some of the comments and answer some of the questions from the first two parts.
Are you using a non-relational database? Have you given up on SQL and relational databases? Are you moving your data to a public or private cloud? Post a comment.
Programming is Life!
Recent news for developers:
- Microsoft offers Visual Studio 2010 release candidate
- Toyota to recall Prius hybrids over ABS software
- Deal Done, Oracle Looks to Reassure Sun Users
- Rails 3.0 beta features merger with Merb Web dev framework
- The 10 best IT jobs right now
- Facebook Makes Major PHP Push With HipHop
- Apps Coming to Amazon Kindle
David Intersimone (David I) is the Vice President of Developer Relations and Chief Evangelist for Embarcadero Technologies. My company blog is at http://blogs.embarcadero.com/davidi
Free Insider Guide