NoSQL

Not Only SQL, also known as non-relational database

“A NoSQL (originally referring to "non SQL" or "non relational") database provides a mechanism for storage and retrieval of data that is modelled in means other than the tabular relations used in relational databases.” — Wikipedia

Properties of NoSQL Database

Four Operations, in general, we want to do in any database: CRUD (Create, Read, Update, Delete)

BASE Property

• Basically Available: The system is guaranteed to be available in event of failure.

• Soft State: The state of the data could change without application interactions due to eventual consistency.

• Eventual Consistency: The system will be eventually consistent after the application input. The data will be replicated to different nodes and will eventually reach a consistent state. But the consistency is not guaranteed at a transaction level.

Different types of NoSQL Database

Graph stores

• The graph relates the data items in the store to a collection of nodes and edges, the edges representing the relationships between the nodes

Node: entities or instances such as people, businesses, accounts, or any item to be tracked

Edges: also termed graphs or relationships, are the lines that connect nodes to other nodes; representing the relationship between them (directed or un-directed)

Properties: information associated to nodes

• Visualisation advantage, great for storing a relationship between different data points in node form and see the complex relationships between different data-points

• Used for semantic queries (and therefore great for fraud detection)

Semantic queries enable the retrieval of both explicitly and implicitly derived information based on syntactic, semantic and structural information contained in data

• neo4J, OrientDB, Titan are examples of graph store as database

Column Stores

Similar to relational database, data is stored in both rows and columns. However, columns may contain multiple values, allowing data to be fetched by row or by column for highly optimised data retrieval

Here in the image, shown one row and each column can store an entity (keys and values) or object

• HBase and Cassandra are the examples of column stores type database

Who should use it?

• most popular for use with IoT (internet of things) technology because it offers fast, real-time insights

• It excels at writing time-based log activities, error logging, and sensor data

• If need fast read and write processing

• also good for those who want to work with SQL-like data types on a NoSQL database

Key-value stores

if speed is the only objective (example: Redis and Memcached)

Strength: Incredibly fast, no indexing is required, direct access using key

Drawbacks:

• It’s impossible to query values, because they’re stored as a blob and can only be returned as such, makes it hard to do reporting or edit parts of values.

• Not all objects are easy to model as key-value pairs

Document stores

• flexible schema, dynamic querying is possible (examples: MongoDB, Couchbase)

• is designed to store and query data as JSON-like documents.

• The flexible, semi-structured, and hierarchical nature of documents and document databases allows them to evolve with applications’ needs.

• The document model works well with use cases such as catalogs, user profiles, and content management systems where each document is unique and evolves over time.

• Document databases enable flexible indexing, powerful ad hoc queries, and analytics over collections of documents.

• Drawbacks:

  • Document databases sacrifice ACID compliance for flexibility

  • Also, while querying can be done in a document, it’s not possible across documents

• Who should use it?: When schema is ever evolving and can not be fixed (example mobile app)

Search Engine

Elasticsearch is a document-based data storage and retrieval solution specifically arranged and optimised for the storage and rapid retrieval of data. Data is indexed while storing and is based on powerful Lucene library for search

• Properties: distributed, multitenant-capable, full-text search engine with an HTTP web interface, and schema-free JSON documents
• Strengths:

  • Elasticsearch is very scalable

  • It features flexible schema and fast retrieval of records, with advanced search options including full-text search, suggestions, and complex search expressions

• Drawbacks:

  • Elasticsearch is used more as an intermediary or supplementary store than a primary database.

  • It has low durability and poor security. There’s no innate authentication or access control, doesn’t support transactions

Time Series Data Stores

• Time series data is a set of values organised by time

• Time series data stores must support a very high number of writes, as they typically collect large amounts of data in real time from a large number of sources

• Updates are rare, and deletes are often done as bulk operations

Object Data Stores

Object data stores are optimised for storing and retrieving large binary objects or blobs such as images, text files, video and audio streams, large application data objects and documents, and virtual machine disk images

Comparison

Database Type	Performance	Scalability	Flexibility	Complexity
Graph-Based	Variable	Variable	High	High
Column	High	High	Moderate	Low
Key-Value	High	High	High	High
Document	High	Variable to High	High	Low

Apart from permanent storage, database is required for multiple use cases; here are few commonly used database for different tasks

• Caching (in-memory): Memcached, Redis

• For Message Broker or Queue: RabbitMQ, Redis, AWS SQS

• For Streaming purpose (for data flow pipeline ): Kafka

• For Search Engine: Elasticsearch, Solr

• For Objects (image, pdf and other files storage): aws s3

• Relationship Storage (Knowledge Graph): Neo4j (graph data)

References for Further Reading

Non-relational data and NoSQL - Azure Architecture Centerdocsmsft

Must Read

How to choose the right database for your enterpriseInfoWorld

Graph databaseWikipedia