> For the complete documentation index, see [llms.txt](https://ankit-apdc.gitbook.io/system-design/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ankit-apdc.gitbook.io/system-design/building-blocks/database/nosql.md). # NoSQL Not Only SQL, also known as non-relational database > “A **NoSQL** (originally referring to "non [SQL](https://en.wikipedia.org/wiki/SQL)" or "non relational") [database](https://en.wikipedia.org/wiki/Database) provides a mechanism for [storage](https://en.wikipedia.org/wiki/Computer_data_storage) and [retrieval](https://en.wikipedia.org/wiki/Data_retrieval) of data that is modelled in means other than the tabular relations used in [relational databases](https://en.wikipedia.org/wiki/Relational_database).” — Wikipedia ## Properties of NoSQL Database Four Operations, in general, we want to do in any database: CRUD (Create, Read, Update, Delete) {% hint style="success" %} **BASE Property** * **B**asically **A**vailable: The system is guaranteed to be available in event of failure. * **S**oft State: The state of the data could change without application interactions due to eventual consistency. * **E**ventual Consistency: The system will be eventually consistent after the application input. The data will be replicated to different nodes and will eventually reach a consistent state. But the consistency is not guaranteed at a transaction level. {% endhint %} ## Different types of NoSQL Database ### **Graph stores** > * The graph relates the data items in the store to a collection of nodes and edges, the edges representing the relationships between the nodes {% hint style="info" %} **Node**: entities or instances such as people, businesses, accounts, or any item to be tracked **Edges**: also termed graphs or relationships, are the lines that connect nodes to other nodes; representing the relationship between them (directed or un-directed) **Properties**: information associated to nodes {% endhint %} > * Visualisation advantage, great for storing a relationship between different data points in node form and see the complex relationships between different data-points > * Used for semantic queries (and therefore great for fraud detection) {% hint style="success" %} **Semantic queries** enable the retrieval of both explicitly and implicitly derived information based on [syntactic](https://en.wikipedia.org/wiki/Syntax), [semantic](https://en.wikipedia.org/wiki/Semantics) and [structural information](https://en.wikipedia.org/wiki/Structural_information_theory) contained in data {% endhint %} > * neo4J, OrientDB, Titan are examples of graph store as database ### ### **Column Stores** > Similar to relational database, data is stored in both rows and columns. However, columns may contain multiple values, allowing data to be fetched by row or by column for highly optimised data retrieval > > Here in the image, shown one row and each column can store an entity (keys and values) or object ![](https://679135566-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-Ma8GKQU92FWew4_578V%2F-MaI6g8nKGM9M_h6hQOM%2F-MaIBZzbU9gu2dh8H0wG%2Fimage.png?alt=media\&token=e43b47b3-70e5-45d6-be08-dc6bb0e3ea70) > * HBase and Cassandra are the examples of column stores type database > > **Who should use it?** > > * most popular for use with IoT (internet of things) technology because it offers fast, real-time insights > * It excels at writing time-based log activities, error logging, and sensor data > * If need fast read and write processing > * also good for those who want to work with SQL-like data types on a NoSQL database ### **Key-value stores** > if speed is the only objective (example: Redis and Memcached) > > **Strength**: Incredibly fast, no indexing is required, direct access using key > > **Drawbacks**: > > * It’s impossible to query values, because they’re stored as a blob and can only be returned as such, makes it hard to do reporting or edit parts of values. > * Not all objects are easy to model as key-value pairs ### **Document stores** > * flexible schema, dynamic querying is possible (examples: MongoDB, Couchbase) > * is designed to store and query data as JSON-like documents. > * The flexible, semi-structured, and hierarchical nature of documents and document databases allows them to evolve with applications’ needs. > * The document model works well with use cases such as catalogs, user profiles, and content management systems where each document is unique and evolves over time. > * Document databases enable flexible indexing, powerful ad hoc queries, and analytics over collections of documents. > * **Drawbacks**: > > * Document databases sacrifice ACID compliance for flexibility > > * Also, while querying can be done in a document, it’s not possible across documents > * **Who should use it?**: When schema is ever evolving and can not be fixed (example mobile app) ### **Search Engine** > Elasticsearch is a document-based data storage and retrieval solution specifically arranged and optimised for the storage and rapid retrieval of data. Data is indexed while storing and is based on powerful [Lucene](https://en.wikipedia.org/wiki/Apache_Lucene) library for search > > * **Properties**: distributed, [multitenant](https://en.wikipedia.org/wiki/Multitenancy)-capable, full-text search engine with an HTTP web interface, and schema-free JSON documents > * **Strengths**: > * Elasticsearch is very scalable > * It features flexible schema and fast retrieval of records, with advanced search options including full-text search, suggestions, and complex search expressions > * **Drawbacks**: > * Elasticsearch is used more as an intermediary or supplementary store than a primary database. > * It has low durability and poor security. There’s no innate authentication or access control, doesn’t support transactions ### Time Series Data Stores > * Time series data is a set of values organised by time > * Time series data stores must support a very high number of writes, as they typically collect large amounts of data in real time from a large number of sources > * Updates are rare, and deletes are often done as bulk operations ### Object Data Stores > Object data stores are optimised for storing and retrieving large binary objects or blobs such as images, text files, video and audio streams, large application data objects and documents, and virtual machine disk images ## Comparison | Database Type | Performance | Scalability | Flexibility | Complexity | | ------------- | ----------- | ---------------- | ----------- | ---------- | | Graph-Based | Variable | Variable | High | High | | Column | High | High | Moderate | Low | | Key-Value | High | High | High | High | | Document | High | Variable to High | High | Low | {% hint style="info" %} Apart from permanent storage, database is required for multiple use cases; here are few commonly used database for different tasks * **Caching (in-memory)**: Memcached, Redis * **For Message Broker or Queue**: RabbitMQ, Redis, AWS SQS * **For Streaming purpose** (for data flow pipeline ): Kafka * **For Search Engine**: Elasticsearch, Solr * **For Objects** (image, pdf and other files storage): aws s3 * **Relationship Storage** (Knowledge Graph): Neo4j (graph data) {% endhint %} ## References for Further Reading {% embed url="." %} Must Read {% endembed %} {% embed url="" %} {% embed url="" %}