BangDB - nosqldb

BangDB - Multiflavored, distributed, transactional, high performance nosql database written in C/C++ from scratch for scale out apps suitable for heavy lifting. BangDB is available as;

Embedded Datastore

Client Server Model

Data Grid / Elastic Data Store

BSD license for embedded and client server model, download and use freely

Persistent / In-memory Key value Store

High Performance

BangDB is a high performance database. It leverages the available cpus (or cores) as best as possible and provides high concurrency resulting in better performance

The db implements its own buffer pool, adaptive page pre fetch and flush mechanism, almost lock free index structure access which helps optimize the performance

Please check out the results of get and put for 1 billion operations on a commodity machine (8GB RAM, 4 cores) in the adjacent graph

Multiple Table Types

BangDB provides explicit native support for different use cases and data types for enhanced efficiency, performance and ease of programming

• Normal Table - simple key and value, where value is opaque
• Primitive Table - primitive key and value types, (int, long etc...)
• Wide Table - structured value (ex; json string) with multiple indexes

User may create as many different tables as required to suit the different requirement for different types(kinds) of data within an application. This is a important shift from usual way of dealing all data types in same fashion which is inherently not efficient

Indexing

Create multiple indexes on structured or non-structured data for quicker and efficient access. The user may create as many indexes as required and select the way index should be created/stored in the db for ex; using Btree or Exthash, sorted in ascending or descending order, unique or with duplicates, data type of indexed field etc...

The index can be created and dropped at runtime using simple API, for ex; in c#

tbl.AddIndex_Num(idxName1, allowDuplicate);

tbl.AddIndex_Str(idxName2, 24, allowDuplicate);

tbl.DropIndex(idxName1);

Abstraction for analytics

BangDB provides fully baked up abstraction for real time data analysis. The abstraction hides all the complexities and stores data in a fashion which avoids post processing during the query time, resulting in highly efficient and performant retrieval of insight at run time.

While BangDB features can be used to create different abstractions by the user for different kinds of analysis, these abstractions are provided to address selected query types and works backword to fully implement what it takes to enable the analytics

• Sliding Window - An abstraction which enables user to store and retrieve data for a given sliding window
• Counting - Absolute, probabilistic, unique, non-unique counting, within or outside sliding window
• TopK - Top k items, within or outside sliding window

Many of the user queries can be mapped to combinations of the above abstractions. More specific abstractions for different use cases would be added in coming days

Transaction

BangDB supports full ACID transactions for embedded and client server model. To achieve high performance with full ACID transaction, the db implements OCC (optimistic concurrency control) in order to achive high concurrency

The transaction is a property for BangDB and can be switched on/off using config file

Buffer Pool, Page Cache

BangDB implements its own buffer pool, page cache, block IO layer, custom memory manager, quasi adaptive page flush and prefetch module for temporal locality for write and spatial locality for read. More control on demand paging, swapping, block IO helps improve the performance of the db on commodity machine with limited amount of RAM

The buffer pool and other data structure allows concurrent access by different threads simultaneously which increases the performance of db effectively

Write ahead logging

BangDB implements modified ARIES algorithm for better logsize/datasize optimization and low overhead for sequential write ahead logging.

It has built in crash recovery which automatically kicks in when db is restarted

Log can be enabled or disabled using config param

Shared or Private log as options for multiple tables

Access Methods

BangDB implements two ways to access data using index, one is B+link Tree and other one is Ext Hash

BTree option has O(logn) complexity whereas hash has O(1), but Btree allows range scan, provides cursor for itaration etc.

DB allows concurrent access to these data structure which along with concurrent buffer pool implementation makes the db leverage number of cores in the mcachine

Staged Event Driven Server

BangDB server is highly concurrent, event driven, replicated, high performance nosql database server

Designed to handle tens of thousands of concurrent connections with low memory overhead in optimized manner. It implements its own message protocol and framework, slab allocator, event managers (pluggable), async operation management, replication engine, timeouts, epoll based multiplexing etc... in order to achieve high peroformance

Typical read and write can be done at the rate of 200K IOPS using several hundreds connections. Client API is same as of embedded db hence can be ported seamlessly from embedded to server

Replication

Client Server flavor of the db supports replication to the standby (slaves)

Replication to the slaves happen in sync /async fashion based on setting and the replica (slaves) can be added at run time without bringing the master to halt and all operations on the server can continue while initial sync and replication is in progress

BangDB Core API

BangDB has set of API to perform various activites such as CRUD. User can create many tables, perform actions and do various queries in easy fashion. Following example code is in java

	Database db = new DatabaseImpl("mydb", null, DB_MULTIOPS_TRANSACTION_NONE, null);
		
	Table tbl = db.getTable("mytable", DBACCESS.OPENCREATE, null);
		
	Connection conn = tbl.getConnection();
			
	conn.put(k1, v1, INSERTOPTIONS.INSERT_UNIQUE);
						
	byte[] v2 = conn.get(k1);
						
	ScanFilter sf = new ScanFilter();
		
	ResultSet rs = conn.scan(sk, ek, sf);
		
	while(rs.hasNext())
	{
		rs.moveNext();
	}
	rs.clear();
						
	db.closeDatabase(DBCLOSE.DEFAULT);

Simple SQL Support

BangDB support minimum set of SQL commands (SQL like commands) as of now, which can be used against server from command line tool

	create table 'mytable';
	
	describe database;
	
	describe table 'mytable';
	
	insert into 'mytable' values 'key1' 'value1';
	
	select * from 'mytable' where key >= 'key1' limit 10;
	
	update 'mytable' set val = 'value2' where key = 'key1';
	
	delete from 'mytable1' where key > 'k' and key <= 'k2'

GRID Creation and Access

Simple commands to create, join, leave and get stats for a node in the grid

	create grid -i '192.168.1.2:9394'
	
	join grid -i '192.168.1.3:9394' -s '192.168.1.2:9394'
	
	stat grid -i '192.168.1.3:9394'
	
	leave grid -i '192.168.1.2:9394'

Batch to Real Time Processing

BigData Buffer allows user to collect data in elastic space in real time fashion and does simple aggregation and corelation in quasi real time fashion which overall moves the batch mode analysis to real time analysis (Upcoming...)

Focus on Activity Data

Activity data allow operational intelligence to be integrated with the business processes in real time. Numerous use cases for this (upcoming...)

Download BangDB

BangDB embedded and client server are available freely under BSD license, please download and use as you may wish

BangDB - Forum / Contact us

If you have any technical or product related questions that can be put on public forum please go to the BangDB forum and post them

Or else please send direct quries at contact us

Subscribe, follow us

Please subscribe to the monthly newsletter

Also please follow us for regular updates and activities

• Follow Iqlect