GNU libdbh - Summary

This software is part of the GNU Project.

Library to create and manage 64-bit disk based hash tables residing
 on disk. Associations are made between keys and
 values so that for a given a key the value can be
 found and loaded into memory quickly. Being disk based
 allows for large and persistent hashes. 64 bit support
 allows for hashtables with sizes over 4 Gigabytes on 32
 bit systems. Quantified key generation allows for
 minimum access time on balanced multidimensional trees.

Libdbh is similar to GDBM, but different in basic design. While GDBM
uses quite a lot of memory when working with large files, libdbh use
of memory is tiny. While GDBM outperforms libdbh in tables with a
small number of records, libdbh outperforms GDBM when a large number
of records is involved (full performance tests are under way). Other
similarities and differences include:

Similarities:

• Both use a unique key to access a particular data element
• Both have variable size data records
• Both have functions to search, retrieve, modify, erase records
• Both have sequential iteration over all key/data pairs

Differences:

• GDBM also has variable size keys
• GDBM works with a memory hashtable to a flat data table file, with

records cached in virtual memory.

• LIBDBH is a structured multidimensional table file which takes

advantage of the built in cache mechanisms built into disk devices and
the OS file cache.

• GDBM is limited to < 2GB records, LIBDBH does not have this limit.
• GDBM reads everything into its cache, which efectively limits the

size of the data table to the available virtual memory before swapping
begins and performance deteriorates, the table size for LIBDBH does not
have this limitation.

• LIBDBH has threadsafe mode (for concurrent operation between threads)

and parallel safe mode (for concurrent operation between independent
processes).

• GDBM has several cool things, which can easily be incorporated into

LIBDBH, such as the definition of a user defined fatal function, and
import/export functionality.

• GDBM is two dimensional. There is only one way to go through the

table, from first to end. LIBDBH is multidimensional, you can go through
the table by dimensions (sweep) or across dimensions (fanout), and
each data record is a complete table in itself, so you can do
selective foreach procedures.


And in the roadmap as libdbh moves to the GNU distribution procedures,
the next release intends to:
1. Document code fully
2. Incorporate GDBM  to enable variable size keys and optionally match
GDBM performance on small files

Registration Date: Fri 11 Apr 2014 03:37:50 PM UTC
License: GNU General Public License v3 or later
Development Status: 5 - Production/Stable