GNU libdbh - Summary
This software is part of the GNU Project.
Library to create and manage 64-bit disk based hash tables residing
on disk. Associations are made between keys and
values so that for a given a key the value can be
found and loaded into memory quickly. Being disk based
allows for large and persistent hashes. 64 bit support
allows for hashtables with sizes over 4 Gigabytes on 32
bit systems. Quantified key generation allows for
minimum access time on balanced multidimensional trees.
Libdbh is similar to GDBM, but different in basic design. While GDBM
uses quite a lot of memory when working with large files, libdbh use
of memory is tiny. While GDBM outperforms libdbh in tables with a
small number of records, libdbh outperforms GDBM when a large number
of records is involved (full performance tests are under way). Other
similarities and differences include:
Similarities:
- Both use a unique key to access a particular data element
- Both have variable size data records
- Both have functions to search, retrieve, modify, erase records
- Both have sequential iteration over all key/data pairs
Differences:
- GDBM also has variable size keys
- GDBM works with a memory hashtable to a flat data table file, with
records cached in virtual memory.
- LIBDBH is a structured multidimensional table file which takes
advantage of the built in cache mechanisms built into disk devices and
the OS file cache.
- GDBM is limited to < 2GB records, LIBDBH does not have this limit.
- GDBM reads everything into its cache, which efectively limits the
size of the data table to the available virtual memory before swapping
begins and performance deteriorates, the table size for LIBDBH does not
have this limitation.
- LIBDBH has threadsafe mode (for concurrent operation between threads)
and parallel safe mode (for concurrent operation between independent
processes).
- GDBM has several cool things, which can easily be incorporated into
LIBDBH, such as the definition of a user defined fatal function, and
import/export functionality.
- GDBM is two dimensional. There is only one way to go through the
table, from first to end. LIBDBH is multidimensional, you can go through
the table by dimensions (sweep) or across dimensions (fanout), and
each data record is a complete table in itself, so you can do
selective foreach procedures.
And in the roadmap as libdbh moves to the GNU distribution procedures,
the next release intends to:
1. Document code fully
2. Incorporate GDBM to enable variable size keys and optionally match
GDBM performance on small files
Registration Date: Fri 11 Apr 2014 03:37:50 PM UTC
License: GNU General Public License v3 or later
Development Status: 5 - Production/Stable
- Browse open items
Submit a new item
Powered by Savane 3.14-f13d.
Corresponding source code