1/*  Part of SWI-Prolog
    2
    3    Author:        Jan Wielemaker
    4    E-mail:        J.Wielemaker@vu.nl
    5    WWW:           http://www.swi-prolog.org
    6    Copyright (c)  2015, University of Amsterdam
    7                         VU University Amsterdam
    8    All rights reserved.
    9
   10    Redistribution and use in source and binary forms, with or without
   11    modification, are permitted provided that the following conditions
   12    are met:
   13
   14    1. Redistributions of source code must retain the above copyright
   15       notice, this list of conditions and the following disclaimer.
   16
   17    2. Redistributions in binary form must reproduce the above copyright
   18       notice, this list of conditions and the following disclaimer in
   19       the documentation and/or other materials provided with the
   20       distribution.
   21
   22    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
   23    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
   24    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
   25    FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
   26    COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
   27    INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
   28    BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
   29    LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
   30    CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
   31    LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
   32    ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
   33    POSSIBILITY OF SUCH DAMAGE.
   34*/
   35
   36:- module(bdb,
   37          [ bdb_init/1,                 % +Options
   38            bdb_init/2,                 % -Environment, +Options
   39            bdb_close_environment/1,    % +Environment
   40            bdb_current_environment/1,  % -Environment
   41            bdb_environment_property/2, % ?Environment, ?Property
   42
   43            bdb_open/4,                 % +File, +Mode, -Handle, +Options
   44            bdb_close/1,                % +Handle
   45            bdb_closeall/0,             %
   46            bdb_current/1,              % -DB
   47
   48            bdb_put/3,                  % +DB, +Key, +Value
   49            bdb_del/3,                  % +DB, +Key, ?Value
   50            bdb_delall/3,               % +DB, +Key, +Value
   51            bdb_enum/3,                 % +DB, -Key, -Value
   52            bdb_get/3,                  % +DB, +Key, -Value
   53            bdb_getall/3,               % +DB, +Key, -ValueList
   54
   55            bdb_transaction/1,          % :Goal
   56            bdb_transaction/2,          % :Goal, +Environment
   57
   58            bdb_version/1               % -Version
   59          ]).   60:- use_foreign_library(foreign(bdb4pl)).   61:- meta_predicate
   62    bdb_transaction(0),
   63    bdb_transaction(+, 0).

 bdb_init(+Options) is det

 bdb_init(-Environment, +Options) is det

Initialise a DB environment. The predicate bdb_init/1 initialises the default environment, while bdb_init/2 creates an explicit environment that can be passed to bdb_open/4 using the environment(+Environment) option. If bdb_init/1 is called, it must be called before the first call to bdb_open/4 that uses the default environment. If bdb_init/1 is not called, the default environment can only handle plain files and does not support multiple threads, locking, crash recovery, etc.

Initializing a BDB environment always requires the home(+Dir) option. If the environment contains no databases, the argument create(true) must be supplied as well.

The currently supported options are listed below. The name of the boolean options are derived from the DB flags by dropping the =DB_= prefix and using lowercase, e.g. DB_INIT_LOCK becomes init_lock. For details, please refer to the DB manual.

create(+Bool)

If true, create any underlying file as required. By default, no new files are created. This option should be set for prograns that create new databases.

failchk(+Bool)

home(+Home)

Specify the DB home directory, the directory holding the database files. The directory must exist prior to calling these predicates.

init_lock(+Bool)

Enable locking (DB_INIT_LOCK). Implied if transactions are used.

init_log(+Bool)

Enable logging the DB modifications (DB_INIT_LOG). Logging enables recovery of databases in case of system failure. Normally it is used in combination with transactions.

init_mpool(+Bool)

Initialize memory pool. Impicit if mp_size(+Size) or mp_mmapsize(+Size) is specified.

init_rep(+Bool)

Init database replication. The rest of the replication logic is not yet supported.

init_txn(+Bool)

Init transactions. Implies init_log(true).

lockdown(+Bool)

mp_size(+Integer)

mp_mmapsize(+Integer)

Control memory pool handling (DB_INIT_MPOOL). The mp_size option sets the memory-pool used for caching, while the mp_mmapsize controls the maximum size of a DB file mapped entirely into memory.

private(+Bool)

recover(+Bool)

Perform recovery before opening the database.

recover_fatal(+Bool)

Perform fatal recovery before opening the database.

register(+Bool)

server(+Host,[+ServerOptions])

Initialise the DB package for accessing a remote database. Host specifies the name of the machine running berkeley_db_svc. Optionally additional options may be specified:

server_timeout(+Seconds)

Specify the timeout time the server uses to determine that the client has gone. This implies the server will terminate the connection to this client if this client does not issue any requests for the indicated time.

client_timeout(+Seconds)

Specify the time the client waits for the server to handle a request.

system_mem(+Bool)

transactions(+Bool)

Enable transactions, providing atomicy of changes and security. Implies logging and locking. See bdb_transaction/1.

thread(+Bool)

Make the environment accessible from multiple threads.

thread_count(+Integer)

Declare an approximate number of threads in the database environment. See DB_ENV->set_thread_count().

use_environ(+Bool)

use_environ_root(+Bool)

config(+ListOfConfig)

Specify a list of configuration options, each option is of the form Name(Value). Currently unused.

 bdb_close_environment(+Environment) is det

Close a database environment that was explicitly created using bdb_init/2.

 bdb_current_environment(-Environment) is nondet

True when Environment is a currently known environment.

  201bdb_current_environment(Environment) :-
  202    bdb_current_environment_(Environment),
  203    bdb_is_open_env(Environment).
  204
  205bdb_current_environment_(Env) :-
  206    (   var(Env)
  207    ->  (   Env = default
  208        ;   current_blob(Env, bdb_env)
  209        )
  210    ;   (   Env == default
  211        ->  true
  212        ;   current_blob(Env, bdb_env)
  213        )
  214    ).

 bdb_environment_property(?Environment, ?Property) is nondet

True when Property is a property of Environment. Defined properties are all boolean options defined with bdb_init/2 and the following options:

home(-Path)

Path is the absolute path name for the directory used as database environment.

open(-Boolean)

True if the environment is open.

  228bdb_environment_property(Env, Property) :-
  229    bdb_current_environment_(Env),
  230    (   bdb_is_open_env(Env)
  231    ->  (   var(Property)
  232        ->  env_property(Property),
  233            bdb_env_property_(Env, Property)
  234        ;   bdb_env_property_(Env, Property)
  235        )
  236    ;   Property = open(false)
  237    ).
  238
  239bdb_env_property_(Env, home(Home)) :-
  240    !,
  241    bdb_env_property(Env, home(Home0)),
  242    prolog_to_os_filename(Home, Home0).
  243bdb_env_property_(Env, Prop) :-
  244    bdb_env_property(Env, Prop).
  245
  246env_property(open(true)).
  247env_property(home(_)).
  248env_property(init_lock(_)).
  249env_property(init_log(_)).
  250env_property(init_mpool(_)).
  251env_property(init_rep(_)).
  252env_property(init_txn(_)).
  253env_property(recover(_)).
  254env_property(recover_fatal(_)).
  255env_property(use_environ(_)).
  256env_property(use_environ_root(_)).
  257env_property(create(_)).
  258env_property(lockdown(_)).
  259env_property(failchk(_)).
  260env_property(private(_)).
  261env_property(register(_)).
  262env_property(system_mem(_)).
  263env_property(thread(_)).

 bdb_open(+File, +Mode, -DB, +Options) is det

Open File holding a database. Mode is one of read, providing read-only access or update, providing read/write access. Options is a list of options. Supported options are below. The boolean options are passed as flags to DB->open(). The option name is derived from the flag name by stripping the DB_ prefix and converting to lower case. Consult the Berkeley DB documentation for details.

auto_commit(+Boolean)

Open the database in a transaction. Ensures no database is created in case of failure.

create(+Boolean)

Create a new database of the database does not exist.

dup(+Boolean)

Do/do not allow for duplicate values on the same key. Default is not to allow for duplicates.

excl(+Boolean)

Combined with create(true), fail if the database already exists.

multiversion(+Boolean)

Open the database with support for multiversion concurrency control. The flag is passed, but no further support is provided yet.

nommap(+Boolean)

Do not map this database into process memory.

rdonly(+Boolean)

Open the database for reading only.

read_uncommitted(+Boolean)

Read operations on the database may request the return of modified but not yet committed data. This flag must be specified on all DB handles used to perform dirty reads or database updates, otherwise requests for dirty reads may not be honored and the read may block.

thread(+Boolean)

Enable access to the database handle from multiple threads. This is default if the corresponding flag is specified for the environment.

truncate(+Boolean)

When specified, truncate the underlying file, i.e., start with an empty database.

database(+Name)

If File contains multiple databases, address the named database in the file. A DB file can only consist of multiple databases if the bdb_open/4 call that created it specified this argument. Each database in the file has its own characteristics.

environment(+Environment)

Specify a database environment created using bdb_init/2.

key(+Type)

value(+Type)

Specify the type of the key or value. Allowed values are:

term

Key/Value is a Prolog term (default). This type allows for representing arbitrary Prolog data in both keys and value. The representation is space-efficient, but Prolog specific. See PL_record_external() in the SWI-Prolog Reference Manual for details on the representation. The other representations are more neutral. This implies they are more stable and sharing the DB with other languages is feasible.

atom

Key/Value is an atom. The text is represented as a UTF-8 string and its length.

c_blob

Key/Value is a blob (sequence of bytes). On output, a Prolog string is used. The input is either a Prolog string or an atom holding only characters in the range [0..255].

c_string

Key/Value is an atom. The text is represented as a C 0-terminated UTF-8 string.

c_long

Key/Value is an integer. The value is represented as a native C long in machine byte-order.

Arguments:

DB	- is unified with a blob of type db. Database handles are subject to atom garbage collection.

Errors

- permission_error(access, bdb_environment, Env) if an environment is not thread-enabled and accessed from multiple threads.

 bdb_close(+DB) is det

Close BerkeleyDB database indicated by DB. DB becomes invalid after this operation. An attempt to access a closed database is detected reliably and results in a permission_error exception.

 bdb_put(+DB, +Key, +Value) is det

Add a new key-value pair to the database. If the database does not allow for duplicates the possible previous associated with Key is replaced by Value.

 bdb_del(+DB, ?Key, ?Value) is nondet

Delete the first matching key-value pair from the database. If the database allows for duplicates, this predicate is non-deterministic, otherwise it is semidet. The enumeration performed by this predicate is the same as for bdb_get/3. See also bdb_delall/3.

 bdb_delall(+DB, +Key, ?Value) is det

Delete all matching key-value pairs from the database. With unbound Value the key and all values are removed efficiently.

  375bdb_delall(DB, Key, Value) :-
  376    var(Value),
  377    !,
  378    bdb_del(DB, Key).               % this is much faster
  379bdb_delall(DB, Key, Value) :-
  380    (   bdb_del(DB, Key, Value),
  381        fail
  382    ;   true
  383    ).

 bdb_get(+DB, ?Key, -Value) is nondet

Query the database. If the database allows for duplicates this predicate is non-deterministic, otherwise it is semidet. Note that if Key is a term this matches stored keys that are variants of Key, not unification. See =@=/2. Thus, after bdb_put(DB, f(X), 42), we get the following query results:

• bdb_get(DB, f(Y), V) binds Value to 42, while Y is left unbound.
• bdb_get(DB, f(a), V) fails.
• bdb_enum(DB, f(a), V) succeeds, but does not perform any indexing, i.e., it enumerates all key-value pairs and performs the unification.

 bdb_enum(+DB, -Key, -Value)

Enumerate the whole database, unifying the key-value pairs to Key and Value. Though this predicate can be used with an instantiated Key to enumerate only the keys unifying with Key, no indexing is used by bdb_enum/3.

 bdb_getall(+DB, +Key, -Values) is semidet

Get all values associated with Key. Fails if the key does not exist (as bagof/3).

 bdb_current(?DB) is nondet

True when DB is a handle to a currently open database.

  416bdb_current(DB) :-
  417    current_blob(DB, bdb),
  418    bdb_is_open(DB).

 bdb_closeall is det

Close all currently open databases and environments. This is called automatically after loading this library on process terminatation using at_halt/1.

  426bdb_closeall :-
  427    close_databases,
  428    close_environments.
  429
  430close_databases :-
  431    forall(bdb_current(DB),
  432           catch(bdb_close(DB),
  433                 E,
  434                 print_message(warning, E))).
  435
  436close_environments :-
  437    forall(bdb_current_environment(DB),
  438           catch(bdb_close_environment(DB),
  439                 E,
  440                 print_message(warning, E))).
  441
  442terminate_bdb :-
  443    (   current_predicate(bdb_init/1)       % library was loaded ok
  444    ->  bdb_closeall
  445    ;   true
  446    ).
  447
  448:- at_halt(terminate_bdb).

 bdb_transaction(:Goal) is semidet

 bdb_transaction(+Environment, :Goal) is semidet

Start a transaction, execute Goal and terminate the transaction. Only if Goal succeeds, the transaction is commited. If Goal fails or raises an exception, the transaction is aborted and bdb_transaction/1 either fails or rethrows the exception. Of special interest is the exception

error(package(db, deadlock), _)

This exception indicates a deadlock was raised by one of the DB predicates. Deadlocks may arise if multiple processes or threads access the same keys in a different order. The DB infra-structure causes one of the processes involved in the deadlock to abort its transaction. This process may choose to restart the transaction.

For example, a DB application may define {Goal} to realise transactions and restart these automatically is a deadlock is raised:

{Goal} :-
    catch(bdb_transaction(Goal), E, true),
    (   var(E)
    ->  true
    ;   E = error(package(db, deadlock), _)
    ->  {Goal}
    ;   throw(E)
    ).

Arguments:

Environment

- defines the environment to which the transaction applies. If omitted, the default environment is used. See bdb_init/1 and bdb_init/2.

 bdb_version(-Version:integer) is det

True when Version identifies the database version. Version is an integer defined as:

DB_VERSION_MAJOR*10000 +
DB_VERSION_MINOR*100   +
DB_VERSION_PATCH

  501                 /*******************************
  502                 *             MESSAGES         *
  503                 *******************************/
  504
  505:- multifile
  506    prolog:message/3.  507
  508prolog:message(error(bdb(Code, Message, Obj), _)) -->
  509    [ 'BDB: Error ~w on ~p: ~w'-[Code, Obj, Message] ]