DYSTOPIA(3)

NAME
dystopia - the core API

DESCRIPTION

Indexed database is a directory containing a hash database file and its index files. The key of each record is a positive number. The value of each record is an arbitrary text data whose encoding is UTF-8. See `dystopia.h' for entire specification.

To use the core API, include `dystopia.h' and related standard header files. Usually, write the following description near the front of a source file.

    #include <dystopia.h>
    #include <stdlib.h>
    #include <stdbool.h>
    #include <stdint.h>

Objects whose type is pointer to `TCIDB' are used to handle indexed databases. A remote database object is created with the function `tcidbnew' and is deleted with the function `tcidbdel'. To avoid memory leak, it is important to delete every object when it is no longer in use.

Before operations to store or retrieve records, it is necessary to open a database directory and connect the indexed database object to it. The function `tcidbopen' is used to open a database directory and the function `tcidbclose' is used to close the database directory. To avoid data missing or corruption, it is important to close every database directory when it is no longer in use.

API

The function `tcidberrmsg' is used in order to get the message string corresponding to an error code.

    const char *tcidberrmsg(int ecode);
        `ecode' specifies the error code.
        The return value is the message string of the error code.

The function `tcidbnew' is used in order to create an indexed database object.

    TCIDB *tcidbnew(void);
        The return value is the new indexed database object.

The function `tcidbdel' is used in order to delete an indexed database object.

    void tcidbdel(TCIDB *idb);
        `idb' specifies the indexed database object.
        If the database is not closed, it is closed implicitly. Note that the deleted object and its derivatives can not be used anymore.

The function `tcidbecode' is used in order to get the last happened error code of an indexed database object.

    int tcidbecode(TCIDB *idb);
        `idb' specifies the indexed database object.
        The return value is the last happened error code.
        The following error code is defined: `TCESUCCESS' for success, `TCETHREAD' for threading error, `TCEINVALID' for invalid operation, `TCENOFILE' for file not found, `TCENOPERM' for no permission, `TCEMETA' for invalid meta data, `TCERHEAD' for invalid record header, `TCEOPEN' for open error, `TCECLOSE' for close error, `TCETRUNC' for trunc error, `TCESYNC' for sync error, `TCESTAT' for stat error, `TCESEEK' for seek error, `TCEREAD' for read error, `TCEWRITE' for write error, `TCEMMAP' for mmap error, `TCELOCK' for lock error, `TCEUNLINK' for unlink error, `TCERENAME' for rename error, `TCEMKDIR' for mkdir error, `TCERMDIR' for rmdir error, `TCEKEEP' for existing record, `TCENOREC' for no record found, and `TCEMISC' for miscellaneous error.

The function `tcidbtune' is used in order to set the tuning parameters of an indexed database object.

    bool tcidbtune(TCIDB *idb, int64_t ernum, int64_t etnum, int64_t iusiz, uint8_t opts);
        `idb' specifies the indexed database object which is not opened.
        `ernum' specifies the expected number of records to be stored. If it is not more than 0, the default value is specified. The default value is 1000000.
        `etnum' specifies the expected number of tokens to be stored. If it is not more than 0, the default value is specified. The default value is 1000000.
        `iusiz' specifies the unit size of each index file. If it is not more than 0, the default value is specified. The default value is 536870912.
        `opts' specifies options by bitwise or: `IDBTLARGE' specifies that the size of the database can be larger than 2GB by using 64-bit bucket array, `IDBTDEFLATE' specifies that each page is compressed with Deflate encoding, `IDBTBZIP' specifies that each page is compressed with BZIP2 encoding, `IDBTTCBS' specifies that each page is compressed with TCBS encoding.
        If successful, the return value is true, else, it is false.
        Note that the tuning parameters should be set before the database is opened.

The function `tcidbsetcache' is used in order to set the caching parameters of an indexed database object.

    bool tcidbsetcache(TCIDB *idb, int64_t icsiz, int32_t lcnum);
        `idb' specifies the indexed database object which is not opened.
        `icsiz' specifies the capacity size of the token cache. If it is not more than 0, the default value is specified. The default value is 134217728.
        `lcnum' specifies the maximum number of cached leaf nodes of B+ tree. If it is not more than 0, the default value is specified. The default value is 64 for writer or 1024 for reader.
        If successful, the return value is true, else, it is false.
        Note that the caching parameters should be set before the database is opened.

The function `tcidbsetfwmmax' is used in order to set the maximum number of forward matching expansion of an indexed database object.

    bool tcidbsetfwmmax(TCIDB *idb, uint32_t fwmmax);
        `idb' specifies the indexed database object.
        `fwmmax' specifies the maximum number of forward matching expansion.
        If successful, the return value is true, else, it is false.
        Note that the matching parameters should be set before the database is opened.

The function `tcidbopen' is used in order to open an indexed database object.

    bool tcidbopen(TCIDB *idb, const char *path, int omode);
        `idb' specifies the indexed database object.
        `path' specifies the path of the database directory.
        `omode' specifies the connection mode: `IDBOWRITER' as a writer, `IDBOREADER' as a reader. If the mode is `IDBOWRITER', the following may be added by bitwise or: `IDBOCREAT', which means it creates a new database if not exist, `IDBOTRUNC', which means it creates a new database regardless if one exists. Both of `IDBOREADER' and `IDBOWRITER' can be added to by bitwise or: `IDBONOLCK', which means it opens the database directory without file locking, or `IDBOLCKNB', which means locking is performed without blocking.
        If successful, the return value is true, else, it is false.

The function `tcidbclose' is used in order to close an indexed database object.

    bool tcidbclose(TCIDB *idb);
        `idb' specifies the indexed database object.
        If successful, the return value is true, else, it is false.
        Update of a database is assured to be written when the database is closed. If a writer opens a database but does not close it appropriately, the database will be broken.

The function `tcidbput' is used in order to store a record into an indexed database object.

    bool tcidbput(TCIDB *idb, int64_t id, const char *text);
        `idb' specifies the indexed database object connected as a writer.
        `id' specifies the ID number of the record. It should be positive.
        `text' specifies the string of the record, whose encoding should be UTF-8.
        If successful, the return value is true, else, it is false.

The function `tcidbout' is used in order to remove a record of an indexed database object.

    bool tcidbout(TCIDB *idb, int64_t id);
        `idb' specifies the indexed database object connected as a writer.
        `id' specifies the ID number of the record. It should be positive.
        If successful, the return value is true, else, it is false.

The function `tcidbget' is used in order to retrieve a record of an indexed database object.

    char *tcidbget(TCIDB *idb, int64_t id);
        `idb' specifies the indexed database object connected as a writer.
        `id' specifies the ID number of the record. It should be positive.
        If successful, the return value is the string of the corresponding record, else, it is `NULL'.
        Because the region of the return value is allocated with the `malloc' call, it should be released with the `free' call when it is no longer in use.

The function `tcidbsearch' is used in order to search an indexed database.

    uint64_t *tcidbsearch(TCIDB *idb, const char *word, int smode, int *np);
        `idb' specifies the indexed database object.
        `word' specifies the string of the word to be matched to.
        `smode' specifies the matching mode: `IDBSSUBSTR' as substring matching, `IDBSPREFIX' as prefix matching, `IDBSSUFFIX' as suffix matching, `IDBSFULL' as full matching, `IDBSTOKEN' as token matching, `IDBSTOKPRE' as token prefix matching, or `IDBSTOKSUF' as token suffix matching.
        `np' specifies the pointer to the variable into which the number of elements of the return value is assigned.
        If successful, the return value is the pointer to an array of ID numbers of the corresponding records. `NULL' is returned on failure.
        Because the region of the return value is allocated with the `malloc' call, it should be released with the `free' call when it is no longer in use.

The function `tcidbsearch2' is used in order to search an indexed database with a compound expression.

    uint64_t *tcidbsearch2(TCIDB *idb, const char *expr, int *np);
        `idb' specifies the indexed database object.
        `expr' specifies the string of the compound expression.
        `np' specifies the pointer to the variable into which the number of elements of the return value is assigned.
        If successful, the return value is the pointer to an array of ID numbers of the corresponding records. `NULL' is returned on failure.
        Because the region of the return value is allocated with the `malloc' call, it should be released with the `free' call when it is no longer in use.

The function `tcidbiterinit' is used in order to initialize the iterator of an indexed database object.

    bool tcidbiterinit(TCIDB *idb);
        `idb' specifies the indexed database object.
        If successful, the return value is true, else, it is false.
        The iterator is used in order to access the ID number of every record stored in a database.

The function `tcidbiternext' is used in order to get the next ID number of the iterator of an indexed database object.

    uint64_t tcidbiternext(TCIDB *idb);
        `idb' specifies the indexed database object.
        If successful, the return value is the ID number of the next record, else, it is 0. 0 is returned when no record is to be get out of the iterator.
        It is possible to access every record by iteration of calling this function. It is allowed to update or remove records whose keys are fetched while the iteration. However, it is not assured if updating the database is occurred while the iteration. Besides, the order of this traversal access method is arbitrary, so it is not assured that the order of storing matches the one of the traversal access.

The function `tcidbsync' is used in order to synchronize updated contents of an indexed database object with the files and the device.

    bool tcidbsync(TCIDB *idb);
        `idb' specifies the indexed database object connected as a writer.
        If successful, the return value is true, else, it is false.
        This function is useful when another process connects the same database directory.

The function `tcidboptimize' is used in order to optimize the files of an indexed database object.

    bool tcidboptimize(TCIDB *idb);
        `idb' specifies the indexed database object connected as a writer.
        If successful, the return value is true, else, it is false.
        This function is useful to reduce the size of the database files with data fragmentation by successive updating.

The function `tcidbvanish' is used in order to remove all records of an indexed database object.

    bool tcidbvanish(TCIDB *idb);
        `idb' specifies the indexed database object connected as a writer.
        If successful, the return value is true, else, it is false.

The function `tcidbcopy' is used in order to copy the database directory of an indexed database object.

    bool tcidbcopy(TCIDB *idb, const char *path);
        `idb' specifies the indexed database object.
        `path' specifies the path of the destination directory. If it begins with `@', the trailing substring is executed as a command line.
        If successful, the return value is true, else, it is false. False is returned if the executed command returns non-zero code.
        The database directory is assured to be kept synchronized and not modified while the copying or executing operation is in progress. So, this function is useful to create a backup directory of the database directory.

The function `tcidbpath' is used in order to get the directory path of an indexed database object.

    const char *tcidbpath(TCIDB *idb);
        `idb' specifies the indexed database object.
        The return value is the path of the database directory or `NULL' if the object does not connect to any database directory.

The function `tcidbrnum' is used in order to get the number of records of an indexed database object.

    uint64_t tcidbrnum(TCIDB *idb);
        `idb' specifies the indexed database object.
        The return value is the number of records or 0 if the object does not connect to any database directory.

The function `tcidbfsiz' is used in order to get the total size of the database files of an indexed database object.

    uint64_t tcidbfsiz(TCIDB *idb);
        `idb' specifies the indexed database object.
        The return value is the size of the database files or 0 if the object does not connect to any database directory.

COMPOUND EXPRESSION OF SEARCH

The function `tcidbsearch2' searches with a compound expression. In the compound expression, tokens are separated by one or more white space characters. If one token is specified, records including the specified pattern are searched for. Upper or lower case is not distinguished. Accent marks and diacritical marks are ignored. If two or more tokens are specified, records including all of the patterns are searched for. The compound expression includes the following sub expressions.

    A B : searches for records including the two tokens.
    A && B : searches for records including the two tokens.
    A || B : searches for records including the one or both of the two tokens.
    "A B..." : searches for records including the phrase.
    [[A]] : searches for records including words exactly matching the token.
    [[A*]] : searches for records including words beginning with the token.
    [[*A]] : searches for records including words ending with the token.
    [[[[A : searches for records beginning with the token.
    A]]]] : searches for records ending with the token.

Note that the priority of "||" is higher than the one of "&&".

SEE ALSO

dysttest(1), dystmgr(1), tcqdb(3), laputa(3)