Qore Programming Language Reference Manual: Module Description

Qore Programming Language Reference Manual 0.8.11

- Main Page
- Related Pages
- Modules
- Namespaces
- Classes

All Classes Namespaces Functions Variables Groups Pages
Module Description

# Module Overview

Qore modules allow the Qore language to be extended at run-time.

To load a module at parse time (normally required for most modules), use the %requires or %try-module parse directive. If the named feature is not already present in Qore, Qore looks for a module with this name in the directories listed in the `QORE_MODULE_DIR` environment variable (see Environment Variables).

Use the load_module() function to load Qore modules at run-time; however, note that any module providing parse support (classes, constants, functions, etc) must be loaded at parse time using the %requires or %try-module parse directive.

From Qore 0.7.1 onwards, you can specify a comparison operator (one of < <=, =, >=, or >) and version information after the module name as well. Version numbers are compared via integer comparisons of each element, where elements are separated by a '.'. If one of the versions does not have as many elements as another, the missing elements are assumed to be '0' (i.e. version `"1.0"` compared with version `"1.0.1"` will be extended to `"1.0.0"`).

Also note that DBI drivers are loaded on demand by the Qore::SQL::Datasource and Qore::SQL::DatasourcePool classes.

There are two types of modules: Binary Modules and User Modules.

At the time of writing this documentation, the following modules exist for Qore:

Modules Provided With Qore

Type | Module | Description
user | CsvUtil | Provides code to help parse CSV or other structured text files and also to easily generate such files
user | HttpServer | Provides a multi-threaded HTTP server and request handler APIs; provides the infrastructure for server-side HTTP services
user | MailMessage | Provides supporting classes for the Pop3Client and SmtpClient modules; mail message serialization and deserialization, attachment handling
user | Mapper | Provides data mapping classes and interators
user | Mime | Provides MIME definitions and functions; MIME type lookups, MIME encoding and decoding functions, MIME multipart handling, etc
user | MysqlSqlUtil | Provides a high-level DB-independent API for working with MySQL database objects; loaded automatically by the SqlUtil module when working with MySQL databases for automated schema management, programmatic DB access, schema and data synchronization, and more
user | OracleSqlUtil | Provides a high-level DB-independent API for working with Oracle database objects; loaded automatically by the SqlUtil module when working with Oracle databases for automated schema management, programmatic DB access, schema and data synchronization, and more
user | Pop3Client | Provides POP3 client functionality; provides an API to retrieve email messages from a POP3 server
user | PgsqlSqlUtil | Provides a high-level DB-independent API for working with PostgreSQL database objects; loaded automatically by the SqlUtil module when working with PostgreSQL databases for automated schema management, programmatic DB access, schema and data synchronization, and more
user | Qorize | Provides basic support for automatically-generating Qore code from data
user | RestClient | Provides a simple API for communicating with HTTP servers implementing REST services
user | RestHandler | Provides an easy to use interface to the Qore HttpServer module for implementing server-side REST services
user | Schema | Provides automatic schema management functionality as a meta-layer for SqlUtil's medium and low-level schema management functionality
user | SmtpClient | Provides SMTP client functionality; provides an API for sending emails via an SMTP server
user | SqlUtil | Provides a high-level DB-independent API for working with databases; for automated schema management, programmatic DB access, schema and data synchronization, and more
user | TableMapper | Provides a data mapping functionality and iterator support using SqlUtil and Mapper to map arbitrary data to an SQL table target
user | TelnetClient | Provides Telnet client functionality
user | Util | Provides a some miscellaneous generally useful routines; often used by other user modules for example
user | WebSocketClient | Provides an event-driven client API for connecting to WebSocket servers
user | WebSocketHandler | Provides an interface to the Qore HttpServer module for implementing server-side WebSocket services
user | WebSocketUtil | Provides common client and server code for implementing WebSocket protocol services in Qore
user | WebUtil | Provides higher-level classes for implementing more complex web services in Qore; works with the HttpServer module

Modules Provided Separately

Type | Module | Description
binary | freetds | Provides a FreeTDS-based DBI driver for communicating with Sybase and MS SQL Server databases
binary | fsevent | Provides an event-driven filesystem event API
binary | glut | Provides GLUT functionality
binary | json | Provides JSON and JSON-RPC client functionality, also provides the following user modules: - JsonRpcHandler: provides infrastructure for implementing JSON-RPC server-side services using the HttpServer module
binary | linenoise | Provides a readline-like API to Qore under a permissive license
binary | mysql | Provides a MySQL / MariaDB / Percona DBI driver
binary | oracle | Provides an Oracle DBI driver, providing many advanced features such as support for named types and collections, advanced queuing, etc, also provides the following user modules: - OracleExtensions: provides infrastructure for SQL tracing
binary | opengl | Provides an OpenGL API to Qore
binary | openldap | Provides an OpenLDAP API to Qore
binary | pgsql | Provides a PostgreSQL DBI driver
binary | sqlite3 | Provides an SQLite3 DBI driver
binary | ssh2 | Provides SSH2 and SFTP functionality, also provides the following user module: - SftpPoller: provides event-driven support for polling an SFTP server for new data
binary | sybase | Provides a Sybase DBI driver
binary | sysconf | Provides sysconf, pathconf, and confstr APIs
binary | tibae | Provides TIBCO ActiveEnterprise(TM) (TIBCO, Inc) functionality
binary | tibrv | Provides TIBCO Rendezvous(R) (TIBCO, Inc) functionality
binary | uuid | Provices an API for generating UUIDs
binary | xml | Provides XML (SAX and DOM parsers), XPath, XML-RPC, SOAP client and server, etc functionality, also provides the following user modules: - SoapClient: provides an easy to use API for making requests to SOAP servers - SoapHandler: provides infrastructure for implementing SOAP server-side services using the HttpServer module - WSDL: provides underlying web service/WSDL support for the SoapClient and SoapHandler modules - XmlRpcHandler: provides infrastructure for implementing XML-RPC server-side services using the HttpServer module
binary | xmlsec | Provides xmldsig and xmlenc functionality
binary | yaml | Provides YAML functionality, also provides the following user modules: - DataStreamClient: provides a DataStream client API extending the RestClient - DataStreamRequestHandler: provides a DataStream server-side handler API extending the RestHandler - DataStreamUtil: provides underlying DataStream client and server protocol support - YamlRpcClient: provides an API for easily making YAML-RPC calls over the network - YamlRpcHandler: provides infrastructure for implementing YAML-RPC server-side services using the HttpServer module

Other Modules Provided Separately

Type | Module | Description
binary | asn1 | Provides some ASN.1 functionality; stable but only partial ASN.1 functionality is provided, uses very old APIs
binary | db2 | Provides an IBM DB2 driver; stuck in protype phase due to lack of development
binary | ncurses | Provides curses APIs; stable and works, but uses old APIs, needs updating
binary | qt4 | Provides Nokia (formerly Trolltech) QT4 APIs for GUI development; this module works for a subset of the QT API, but due to the need for complete object lifecycle control, it's not recommended for use until we can implement a new garbage collector for Qore objects created from external modules
binary | tuxedo | Provides Oracle (ex Bea) Tuxedo functionality; this module has not been updated for some time, uses old APIs

# Binary Modules

Binary modules are written in C++ and delivered in binary form. They must conform to the Qore Module API and have the file extension `"*.qmod"`. Binary modules normally depend on other shared libraries and therefore can only be loaded if the libraries they require are present on the system and can be found in the library path.

Binary modules are merged into Program objects in the same way as the static system namespace objects are imported. It is possible to import a binary module and be able to use only part of its functionality, for example, if the importing Program cannot access the filesystem, and the module has functions that access the filesystem, only the functions that access the filesystem will not be available. In user modules, the functional domain is set on the module level and not on the individual function or method level as with builtin objects, so user modules are either completely imported or not at all.

Please note that as of version 0.7.0 onwards, the source code for binary modules has been split from the main Qore library source code into separate projects; see Qore's home page for more information.

# User Modules

User modules are written in Qore and delivered in source form. They must have the extension `"*.qm"`.

User modules will have their own dedicated Program object; the Program object is created automatically when the module is loaded and initialized. The module's Program object is created with locked parse options as follows:

- PO_NO_TOP_LEVEL_STATEMENTS: modules provide API services and are not designed to be executed at the top-level. Put any initialization code in the module's `init` closure
- PO_REQUIRE_PROTOTYPES: to ensure that module's APIs are transparent, all method and function declarations must include parameter and return types
- PO_REQUIRE_OUR: this parse option is meant to ensure that typos in variable names are caught at parse time and to ensure transparency of variable types in the module's source code
- PO_IN_MODULE: this parse option is set so that module code is recognized by the system when parsing modules

When parsing module code, the default warning mask is set to WARN_MODULES, and any warnings are treated as errors. Furthermore, any restrictions that the importing Program object has will be added to the module's Program object.

Any namespaces, classes, constants, functions, and global variables declared as public will be exported into the importing Program object; all other declarations will be private to the module's Program object. Note that the root namespace of a module Program is always public by default.

Note that global variables exported from a module's Program object are exported as references; each global variable declared in a module is unique, and they reside in the module's Program object.

Furthermore, when using an environment with multiple Program objects, if a user module has already been loaded and initialized, then it's functional domain mask is compared against any importing Program object's restrictions; if the module uses functionality that is not allowed in the importing Program object, then an exception is raised and the module is not imported.

Also note that the Qore::Program::constructor() applies a mask to the parse option mask option when Program objects are created in a user module; in this case the parse options passed to the child Program object are masked with the current parse options in the user module, and they are locked so that they cannot be made less restrictive. This is to prevent user modules from circumventing functional restrictions imposed by parse options.

## User Module Declarations

User modules are declared with a special syntax in Qore:

User Module Declaration Syntax
`module` name `{`
`version = "`version string`";`
`desc = "`description string`";`
`author = "`author string`";`
`[url = "`URL string`";]`
`[license = "`license string`";]`
`[init =` initialization closure`;]`
`[del =` deletion closure`;]`
`}`

Module properties are as follows:

- `version:` (required) must be assigned to a string giving the version of the module
- `desc:` (required) must be assigned to a string giving a text description of the module
- `author:` (required) must be assigned to a string giving the module's author's name
- `url:` (optional) if present, must be a string giving the URL of the module
- `license:` (optional) if present, must be a string giving the license of the module
- `init:` (optional) if present, must be a closure that will be executed when the module is loaded; this can be used to initialize the module, for example
- `del:` (optional) if present, must be a closure that will be executed when the module is unloaded; this can be used to stop running services or free resources managed and still allocated by the module, for example

Note
  - any unhandled exceptions in the `init` closure will cause the module to fail to load
  - unhandled exceptions in the `init` and `del` closures are displayed on stdout

## The "public" Keyword

Only objects defined with the `public` keyword are made available in Program objects importing the user module. All other declarations and definitions are private to the module.

The `public` keyword also affects inheritance in child Program objects as well as controlling exported symbols from user modules.

The `public` keyword must be used with the following declarations in the module to export them:

- namespaces: namespaces themselves must be declared `public` in order for any of their contents also to be exported (it is an error to try to declare public members of a module-private namespace). ex:
  public namespace MyNamespace { ... }
- classes: classes not declared `public` will not be exported; there is no way to export part of a class; either the entire class is exported or it is not. ex:
  public MyNamespace {
  public MyClass { ... }
  }
- constants: constants must be declared `public` to be exported; ex:
  public namespace MyNamespace {
  public const MyConst = 100;
  }
- functions: function variants must be declared `public` in order to be exported. ex:
  public MyNamespace {
  public int sub my_func() { ... }
  }
- global variables: only global variables declared `public` will be exported. ex:
  public namespace MyNamespace {
  public our int $OurInt;
  }

Note
  Global variable declarations in a namespace declaration cannot be initialized at the point they are declared, also, since Qore::PO_NO_TOP_LEVEL_STATEMENTS is set for user module Program objects, global variables also cannot be initialized at the top-level. Use the `init` closure to initialize the module and any global variables requiring initialization.

User modules are only imported into a Program if the importing Program can use all of the capabilities used in the user module. It is not possible to partially import a user module (in contrast with binary modules, which can be imported even if they expose functionality not allowed in the importing Program, however that functionality will not be available in that case). User module Program objects have a functional domain attribute set on the Program level, so either a user module is imported in its entirety or not at all.

## Module Example

Here is an example declaring user module `"foo"` version `"1.0"`:

module foo {
version = "1.0";
desc = "test module";
author = "Foobar Industries, inc";
url = "http://example.com";
license = "MIT";
init = sub () {
Foo::initialized = True;
Bar::OurBool = False;
};
del = sub () {
print("goodbye, cruel world!\n");
}
}
# do not use "$", assume local scope for variables
%new-style
# nothing in namespace ::Foo is exported
namespace Foo {
# inline global variable declarations cannot be initialized when declared
our bool initialized;
class SuperClass {
}
class NotSoGreatClass {
}
class UnstableClass {
}
}
# public members of namespace ::Bar are exported
public namespace Bar {
# Bar::SomeClass is exported
public class SomeClass {
}
# Bar::add(int, int) is exported
public int sub add(int x, int y) { return x + y; }
# Bar::OurBool is exported
public our bool OurBool;
# Bar::PrivateClass is not exported
class PrivateClass {
}
}

Since
  Qore 0.8.4 user modules are supported

## Program Scope in Object-Oriented Programs Using User Modules Providing Their Own Threads

When using the %exec-class parse directive, the application object will go out of scope as soon as the constructor terminates (unless there are valid scope-extending references to the application object, such as making an assignment of `$self` to a global variable in the contructor).

Therefore when using a module that provides services in its own threads (such as, for example, the HttpServer module), it's important to make sure that the application object does not go out of scope while non-static method call references to the application object are needed by active threads inthe user module (for example, non-static method call references passed as callbacks to the HttpServer module, etc).

This also applies to call references to non-static methods passed to set_signal_handler().

If a module thread tries to use a callback or closure in a Program that has already gone out of scope, an `OBJECT-ALREADY-DELETED` exception (`"attempt to access member of an already-deleted object"`) is thrown.

Therefore in such cases it's best to wait for all threads in any modules to terminate before allowing the application object's constructor to terminate.

For an example of this, see the example program `examples/httpserver.q` where the main Program calls HttpServer::waitStop() before exiting the application object's constructor for this reason.