firstworks SQL Relay Configuration Guide

SQL Relay Configuration Guide

- Minimal Configuration
- Database Connections
- Database Cursors
- Dynamic Scaling
- Listener Configuration
- Database Configuration
- Replicated or Clustered Databases
- Master-Slave Query Routing
- Session-Queries
- Multiple Instances
- Multiple Files
- Starting Instances Automatically
- Advanced Configuration

Minimal Configuration

The SQL Relay configuration file is usually located at /usr/local/firstworks/etc/sqlrelay.conf

The most minimal configuration would be something like:

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="example">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="user=scott;password=tiger;oracle_sid=orcl"/>
> </connections>
> </instance>
> </instances>

This configuration defines an instance of SQL Relay named example, secured by a user and password, that opens and maintains 5 persistent connections to the orcl instance of an Oracle database using scott/tiger credentials.

The instance can be started using:

> `sqlr-start -id example`

By default, SQL Relay listens on all available network interfaces, on port 9000. It can be accessed remotely by hostname. For example, if the server running SQL Relay is named sqlrserver then it can be accessed using:

> `sqlrsh -host sqlrserver -user sqlruser -password sqlrpassword`

It can also be accessed from the local server using localhost.

> `sqlrsh -host localhost -user sqlruser -password sqlrpassword`

(NOTE: For versions of SQL Relay prior to 0.56, additional parameters were required for a minimal configuration. In particular, either the port or socket parameter was required in the instance tag, the dbtype parameter was required in the instance tag, and connectionid and metric parameters were required in the connection tag. See below, or see the Configuration Reference for information about these parameters.)

Database Connections

By default, SQL Relay opens and maintains 5 persistent database connections, but the number of connections can be configured using the connections parameter.

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="example" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="user=scott;password=tiger;oracle_sid=orcl"/>
> </connections>
> </instance>
> </instances>

The number of connections determines how many client applications can access the database simultaneously . In this case, up to 10, assuming each client only needs one connection. Additional clients can connect but get queued up and have to wait for one of the first 10 to disconnect before being able to access the database.

Any number of connections can be opened. A good rule of thumb is to open as many as you can. The optimal number is application specific though and is limited by database, system and network resources.

Database Cursors

Database cursors are used to execute queries and step through result sets. Most applications only need to use one cursor at a time. Some apps require more though, either because they run nested queries, or sometimes because they just don't properly free them.

SQL Relay maintains persistent cursors as well as connections. By default, each connection opens one cursor, but the number of cursors opened by each connection can be configured using the cursors parameter.

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="example" connections="10" cursors="2">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="user=scott;password=tiger;oracle_sid=orcl"/>
> </connections>
> </instance>
> </instances>

Any number of cursors can be opened. A good rule of thumb is to open as few as possible but as many as you know that your application will need.

Dynamic Scaling

Both connections and cursors can be configured to scale dynamically - open on demand and then die off when no longer needed. This feature is useful if you have spikes in traffic during certain times of day or if your application has a few modules that occasionally need more cursors than usual.

The maxconnections and maxcursors parameters define the upper bounds.

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="example" connections="10" maxconnections="20" cursors="2" maxcursors="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="user=scott;password=tiger;oracle_sid=orcl"/>
> </connections>
> </instance>
> </instances>

In this example, 10 connections will be started initially but more will be be started as necessary, up to 20. Each of the newly spawned connections will die off if they are inactive for longer than 1 minute.

In this example, each connection will initially open 2 cursors but more will be opened as necessary, up to 10. Each newly opened cursor will be closed as soon as it is no longer needed.

Other parameters that control dynamic scaling behavior include:

- maxqueuelength
- growby
- ttl
- cursors_growby

See the SQL Relay Configuration Reference for more information on these parameters.

Listener Configuration

By default, SQL Relay listens for client connections on port 9000, on all available network interfaces.

It can be configured to listen on a different port though...

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="example" port="9001" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="user=scott;password=tiger;oracle_sid=orcl"/>
> </connections>
> </instance>
> </instances>

...and accessed using:

> `sqlrsh -host sqlrserver -port 9001 -user sqlruser -password sqlrpassword`

It can also be configured to listen on a unix socket...

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="example" socket="/tmp/example.socket" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="user=scott;password=tiger;oracle_sid=orcl"/>
> </connections>
> </instance>
> </instances>

...and accessed from the local server using:

> `sqlrsh -socket /tmp/example.socket -user sqlruser -password sqlrpassword`

If the server has multiple network interfaces, SQL Relay can also be configured to listen on specific IP addresses.

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="example" addresses="192.168.1.50,192.168.123.51" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="user=scott;password=tiger;oracle_sid=orcl"/>
> </connections>
> </instance>
> </instances>

When configured this way, it can be accessed on 192.168.1.50 and 192.168.1.51 but not on 127.0.0.1 (localhost).

If the socket option is specified but port and addresses options are not, then SQL Relay will only listen on the socket. If addresses/port and socket options are both specified then it listens on both.

Database Configuration

By default, SQL Relay assumes that it's connecting to an Oracle database, but SQL Relay supports many other databases. The dbase parameter can be used to specify the database type.

In the following example, SQL Relay is configured to connect to a SAP/Sybase database.

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="example" dbase="sybase" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="sybase=/opt/sybase;lang=en_US;server=TESTDB;user=testuser;password=testpassword;db=testdb"/>
> </connections>
> </instance>
> </instances>

In this example, SQL Relay is configured to connect to an IBM DB2 database.

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="example" dbase="db2" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="db=testdb;user=db2inst1;password=db2inst1pass;lang=C;timeout=0"/>
> </connections>
> </instance>
> </instances>

When connecting to an Oracle database, the dbase parameter may be omitted or set to "oracle8". The 8 is for historical reasons. SQL Relay originally supported Oracle OCI 7 and 8 and the dbase parameter could be set to "oracle7" or "oracle8". Eventually "oracle7" was dropped and "oracle8" was updated to support all future versions of OCI but the "oracle8" label remained.

SQL Relay supports many different databases and the connect string options (options in the string attribute of the connection tag) are different for each database. See the SQL Relay Configuration Reference for valid dbase settings and information on connect string options for each database.

Replicated or Clustered Databases

SQL Relay can be configured to distribute connections over the nodes of a database cluster or pool of replicated servers. Each connection tag defines a node to connect to.

In this example, SQL Relay is configured to distribute over two instances of Oracle - orcl1 and orcl2

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="example" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="user=scott;password=tiger;oracle_sid=orcl1" metric="10"/>
> <connection string="user=scott;password=tiger;oracle_sid=orcl2" metric="15"/>
> </connections>
> </instance>
> </instances>

Any number of connection tags may be defined.

The metric parameter is especially important if the nodes are heterogenous (ie. if some nodes are more powerful than others). SQL Relay uses it to decide how many connections to open to each node. It doesn't specify the number of connections to open to each node, but the higher the metric relative to the other metrics, the more connections to that node will be started. For example, if the metric for the first node is twice as large as the metric for the second node, then SQL Relay will open twice as many connections to the first node as the second.

In this example, 15 is 1.5 times 10, so 1.5 as many connections will be started to orcl2 as to orcl1. Since a total of 10 connections will be started, 4 will be started to orcl1 and 6 to orcl2.

A similar configuration could be used to distribute connections over the nodes of a cluster.

When the nodes are located behind a load balancer or share a common IP address, only one connection tag should be used, with the behindloadbalancer attribute.

In this example, SQL Relay is configured to distribute over an Oracle RAC.

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="example" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="user=scott;password=tiger;oracle_sid=orclrac" behindloadbalancer="yes"/>
> </connections>
> </instance>
> </instances>

See the SQL Relay Configuration Reference for more information on this parameter.

Master-Slave Query Routing

In a master-slave replication scenario, SQL Relay can be configured to route DML and DDL queries to the master and distribute selects over the slaves.

This actually requires 3 instances of SQL Relay. One to connect to the master, one to connect to the slaves, and a third to route queries to the first two.

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <!-- This instance maintains connections to the "master" MySQL database
> on the masterdb machine. This instance only listens on the
> unix socket /tmp/master.socket and thus cannot be connected to
> by clients from another machine. -->
> <instance id="master" socket="/tmp/master.socket" dbase="mysql" connections="10">
> <users>
> <user user="masteruser" password="masterpassword"/>
> </users>
> <connections>
> <connection string="user=masteruser;password=masterpassword;host=masterdb;db=master;"/>
> </connections>
> </instance>
> <!-- This instance maintains connections to 4 "slave" MySQL databases
> on 4 slave machines. This instance only listens on the unix
> socket /tmp/slave.socket and thus cannot be connected to by
> clients from another machine. -->
> <instance id="slave" socket="/tmp/slave.socket" dbase="mysql" connections="10">
> <users>
> <user user="slaveuser" password="slavepassword"/>
> </users>
> <connections>
> <connection string="user=slaveuser;password=slavepassword;host=slavedb1;db=slave;"/>
> <connection string="user=slaveuser;password=slavepassword;host=slavedb2;db=slave;"/>
> <connection string="user=slaveuser;password=slavepassword;host=slavedb3;db=slave;"/>
> <connection string="user=slaveuser;password=slavepassword;host=slavedb3;db=slave;"/>
> </connections>
> </instance>
> <!-- This instance sends DML (insert,update,delete) and
> DDL (create/delete) queries to the "master" SQL Relay instance
> which, in turn, sends them to the "master" database.
> This instance sends any other queries to the "slave" SQL Relay
> instance which, in turn, distributes them over the "slave"
> databases. -->
> <instance id="router" dbase="router" connections="10">
> <users>
> <user user="routeruser" password="routerpassword"/>
> </users>
> <router>
> <!-- send all DML/DDL queries to "master" -->
> <route host="" port="" socket="/tmp/master.socket"
> user="masteruser" password="masterpassword">
> <query pattern="^drop "/>
> <query pattern="^create "/>
> <query pattern="^insert "/>
> <query pattern="^update "/>
> <query pattern="^delete "/>
> </route>
> <!-- send all other queries to "slave" -->
> <route host="" port="" socket="/tmp/slave.socket"
> user="slaveuser" password="slavepassword">
> <query pattern=".*"/>
> </route>
> </router>
> </instance>
> </instances>

The first two instances use familiar configuration options, but the third uses a dbtype of "router" and rather than defining connections, uses router, route and query tags to define query routing rules.

The router tag defines a set of routes. The attributes of each route tag specify an instance of SQL Relay to connect to. The pattern attribute of each query tag defines a regular expression used to match queries. Queries that match the set of patterns defined in the route tag are sent to the instance of SQL Relay designated in the route tag.

The three instances can be started using:

> `sqlr-start -id master
> sqlr-start -id slave
> sqlr-start -id router`

Client applications should connect to the router instance rather than the master or slave instances.

> `sqlrsh -host sqlrserver -user routeruser -password routerpassword`

See the Query Routing and Filtering for more information on query routing.

Session-Queries

SQL Relay can be configured to run a set of queries at the beginning and end of each client session.

By far the most common use for this feature is that some database parameter needs to be reconfigured but you don't have permission or bouncing the database is out of the question, or something like that. For example, lets say you are using an Oracle database, but your app requires dates to be formatted like MM/DD/YYYY instead of DD-MON-YYYY. Ideally you'd alter the nls_date_format in the instance but you can't, for some reason.

You can use SQL Relay's session queries to work around the problem.

In this example, the date format is set to MM/DD/YYYY at the beginning of the session and then reset back to DD-MON-YYYY at the end.

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="example" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <session>
> <start>
> <runquery>alter session set nls_date_format='MM/DD/YYYY'</runquery>
> </start>
> <end>
> <runquery>alter session set nls_date_format='DD-MON-YYYY'</runquery>
> </end>
> </session>
> <connections>
> <connection string="user=scott;password=tiger;oracle_sid=orcl"/>
> </connections>
> </instance>
> </instances>

Actually, in this example, there's no need to set the date format back to DD-MON-YYYY but it's done here for illustrative purposes.

Multiple Instances

Any number of SQL Relay instances can be defined in the configuration file.

In following example, instances that connect to Oracle, SAP/Sybase and DB2 are defined in the same file.

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="oracleexample" port="9000" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="user=scott;password=tiger;oracle_sid=orcl"/>
> </connections>
> </instance>
> <instance id="sybaseexample" dbase="sybase" port="9001" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="sybase=/opt/sybase;lang=en_US;server=TESTDB;user=testuser;password=testpassword;db=testdb"/>
> </connections>
> </instance>
> <instance id="db2example" dbase="db2" port="9002" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="db=testdb;user=db2inst1;password=db2inst1pass;lang=C;timeout=0"/>
> </connections>
> </instance>
> </instances>

These instances can be started using:

> `sqlr-start -id oracleexample
> sqlr-start -id sybaseexample
> sqlr-start -id db2example`

...and accessed using:

> `sqlrsh -host sqlrserver -port 9000
> sqlrsh -host sqlrserver -port 9001
> sqlrsh -host sqlrserver -port 9002
> `

Multiple Files

While any number of SQL Relay instances can be defined in a single configuration file, it might be more convenient to split configurations up into multiple files.

The default the SQL Relay configuration directory is /usr/local/firstworks/etc/sqlrelay.conf.d

Additional configuration files may be created under this directory. These files must follow the same format as the main configuration file.

For example, if you wanted to split up oracle, sybase and db2 configurations into 3 separate files, you could create:

/usr/local/firstworks/etc/sqlrelay.conf.d/oracle.conf

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="oracleexample" port="9000" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="user=scott;password=tiger;oracle_sid=orcl"/>
> </connections>
> </instance>
> </instances>

/usr/local/firstworks/etc/sqlrelay.conf.d/sybase.conf

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="sybaseexample" dbase="sybase" port="9001" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="sybase=/opt/sybase;lang=en_US;server=TESTDB;user=testuser;password=testpassword;db=testdb"/>
> </connections>
> </instance>
> </instances>

/usr/local/firstworks/etc/sqlrelay.conf.d/db2.conf

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="db2example" dbase="db2" port="9002" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="db=testdb;user=db2inst1;password=db2inst1pass;lang=C;timeout=0"/>
> </connections>
> </instance>
> </instances>

Starting Instances Automatically

In all previous examples sqlr-start has been called with the -id option, specifying which instance to start. If sqlr-start is called without the -id option then it will start all intances configured with the enabled attribute set to yes.

For example, if the following instances are defined...

> <?xml version="1.0"?>
> <!DOCTYPE instances SYSTEM "sqlrelay.dtd">
> <instances>
> <instance id="oracleexample" enabled="yes" port="9000" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="user=scott;password=tiger;oracle_sid=orcl"/>
> </connections>
> </instance>
> <instance id="sybaseexample" enabled="yes" dbase="sybase" port="9001" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="sybase=/opt/sybase;lang=en_US;server=TESTDB;user=testuser;password=testpassword;db=testdb"/>
> </connections>
> </instance>
> <instance id="db2example" dbase="db2" port="9002" connections="10">
> <users>
> <user user="sqlruser" password="sqlrpassword"/>
> </users>
> <connections>
> <connection string="db=testdb;user=db2inst1;password=db2inst1pass;lang=C;timeout=0"/>
> </connections>
> </instance>
> </instances>

...then calling sqlr-start without the -id parameter will start oracleexample and sybaseexample because enabled="yes" is configured for those instances. db2example will not be started because enabled="yes" is not configured for that instance.

When installed on most platforms, SQL Relay creates an init script in the appropriate place under /etc. This init script calls sqlr-start with no -id option. If configured to run at boot, it will start all instances for which enabled="yes" is configured.

How to enable the init script depends on what platform you are using.

On Linux, Solaris, SCO and other non-BSD Unixes, this usually involves creating a symlink from /etc/init.d/sqlrelay to /etc/rc3.d/S85sqlrelay.

Redhat-derived Linux distributions have a chkconfig command that can do this for you. Debian-derived Linux distributions provide the update-rc.d command. Older versions of Solaris provide svcadm. Newer versions of Solaris provide smf.

On FreeBSD you must add a line to /etc/rc.conf like:

> sqlrelay_enabled=YES

On NetBSD you must add a line to /etc/rc.conf like:

> sqlrelay=YES

On OpenBSD you must add a line to /etc/rc.conf like:

> sqlrelay_flags=""

Advanced Configuration

The configuration file supports many more attributes and features than the ones described in this guide including tuning options and extension modules. See the SQL Relay Configuration Reference and Tuning SQL Relay for more information.