firstworks Using the SQL Relay drop-in replacement library for MySQL

Using the SQL Relay drop-in replacement library for MySQL

- What is a drop-in replacement library?
- Using the drop-in replacement library with command-line programs
- Using the drop-in replacement library with daemons
- Using the drop-in replacement library with inetd/xinetd helper programs
- Using the drop-in replacement library with PHP
- Using the drop-in replacement library with modules
- Function support

What is a drop-in replacement library?

The SQL Relay drop-in replacement library for MySQL clients allows you to run an application that was written using the native MySQL client API against SQL Relay without rewriting the application.

The drop-in replacement library is a shared object library that implements MySQL client library functions as calls to similar SQL Relay API functions which then map the results back into native MySQL client data structures.

There are a variety of reasons that you might want to use the SQL Relay drop-in replacement library for MySQL clients. An application written for MySQL could, for instance, be made to run queries against an Oracle or Microsoft SQL Server database without modification. Or you could simply put SQL Relay between an application and the MySQL database that it normally runs against to take advantage of SQL Relay's persistence, load balancing or throttling facilities.

Using the drop-in replacement library with command-line programs

To use the SQL Relay drop-in replacement library for MySQL clients, you first need to determine what version of MySQL your existing application was compiled against. There are 4 different drop-in replacement libraries. One each for MySQL versions 3.xx, 4.0, 4.1, 5.0 and 5.1. MySQL's native data structures changed a bit between the different versions. If you use the wrong library your program will most likely crash.

You can find out which library your program uses by running the following command. In this example, the command checks the "mysql" program; the command line client that comes with the MySQL distribution.

>     ldd /usr/bin/mysql

It should return something like the following:

>     libreadline.so.4 => /usr/lib/libreadline.so.4 (0x00625000)
>     libncurses.so.5 => /usr/lib/libncurses.so.5 (0x00976000)
>     libmysqlclient.so.10 => /usr/lib/mysql/libmysqlclient.so.10 (0x009b8000)
>     libz.so.1 => /usr/lib/libz.so.1 (0x00963000)
>     libcrypt.so.1 => /lib/libcrypt.so.1 (0x003dc000)
>     libnsl.so.1 => /lib/libnsl.so.1 (0x003c5000)
>     libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x002ff000)
>     libm.so.6 => /lib/tls/libm.so.6 (0x008ba000)
>     libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x003bb000)
>     libc.so.6 => /lib/tls/libc.so.6 (0x0077f000)
>     libgpm.so.1 => /usr/lib/libgpm.so.1 (0x006d1000)
>     /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00767000)

The line in bold above is the important line.

The 10 at the end of libmysqlclient.so.10 means MySQL protocol version 10. If your application links against version 10 or lower then you can use the MySQL 3.xx version of the drop-in library. If it links against version 11 or 12 then you need to use the MySQL 4.0 version of the drop-in library. If it links against version 13 then you need to use the MySQL 4.1 version of the drop-in library. If it links against version 14 then you may need to use the MySQL 4.1 or 5.0 version of the drop-in library. MySQL 4.1 and 5.0 both use the same protocol version (so far) but there are differences in the data structures, so you'll have to figure out which version it uses some other way.

Once you've figured out which version of MySQL your program was compiled against, you can load the appropriate drop in library and run your program.

The parameters that would ordinarily indicate which host, port, socket, username and password to use to connect to MySQL will be used as parameters to connect to SQL Relay. The parameter that would ordinarily indicate which database to connect to will be ignored. Instances of SQL Relay are configured to connect to a single database, and that database will be used by the client program.

In the following example, we're running the "mysql" program against an instance of SQL Relay running on the localhost, port 8009 against an Oracle database. This instance of SQL Relay is configured with a username/password of oracle8user/oracle8pass.

For sh-based shells:

>     LD_PRELOAD=/usr/local/firstworks/lib/libmysql3sqlrelay.so
>     export LD_PRELOAD
>     mysql -h localhost -P 8009 --user=oracle8user --password=oracle8pass

For csh-based shells:

>     setenv LD_PRELOAD /usr/local/firstworks/lib/libmysql3sqlrelay.so
>     mysql -h localhost -P 8009 --user=oracle8user --password=oracle8pass

The LD_PRELOAD environment variable instructs the dynamic loader to load libmysql3sqlrelay.so before loading any other libraries for any programs. The mysql client program will still load the native MySQL client library, but since it loaded the SQL Relay drop-in replacement library first, function calls that would normally be fulfilled by the native MySQL client library are fulfilled by the SQL Relay drop-in replacement library instead.

If your application was compiled against a different version of MySQL, you will need to replace libmysql3sqlrelay.so with a different library. The following table indicates which library is required for each version of MySQL.

Version of MySQL | Library
--- | ---
3.x | libmysql3sqlrelay.so
4.0.x | libmysql40sqlrelay.so
4.1.x | libmysql41sqlrelay.so
5.0.x | libmysql50sqlrelay.so
5.1.x | libmysql51sqlrelay.so

Below is a sample session using the mysql command line client against an Oracle database through SQL Relay.

>     Welcome to the MySQL monitor.  Commands end with ; or \g.
>     Your MySQL connection id is 0 to server version: 3.23.58

>     Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

>     mysql> create table testtable (col1 varchar2(60), col2 number(5,2));
>     Empty set (0.21 sec)

>     mysql> insert into testtable values ('hello',123.45);
>     Empty set (0.00 sec)

>     mysql> select * from testtable;
>     +-------+--------+
>     | COL1  | COL2   |
>     +-------+--------+
>     | hello | 123.45 |
>     +-------+--------+
>     1 row in set (0.09 sec)

>     mysql> drop table testtable;
>     Empty set (0.21 sec)

>     mysql> quit;

Using the drop-in replacement library with daemons

Using the SQL Relay drop-in replacement library for MySQL with daemons is simlar to using it on the command line. You just need to add the LD_PRELOAD command to the startup script for the daemon before the command that starts the daemon itself.

Unfortunately this approach does not work with PHP. See the section below for how to use the drop-in replacement library with PHP.

Using the drop-in replacement library with inetd/xinetd helper programs

Inetd and xinetd are daemons that listen on ports and run helper programs to service requests on those ports. The helper programs die off after the request is serviced.

The easist way to get an inetd helper program to use the SQL Relay drop-in replacement library for MySQL is to add the LD_PRELOAD command to the startup script for inetd/xinetd. Any command that inetd/xinetd runs will also preload the library.

However, if some of the helper programs need to actually run against MySQL and not against SQL Relay, then you will have to do something different. The easiest thing to do is create a script for each helper program that needs to run against SQL Relay that runs the LD_PRELOAD command and then runs the actual helper program, passing it all the necessary command line arguments.

For example, lets say you have a pop3 server called pop3d that uses MySQL for user identification and you wanted to use SQL Relay instead of MySQL. The inetd.conf entry might look like this:

>     pop3 stream tcp nowait root /usr/local/bin/pop3d

An /etc/xinetd.d entry might look like this:

>     service pop3
>     {
>     	socket_type	= stream
>     	wait		= no
>     	user		= root
>     	server		= /usr/local/bin/pop3d
>     }

You could write the a script called /usr/local/bin/pop3d-sqlrelay as follows:

>     #!/bin/sh
>     LD_PRELOAD=/usr/local/firstworks/lib/libmysql3sqlrelay.so
>     export LD_PRELOAD
>     /usr/local/bin/pop3d $@

And modify the entries to call the script instead of pop3d as follows:

>     pop3 stream tcp nowait root /usr/local/bin/pop3d-sqlrelay

Or for xinetd:

>     service pop3
>     {
>     	socket_type	= stream
>     	wait		= no
>     	user		= root
>     	server		= /usr/local/bin/pop3d-sqlrelay
>     }

Using the drop-in replacement library with PHP

Ideally, if you wanted to use the drop-in replacement library for MySQL with PHP, you'd just add the LD_PRELOAD command to the startup script for the http daemon, but PHP is not typically linked against the MySQL client library, rather the PHP source code includes it's own MySQL client API which is compiled directly into the PHP module itself.

So, to use the drop-in replacement library with PHP, you must rebuild PHP and tell it to link against the MySQL client library.

Follow the instructions at PHP: Installation and Configuration for downloading and extracting the PHP source code. When running the ./configure, script, in addition to whatever other options you use, make sure to use the following option:

> --with-mysql=shared

If you want to use the improved MySQL API, use:

> --with-mysqli=shared

as well. If your mysql_config program is located somewhere other than /usr/bin, then you will need to specify it's path as well. For instance, if it's installed in /usr/local/bin, then you may use the following options:

> --with-mysql=shared,/usr/local/bin/mysql_config --with-mysqli=shared,/usr/local/bin/mysql_config

After the configure script has run, you must edit the Makefile by hand and replace all references to libmysqlclient with references to one of the drop-in replacement libraries. For instance, if libmysqlclient.so is found in /usr/lib/mysql on your system, and you want to use the drop-in replacement library for MySQL version 5.0 then you'll need to make the following replacements:

- replace -Wl,-rpath,/usr/lib/mysql with -Wl,-rpath,/usr/local/firstworks/lib
- replace -L/usr/lib/mysql with -L/usr/local/firstworks/lib
- replace -lmysqlclient with -lmysql50sqlrelay

Afterward you may continue to build and install PHP according to the instructions.

Afterward, you should edit the php.ini file and configure it to load the mysql.so module automatically. When MySQL was compiled into PHP, the MySQL API calls could be made without loading the module, but now that it is not compiled into PHP, the module must be loaded manually. Adding the command to load it automatically in the php.ini file will prevent you from having to go and add dl("mysql.so"); to all of your programs. To automatically load the MySQL module, add a line like the following to php.ini.

> extension=mysql.so

A common thing to forget when configuring php.ini is to set the extension_dir parameter. By default it's usually set to "./" which is wrong. If you installed PHP into /usr/local, then the extension directory will be something like /usr/local/lib/php/extensions/no-debug-non-zts-20060613. On your platform, it may be something else, but similar. Whatever the directory name, it should contain the file mysql.so. Edit php.ini, search for the extension_dir parameter and set it to your extension directory. For example:

> extension_dir=/usr/local/lib/php/extensions/no-debug-non-zts-20060613

After following these steps, your PHP apps which formerly used MySQL directly will now use SQL Relay. You will need to change the connection parameters, to point to SQL Relay , but otherwise it should work.

Using the drop-in replacement library with modules

You may want to use the SQL Relay drop-in replacement library for MySQL clients with a program that isn't compiled against the native MySQL client library but rather loads it as a module such as a program that uses ODBC or Perl DBI, or an Apache/PHP application.

Using the SQL Relay drop-in replacement library with programs that load the native MySQL client library as a module is simlar to using it on the command line. You just need to make sure that the LD_PRELOAD command is run before the program starts.

If the program is a command line program, then run the LD_PRELOAD command before running your program. Even though the program ultimately loads the native MySQL client library, all of its functions will be overriden by the SQL Relay drop-in replacement library.

If the program is a daemon then add the LD_PRELOAD command to the startup script for the daemon.

If the program runs in the address space of a daemon, such as a PHP application running under Apache's mod_php, then add the LD_PRELOAD command to the startup script for the daemon. The caveat here is that all applications running in the address space of the daemon will use the drop-in replacement library instead of the native MySQL library. It is not possible, for example for a web server to run one PHP application directly against MySQL and another PHP application against SQL Relay using the drop-in replacement library; if the drop-in replacement library is loaded, both applications will end up using it.

If the program is spawned by a daemon, such as a cgi spawned by a web-server or an inetd/xinetd helper program, then you can either add the LD_PRELOAD command to the daemon's startup script or write a script to run the LD_PRELOAD command and pass along the command line arguments (see the section Using the drop-in replacement library with inetd/xinetd helper programs above)).

Function support

The SQL Relay drop-in replacement library for MySQL implements most of the native MySQL client library's functions, but there are a few functions that aren't implemented because SQL Relay doesn't have a good way to support them. These functions return safe values or a failure condition.

Here is a list of functions that are implemented and functions that are not. If your application uses one of the functions that is not implemented, you may or may not be able to use it with the SQL Relay drop-in replacement library for MySQL.

Function | Implemented?
--- | ---
mysql_thread_safe | always returns 1
mysql_init | yes
mysql_set_server_option | does nothing but always returns success
mysql_options | does nothing but always returns success
mysql_ssl_set | does nothing but always returns success
mysql_connect | yes
mysql_real_connect | yes
mysql_close | yes
mysql_ping | yes
mysql_stat | returns a string similar to the real mysql_stat but with empty values
mysql_shutdown | always returns CR_UNKNOWN_ERROR
mysql_reload | yes
mysql_refresh | only implemented for refresh_options==REFRESH_GRANT
mysql_thread_id | always returns 0
mysql_list_processes | always returns NULL
mysql_kill | always returns CR_UNKNOWN_ERROR
mysql_get_client_info | returns the version string of the client that the library emulates
mysql_get_client_version | returns the version number of the client that the library emulates
mysql_get_host_info | returns an empty string
mysql_get_proto_info | returns the protocol version number of the client that the libary emulates
mysql_get_server_info | returns the version string of the client that the library emulates
mysql_get_server_version | returns the version number of the client that the libary emulates
mysql_change_user | yes
mysql_character_set_name | always returns latin1
mysql_set_character_set | does nothing, returns success
mysql_debug | does nothing
mysql_dump_debug_info | always returns failure
mysql_create_db | always returns failure
mysql_select_db | always returns failure
mysql_drop_db | always returns failure
mysql_list_dbs | always returns NULL
mysql_list_tables | always returns NULL
mysql_escape_string | yes
mysql_odbc_escape_string | always returns NULL
myodbc_remove_escape | does nothing
mysql_query | yes
mysql_send_query | yes
mysql_read_query_result | yes
mysql_real_escape_string | implemented but only for utf-8 character set
mysql_real_query | yes
mysql_info | always returns an empty string
mysql_insert_id | always returns 0
mysql_store_result | yes
mysql_use_result | yes
mysql_free_result | yes
mysql_more_results | always returns false
mysql_next_result | always returns -1
mysql_list_fields | always returns NULL
mysql_num_fields | yes
mysql_fetch_field | yes
mysql_fetch_fields | yes
mysql_fetch_field_direct | yes
mysql_fetch_lengths | yes
mysql_field_count | yes
mysql_field_seek | yes
mysql_field_tell | yes
mysql_num_rows | yes
mysql_affected_rows | yes
mysql_row_seek | yes
mysql_row_tell | yes
mysql_data_seek | yes
mysql_fetch_row | yes
mysql_eof | yes
mysql_warning_count | always returns 0
mysql_errno | yes
mysql_error | returns the SQL Relay error string
mysql_sqlstate | always returns an empty string
mysql_commit | yes
mysql_rollback | yes
mysql_autocommit | yes
mysql_prepare | yes
mysql_bind_param | yes, for all types except INT24 and GEOMETRY
mysql_bind_result | yes
mysql_execute | yes
mysql_param_count | yes
mysql_param_result | no, the MySQL docs don't even explain what this should do
mysql_fetch | yes
mysql_fetch_column | no, the MySQL docs don't even explain what this should do
mysql_get_metadata | yes
mysql_send_long_data | always returns false
mysql_stmt_num_rows | yes
mysql_stmt_affected_rows | yes
mysql_stmt_row_seek | yes
mysql_stmt_row_tell | yes
mysql_stmt_data_seek | yes
mysql_stmt_close | yes
mysql_stmt_errno | yes
mysql_stmt_error | returns the SQL Relay error string
mysql_stmt_sqlstate | always returns an empty string
mysql_stmt_store_result | yes
mysql_stmt_free_result | yes
mysql_stmt_reset | yes
mysql_server_init | does nothing
mysql_library_init | does nothing
mysql_server_end | does nothing
mysql_library_end | does nothing

The native MySQL client API also defines several data structures. Applications are supposed to access fields of the MYSQL_FIELD and MYSQL_BIND structures directly. Applications also access MYSQL_ROW's directly, but MYSQL_ROW is just a typedef for (char **). The SQL Relay drop-in replacement library for MySQL supports MYSQL_ROW and the following fields of MYSQL_FIELD and MYSQL_BIND. Note that the MYSQL_BIND structure is only available when emulating MySQL 4.1, 5.0 or 5.1 and that the fields of MYSQL_FIELD that are available depend on whether the drop-in library is emulating MySQL 3, 4.0, 4.1, 5.0 or 5.1

MYSQL_FIELD
Field | Supported?
--- | ---
name | yes
org_name | no, is always set to the same value as name
table | no, is always set to an empty string
org_table | no, is always set to an empty string
db | no, is always set to an empty string
catalog | no, is always set to an empty string
def | no, is always set to an empty string
length | yes
max_length | yes
name_length | yes
org_name_length | yes, but is always set to the length of name
table_length | no, is always set to 0
org_table_length | no, is always set to 0
db_length | no, is always set to 0
catalog_length | no, is always set to 0
def_length | no, is always set to 0
flags | yes
decimals | yes
charsetnr | no, is always set to 0
type | yes
MYSQL_BIND
Field | Supported?
--- | ---
length | yes
is_null | yes
buffer | yes
buffer_type | yes
buffer_length | yes
inter_buffer | no
offset | no
internal_length | no
param_number | no
long_data_used | no
binary_data | no
null_field | no
internal_is_null | no
store_param_func | no
fetch_result | no