NONE TITLE 6.html
Keywords: NONE TITLE,fml, flexible mailing list manager, not listserv style but fmlserv emulates listserv or majordomo style

[PREVIOUS CHAPTER] [NEXT CHAPTER]

6 Fml Processes

    It is useful to customize FML if you know the fundamentals of ML
    servers. In this chapter we describe fml internals.

    6.1	Functions as  ML Server

    	mail comes via SMTP.
    		|
    	MTA e.g. sendmail receives it.
    	Here after we suppose MTA is sendmail.
    		|
    	sendmail scans /etc/aliases and runs programs after setuid().
    	sendmail runs and injects an in coming mail into the program.
    		|
    	sendmail -> fml.pl
    	Fml.pl reads and evaluates config.ph and runs.
    	It analyses the header,
    	authenticate the sender is a member or not.
    	Fml distributes a ML article by passing it to MTA.
    		|
    	MTA delivers a passed mail to ML members.

    It is enough to use "sed" for a filter and "sendmail" for delivery.
    If you expect a mailing list driver works for logging, spooling and a
    lot of commands to retrieve articles and so on, you need more
    elaborate programs. Fml provides a lot of functions described below.

    Fml provides, delivery, functions for a lot of commands, digest
    delivery and so on. fml.pl is a delivery program and also provides
    command functions.  Digest delivery (matome okuri) is another process.
    You must set up that cron executes msend.pl (digest delivery system)
    periodically.

    6.2	How "sendmail -> fml.pl" works

    How sendmail passes an in-coming mail to fml.pl? Here it is. Consider
    Elena Mailing list as one example below.

    Firstly sendmail scans /etc/aliases to find

    	Elena: :include:/usr/local/list/Elena
    	owner-Elena:fukachan

    For Elena ML, sendmail executes the content of /usr/local/list/Elena.
    The content of /usr/local/list/Elena is

    	"|/usr/local/fml/fml.pl /var/spool/ml/elena"

    This implies that sendmail passes mail to fml.pl via PIPE (pipe(2)).
    fml.pl receives mail via STDIN. fml.pl works with this input.

    fml.pl recognizes the first argument /var/spool/ml/elena as HOME
    directory of Elena ML. fml.pl reads /var/spool/ml/elena/config.ph and
    sets up Elena ML configurations.

    If you use command line options for fml.pl, please use like this.

    	"|/usr/local/fml/fml.pl /var/spool/ml/elena --ctladdr "

    Arbitrary order is available. fml.pl regards the first argument of
    directory as ML's HOME (e.g. location of config.ph).

    6.3	Order to search library files

    Consider elena ML again. fml.pl is kicked off like

    	/usr/local/fml/fml.pl /var/spool/ml/elena

    fml.pl extracts the directory path (/usr/local/fml) of fml.pl absolute
    path. It searches OS for executable and libraries in the following
    order

    	1   /var/spool/ml/elena
    	2   /usr/local/fml

    If the following arguments are given,

       "|/usr/local/fml/fml.pl /var/spool/ml/elena /usr/lib/uja /lib/aja"

    fml.pl searches OS for executable and libraries like this:

    	1   /var/spool/ml/elena
    	2   /var/spool/ml/etc/fml/
    	3   /usr/local/fml
    	4   /usr/lib/uja
    	5   /lib/uja

    6.4	Priority of configuration file search

    The evaluation order of configuration files is like this.
    "1" is most directive.

    	1 command line options (e.g. fml.pl --ctladdr)
    	2 sitedef.ph
    	3 config.ph
    	3 site_init.ph
    	5 fml defaults

    sitedef.ph and site_init.ph may be useful to be used as a common
    configuration file over mailing lists e.g. to set up command search
    path ,tar, gzip, ...

    fml (after 2.2.1) searchs sitedef.ph and site_init.ph in the following
    order:

    				Example
    	-----------------------------------------------
    	$DIR			/var/spool/ml/elena/
    	$ML_DIR/etc/fml/	/var/spool/ml/etc/fml/
    	$EXEC_DIR		/usr/local/fml/
    	perl standard path	@INC

    It shoule exist in common directory.

    	$ML_DIR/etc/fml/	/var/spool/ml/etc/fml/
    	$EXEC_DIR		/usr/local/fml/

    So it will be either of them. To avoid recnstructuing under
    /usr/local/fml in version up, the first location in fml 2.2.1 search
    path is $ML_DIR/etc/fml/.

    6.5	fml.pl process (first stage)

    fml.pl scans the input mail and splits up it and save them in

    	Header	$Envelope{'Header'}
    	Body	$Envelope{'Body'}

    In default fml.pl is not compatible with hml 1.6 (which is a mailing
    list server may be used only in Japanese). The following descriptions
    are of no means.

    If the mode is compatible with hml 1.6, fml.pl checks the body.  Even
    though fml.pl is not in the compatible mode, the check routine works
    but the result is not used.

    * checks the first $GUIDE_CHECK_LIMIT lines to find $GUIDE_KEYWORD.
      If found, FML sends back guide and do nothing except it.

    * fml.pl scans the first $COMMAND_CHECK_LIMIT to change the mode from
    distribution to command mode. The trap keyword is
    	^#(\s*\w+\s|^\#\s*\w+)
    	^#$CHADDR_KEYWORD

    In default

    	$GUIDE_KEYWORD  = "guide";
    	$CHADDR_KEYWORD = "chaddr|change-address|change";

    6.6	fml process (second stage)

    After parsing, fml.pl sets up %Envelope, locks into critical region.
    Under locked state, fml.pl main part works.

    6.7	fml process, lock and signal handling

    fml.pl uses flock(2) and alarm(3). It is standard technique on
    UNIX. On Micro$oft Windows NT4 with activestate perl, flock(2) works
    but alarm(3) does not work. We emulate it by Win32::Process. Under our
    NT4 alarm emulation, timeout equals to "end of process".

    sub Flock {

        &SetEvent($TimeOut{'flock'} || 3600, 'TimeOut') if $HAS_ALARM;
    	...

        # signal handling
        $SIG{'ALRM'} = 'TimeOut';
        $SIG{'INT'}  = $SIG{'QUIT'} = 'SignalLog';

    Flock timeout is defined by $TimeOut{'flock'}, default is 3600 seconds.

    When timeout occurs, fml.pl notifies it to the maintainer, logs it and
    exits.

    When flock(2) is not used, fml.pl uses link(2) based lock
    algorithm. If timeout, fml.pl notifies it and exits. Please see
    liblock.pl for more details.

    6.8	Event Scheduler

    fml.pl uses flock(2) and alarm(3). It is standard technique on
    UNIX. Software trap by alarm(3) maintains event scheduling inside FML.
    On Micro$oft Windows NT4 with activestate perl, flock(2) works but
    alarm(3) does not work. We emulate it by Win32::Process. The emulation
    is incomplete for the restriction of ntperl. We only enforce fml.pl
    exits if timeout.

    6.9	Another locking algorithm

    fml.pl provides two lock algorithms, flock(2) and link(2) based lock
    algorithm. In default fml.pl uses flock(2). flock(2) is used if

    	$USE_FLOCK    = 1; (default 1)

    link(2) based lock is used if

    	$USE_FLOCK    = 0;

    link(2) style lock is based on whether link(2) works or not.
    The lock file is

    	$LOCK_FILE

    The default lock file is $DIR/var/run/lockfile.v7

    	"$FP_VARRUN_DIR/lockfile.v7"

    This lock algorithm is not fault tolerant e.g. in sudden death of OS
    since the lock file is not removed. After the reboot, maintainer
    should remove left lock files.  Just after reboot, remove all files
    $FP_VARRUN_DIR/lockfile*. From this view, I recommends you that
    you should use the lock mechanism OS provides.

    	"$FP_VARRUN_DIR/lockfile.v7"

    Also in link(2) style, timeout is defined by $MAX_TIMEOUT. The mail is
    saved in var/log. fml.pl notifies it to the maintainer.

    FYI: flock(2)

    	$LOCK_SH                       = 1;
    	$LOCK_EX                       = 2;
    	$LOCK_NB                       = 4;
    	$LOCK_UN                       = 8;

         #include <sys/file.h>
         #define   LOCK_SH   1    /* shared lock */
         #define   LOCK_EX   2    /* exclusive lock */
         #define   LOCK_NB   4    /* don't block when locking */
         #define   LOCK_UN   8    /* unlock */

         int
         flock(int fd, int operation)

    See e.g. A. S. Tannenbaum, "Modern Operating Systems" for exclusive locks.

[PREVIOUS CHAPTER] [NEXT CHAPTER]