Sympa::Message(3Sympa)

NAME
Sympa::Message - Mail message embedding for internal use in Sympa

SYNOPSYS
  use Sympa::Message;
  my $message = Sympa::Message->new($serialized, context => $list);

DESCRIPTION
While processing a message in Sympa, we need to link information to the message, modify headers and such. This was quite a problem when a message was signed, as modifying anything in the message body would alter its MD5 footprint. And probably make the message to be rejected by clients verifying its identity (which is somehow a good thing as it is the reason why people use MD5 after all). With such messages, the process was complex. We then decided to embed any message treated in a "Message" object, thus making the process easier.

  Methods and functions

new ( $serialized, context => $that, KEY => value, ... )
    Constructor. Creates a new Sympa::Message object.

    Parameters:

    $serialized
        Serialized message.

    context => object
        Context. Sympa::List object, Robot or '*'.

    key => value, ...
        Metadata.

    Returns:

    A new Sympa::Message object, or undef, if something went wrong.

new_from_template ( $that, $filename, $rcpt, $data )
    Constructor. Creates Sympa::Message object from template.

    Parameters:

    $that
        Content: Sympa::List, robot or '*'.

    $filename
        Template filename (without extension).

    $rcpt
        Scalar or arrayref: SMTP "RCPT TO:" field.

        If it is a scalar, trys to retrieve information of the user (See also Sympa::User.

    $data
        Hashref used to parse template file, with keys:

        return_path
            SMTP "MAIL FROM:" field if sent by SMTP (see Sympa::Mailer), "Return-Path:" field if sent by spool.

            Note: This parameter is obsoleted. Currently, {envelope_sender} attribute of object is taken from the context.

        to  "To:" header field

        lang
            Language tag used for parsing template. See also Sympa::Language.

        from
            "From:" field if not a full msg

            Note: This parameter is obsoleted. The "From:" field will be filled in by "sympa" address if it is not found.

        subject
            "Subject:" field if not a full msg

        replyto
            "Reply-To:" field if not a full msg

        body
            Body message if $filename is ''.

            Note: This feature has been deprecated.

        headers
            Additional headers, hashref with keys are field names.

    Returns:

    New Sympa::Message instance, or "undef" if something went wrong.

dup ( )
    Copy constructor. Gets deep copy of instance.

to_string ( [ original => 0|1 ] )
    Serializer. Returns serialized data of Message object.

    Parameter:

    original => 0|1
        If set to 1 and content has been decrypted, returns original content. Default is 0.

    Returns:

    Serialized representation of Message object.

add_header ( $field, $value, [ $index ] )
    Instance method. Adds a header field named $field with body $value. If $index is given, the field will be inserted at the place it indicates: If it is 0, the field will be prepended.

delete_header ( $field, [ $index ] )
    Instance method. Deletes all occurences of the header field named $field.

replace_header ( $field, $value, [ $index ] )
    Instance method. Replaces header fields named $field with $value.

head
    Instance method. Gets header of the message as MIME::Head instance.

    Note that returned value is real reference to internal data structure. Even if it was changed, string representaion of message may not be updated. Alternatively, use "add_header"(), "delete_header"() or "replace_header"() to modify header.

check_spam_status ( )
    Instance method. Gets spam status according to spam_status scenario and sets it as {smap_status} attribute.

dkim_sign ( dkim_d => $d, [ dkim_i => $i ], dkim_selector => $selector, dkim_privatekey => $privatekey )
    Instance method. Adds DKIM signature to the message.

check_dkim_signature ( )
    Instance method. Checks DKIM signature of the message and sets or clears {dkim_pass} item of the message object.

remove_invalid_dkim_signature ( )
    Instance method. Verifys DKIM signatures included in the message, and if any of them are invalid, removes them.

as_entity ( )
    Instance method. Gets message content as MIME entity (MIME::Entity instance).

    Note that returned value is real reference to internal data structure. Even if it was changed, string representaion of message may not be updated. Below is better way to modify message.

        my $entity = $message->as_entity->dup;
        # ... Modify $entity...
        $message->set_entity($entity);

set_entity ( $entity )
    Instance method. Updates message with MIME entity (MIME::Entity instance). String representation will be automatically updated.

as_string ( )
    Instance method. Gets a string representation of message.

    Parameter:

    original => 0|1
        If set to 1 and content has been decrypted, returns original content. Default is 0.

    Note that method like "set_string()" does not exist: You would be better to create new instance rather than replacing entire content.

body_as_string ( )
    Instance method. Gets body of the message as string.

    Note that the result won't be decoded.

header_as_string ( )
    Instance method. Gets header part of the message as string.

    Note that the result won't be decoded nor unfolded.

get_header ( $field, [ $sep ] )
    Instance method. Gets value(s) of header field $field, stripping trailing newline.

    In scalar context without $sep, returns first occurrence or "undef". If $sep is defined, returns all occurrences joined by it, or "undef". Otherwise in array context, returns an array of all occurrences or "()".

    Note: Folding newlines will not be removed.

get_decoded_header ( $tag, [ $sep ] )
    Instance method. Returns header value decoded to UTF-8 or undef. Trailing newline will be removed. If $sep is given, returns all occurrences joined by it.

dump ( $output )
    Instance method. Dumps a Message object to a stream.

    Parameters:

    $output
        the stream to which dump the object

    Returns:

    1.  if everything's alright

add_topic ( $output )
    Instance method. Adds topic and puts header X-Sympa-Topic.

    Parameters:

    $output
        the string containing the topic to add

    Returns:

    1.  if everything's alright

get_topic ( )
    Instance method. Gets topic of message.

    Parameters:

    None.

    Returns:

    the topic
        if it exists

    empty string
        otherwise

clean_html ( )
    Instance method. Encodes HTML parts of the message by UTF-8 and strips scripts included in them.

smime_decrypt ( )
    Instance method. Decrypts message using private key of user.

    Note that this method modifys Message object.

    Parameters:

    None.

    Returns:

    True value if message was decrypted. Otherwise false value.

    If decrypting succeeded, {smime_crypted} item is set.

smime_encrypt ( $email )
    Instance method. Encrypts message using certificate of user.

    Note that this method modifys Message object.

    Parameters:

    $email
        E-mail address of user.

    Returns:

    True value if encryption succeeded, or "undef".

smime_sign ( )
    Instance method. Adds S/MIME signature to the message.

    Signing key is taken from what stored in list directory.

    Parameters:

    None.

    Returns:

    True value if message was successfully signed. Otherwise false value.

check_smime_signature ( )
    Instance method. Verifys S/MIME signature of the message, and if verification succeeded, sets {smime_signed} item true.

    Parameters:

    None

    Returns:

    1 if signature is successfully verified. 0 otherwise. "undef" if something went wrong.

personalize ( $list, [ $rcpt ], [ $data ] )
    Instance method. Personalizes a message with custom attributes of a user.

    Parameters:

    $list
        List object.

    $rcpt
        Recipient.

    $data
        Hashref. Additional data to be interpolated into personalized message.

    Returns:

    Modified message itself, or "undef" if error occurred.

test_personalize ( $list )
    Instance method. Tests if personalization can be performed successfully over all subscribers of list.

    Parameters:

    Returns:

    1 if succeed, or "undef".

personalize_text ( $body, $list, [ $rcpt ], [ $data ] )
    Function. Retrieves the customized data of the users then parses the text. It returns the personalized text.

    Parameters:

    $body
        Message body with the TT2.

    $list
        List object.

    $rcpt
        The recipient email.

    $data
        Hashref. Additional data to be interpolated into personalized message.

    Returns:

    Customized text, or "undef" if error occurred.

prepare_message_according_to_mode ( $mode, $list )
    Instance method. Transforms the message according to reception mode: 'mail', 'notice', 'txt' or 'html'.

    By 'nomail', 'digest', 'digestplain' or 'summary' mode, the message is not modified.

decorate ( )
    Instance method. Adds footer/header to a message.

reformat_utf8_message ( )
    Instance method. Reformats bodies of text parts contained in the message using recommended encoding schema and/or charsets defined by MIME::Charset.

    MIME-compliant headers are appended / modified. And custom X-Mailer: header is appended :).

    Parameters:

    $attachments
        ref(ARRAY) - messages to be attached as subparts.

    Returns:

    string

get_plain_body ( )
    Instance method. Gets decoded content of text/plain part.

    The text will be converted to UTF-8. Flowed text (see RFC 3676) will be conjuncted.

check_virus_infection ()
    Instance method. Checks the message using anti-virus plugin, if configuration requests it.

    Returns:

    The name of malware the message contains, if any; "unknown" for unidentified malware; "undef" if checking failed; otherwise 0.

get_plaindigest_body ( )
    Instance method. Returns a plain text version of message, suitable for use in plain text digests.

    •   Most attachments are stripped out and replaced with a note that they've been stripped. text/plain parts are retained.

    •   An attempt to convert text/html parts to plain text is made if there is no text/plain alternative.

    •   All messages are converted from their original character set to UTF-8.

    •   Parts of type message/rfc822 are recursed through in the same way, with brief headers included.

    •   Any line consisting only of 30 hyphens has the first character changed to space (see RFC 1153). Lines are wrapped at 76 columns.

    Parameters:

    None.

    Returns:

    String.

dmarc_protect ( )
    Instance method. Munges the "From:" header field if we are using DMARC Protection mode.

    Parameters:

    None.

    Returns:

    None. "From:" field of the message may be modified.

get_id ( )
    Instance method. Gets unique identifier of instance.

  Context and Metadata
Context and metadata given to constructor are accessible as hash elements of object. These are typically used.

{context}
    Context of the message, Sympa::List object, robot or '*'.

{date}
    The UNIX time messages was initially accepted, or the time message should be delivered.

{domainpart}
{listname}
{listtype}
{localpart}
    Domain, name, type and local part of context.

{priority}
    Priority of the message.

{tag}
    Tag of packet used by bulk spool to control logging. '0' is the first message of multiple packet. 'z' is the last. 's' is the single message with single packet.

{time}
    The Unix time in floating point number when the message was stored into the spool. This is used by bulk spool.

  Attributes
These are accessible as hash elements of objects.

{checksum}
    No longer used. It is kept for compatibility with Sympa 6.1.x or earlier. See also upgrade_send_spool(1).

{envelope_sender}
    Envelope sender, a.k.a. "Unix From". This is not always same as {sender} attribute nor the content of "From:" field.

    '<>' will be used for "null envelope sender".

{family}
    Name of family (see Sympa::Family) the message corresponds to. This is given by familyqueue(8) program.

{gecos}
    Display name of actual sender (see {sender} below), if any.

{md5_check}
    True value indicates that the message has been authenticated by "md5" level (password authentication). This is set by web mailer of WWSympa and used by incoming spool.

{message_id}
    Original message ID of the message.

{rcpt}
    Recipients for delivery. This is kept for compatibility with earlier releases.

{sender}
    Actual sender of the message. This is determined according to "sender_headers" configuration parameter. See also {envelope_sender} above.

{shelved}
    Shelved processing. Hashref with multiple items. Currently these items are available:

    dkim_sign => 1
        Adding DKIM signature.

    dmarc_protect => 1
        DMARC protection. See also "dmarc_protect"().

    merge => 1
        Personalizing.

    smime_encrypt => 1
        Adding S/MIME encryption.

    smime_sign => 1
        Adding S/MIME signature.

    tracking => "dsn"|"mdn"|"r"|"w"|"verp"
        Requesting tracking feature including VERP.

    This is used by bulk spool.

{spam_status}
    Result of spam check. This is set by "check_spam_status"() method.

  Serialization
Sympa::Message object includes number of slots as hash items: metadata, context, attributes and message content.

Metadata is given by spool. On spool based on filesystem, it is typically encoded into file names. For example a file name

  listname-owner@domain.name.143599229.12345

encodes the metadata

  localpart  => 'listname-owner',
  listname   => 'listname',
  listtype   => 'return_path',
  domainpart => 'domain.name',
  date       => 143599229,

Metadata always includes information of context: List, Robot or Site. For example:

- Incoming post bound for <listname@domain.name>:

  context    => Sympa::List <listname@domain.name>,

- Incoming command message bound for <sympa@domain.name>:

  context    => 'domain.name',

- Message sent from Sympa to super-listmaster(s):

  context    => '*'

Context is determined when the object is instantiated, and never changed through its lifetime. Thus, constructor of objects should take context object as an argument.

Logically, objects are stored into physical spool as serialized form and deserialized when they are fetched from spool. Attributes will be serialized and deserialized along with raw message content. Attributes are encoded in "X-Sympa-*:" pseudo-header fields and "Return-Path:" header field. Below is an example of serialized form.

  X-Sympa-Message-ID: 123456789.12345@domain.name : {message_id} attribute
  X-Sympa-Sender: user01@user.sympa.test          : {sender} attribute
  X-Sympa-Display-Name: Infant                    : {gecos} attribute
  X-Sympa-Shelved: dkim_sign; tracking=mdn        : {shelved} attribute
  X-Sympa-Spam-Status: ham                        : {spam_status} attribute
  Return-Path: sympa-request@domain.name          : {envelope_sender} attribute
  Message-Id: <123456789.12345@domain.name>       :   ---
  From: Infant <user@other.host.dom>              :    |
  To: User <user@some.host.name>                  :    |
  Subject: Howdy world                            :    | Raw message content
  X-Sympa-Topic: sometopic                        :    |
                                                  :    |
  Bonjour, le monde.                              :    |
                                                  :   ---

On msg, automatic and bounce spools, "Return-Path:" header fields are given by MDA and "X-Sympa-*:" header fields are given by queue programs. On other spools, they are given by components of Sympa.

Pseudo-header fields should appear at beginning of serialized content. Fields appear at other places (e.g. "X-Sympa-Topic:" field above) are not attributes but are the part of raw message content.

Pseudo-header fields should not be included in actually sent messages.

CAVEAT

  Adding "Return-Path:" field
We trust in "Return-Path:" header field only at the top of message to prevent forgery. To ensure it will be added to messages by MDA,

Sendmail
    Add "P" in the "F=" flags of local mailer line (such as "Mlocal").

Postfix
    local(8)
        Prepending "Return-Path:" is available by default.

    pipe(8)
        Add "R" to the "flags=" attributes in master.cf.

Exim
    Set "return_path_add" to be true with pipe_transport.

qmail
    Use preline(1).

BUGS
get_plaindigest_body() seems to ignore any text after a UUencoded attachment.

HISTORY
Message module appeared on Sympa 3.3.6. It was initially written by:

•   Serge Aumont <sa AT cru.fr>

•   Olivier Salauen <os AT cru.fr>

get_plaindigest_body, ex. "plain_body_as_string" in PlainDigest, was initially written by Chris Hastie. It appeared on Sympa 4.2b.1.

  (c) Chris Hastie 2004 - 2008.

Renamed and merged Sympa::Message appeard on Sympa 6.2.