Log2Timeline(3)

NAME
Log2Timeline - The main engine of log2timeline and the API to interface with.

DESCRIPTION
This is the main engine of the tool log2timeline. This file or engine serves as the communicator between different parts of the tool. This is the API that the front-end talks to, and the engine that iniates both the input and output modules as well as to control the flow of them.

So this is the bread and butter of log2timeline so to speak and the library that can be imported into any tool that wishes to implement a front-end for the tool. And this documentation should serve as a guideline into how to use the API. If the intention is to develop a new front-end or a tool that interacts with the engine, either consult this manual, the tool's wiki ( https://code.google.com/p/log2timeline/) or examine the example front-end found inside the dev/ folder from the source tarball.

SYNOPSIS
  use constant FALSE => 0;
  use constant TRUE => 1;

  # create a new instance of log2timeline
  my $l = Log2Timeline->new(
    'file' => '.',
    'recursive' => FALSE,
    'input' => 'all',
    'output' => 'csv',
    'time_zone' => 'local',
    'offset' => FALSE,
    'exclusions' => '',
    'text' => '',
    'debug' => FALSE,
    'digest' => FALSE,
    'quick' => FALSE,
    'raw' => FALSE,
    'hostname' => '',
    'preprocess' => 0,
  );

  # check if there is a new version available
  print $l->check_upgrade;

  # get the current version number of the tool
  print $l->version;

  # get the help text from an input module
  print $l->get_help_in( 'recycler' );

  # get the help text from an output module
  print $l->get_help_out( 'csv' );

  # change some of the tools settings
  $l->set( 'recursive' => 'yes' );

  # get a list of all the available input modules and lists
  $l->get_inputs;

  # get a list of all the available output modules
  $l->get_outputs;

  # start parsing through the files, gathering timestamps
  $l->start;

  # get a list of all the available timezones
  $l->get_timezone_list;

  # get the currently set timezone
  $l->get_timezone;

METHODS
This documentation contains a list and description of both public and private methods. All private methods start with an underscore character (_) and should not be used or called by front-ends or other tools interacting with the API.

All other methods (excluding private) are considered to be part of the API and can be used or called by various front-ends.

  "new"
The constructor, a very simple one, just returns the value of the secondary constructor. When a new Log2Timeline object is created it can be created without any parameters, the tool will simply accept them all as the default value.

IF however, there is a need to overwrite some of the default behavior of the tool, such as to instruct it to parse another file than the current directory using the local timezone of the machine, etc. then there are two options. Either to use the parameters to the constructor to define the options or to use the set sub routine to change a value of a parameter.

Since this constructor only calls a secondary one, please refer to the description of the "_new" to get a more detailed description of what is done in this phase.

Args:

A hash that defines the parameters and their values. A key and a value, where the key is the name of the variable needed to be changed from the default one and the value is the new value of that particular variable.

Returns:

An instance of this module.

  "_new"
The constructor of the tool does not really do anything except to call the private constructor (this sub routine) and return the value that this sub routine returns.

This routine takes the hash value that is passed to the constructor as a parameter and sets up all the values of the needed variables in the tool. There are some values that need to be set for the tool to properly operate, such as the path to the file/directory that needs to be parsed, the name(s)/list of input modules to load up, time zone of the image and the output, etc.

This routine takes care of assigning values to each of these variables. It compares the hash that is sent to the routine as a parameter and checks if it recognizes the variable. If it does it will assign the value that is passed to it, otherwise it will assign it to the default value that is hardcoded into this sub routine. This means that no values need to be sent to the engine in order for it to work, it is only necessary to define those that need to be changed from the default values (listed below).

The variables that the sub routine recognizes are (default values inside brackets []):

file:
    The path to the directory/file to be parsed/examined.

recursive:
    Boolean value (0/1) that indicates if we should use recursive through a mount point/directory [0/FALSE]

input:
    String, containg a list of all input modules (comma separated) that should be loaded. Names can be either a name of a module or a list file, and it can also be negated with a - sign, indicating that module should be omitted from being loaded. [all]

output:
    String containing the name of the output module used for output. [csv]

time_zone:
    String containing the time zone of the image/file that needs to be parsed. The string can be of any value that the DateTime library supports with the addition of 'local' and 'list'. 'local' will use the local timezone of the computer the tool is run from and 'list' will make the engine build a list of all available time zones and print them out. [local]

out_time_zone:
    String containing the time zone that is printed in the output. If the investigator would like all the output in the same time zone, irrelevant of the input time zone then that can be defined here. [defaults to the same value as time_zone]

offset:
    The time on any given computer can be vastly different from a correct clock, which is essential to correct if the offset to the real clock is known and timestamps from more than one system are being correlated. This option provides a way to do that. This is a string value or an integer value. If it is an int, then it represents the number of seconds the clock differs (can be prepended with a - sign indicating a negative difference). It can also be a string of the form (regular expression) "^-?\d+[hms]?$", whereas 1h means exactly one hour difference, (h = hour, m = minute, s = second).

exclusions:
    A string containg a list of exclusions (comma separated). Sometimes the tool does fail (ohh yes that has actually happened) and fixing that bug is not trivial/done in time/no time to wait. Or that you simply do not want to include certain files in the timeline then this list can be used. It is a comma separated list of strings that are used in regular expressions for exclusions (so do not put something like 'a' in there since that will exclude all files that have the character a somewhere in the path). [empty]

text:
    A string that will be prepended to every path in the output. If the tool is run with the text variable set to 'C:' that text will be prepended to every path printed out in the tool. [empty]

temp:
    A string that contains the path to a temporary directory. The tool sometimes needs to write files to a temporary directory, this occurs for instance when dealing with locked SQLite databases and possible other scenarios. Therefore the tool needs ready access to a temporary directory where it can write data. Different OS's have their default directories, such as the /tmp one in *NIX. The tool does attempt to detect this directory, but for various reasons it may be desired to overwrite the location of it. ['']

debug:
    An integer indicating the debug level of the tool. There are currently three level observed:

    0 = no debug

    1 = debug information turned on.

    2 = excessive debug information turned on.

digest:
    A boolean (0/1) that indicates whether or not we should calculate a MD5 hash for every file to include as an attribute. N.b. this increases the time it takes the tool to complete by considerable amount. [FALSE]

quick:
    Boolean value (0/1). One of the bottlenecks of this tool are the verification of each and every file passed to the tool, making the verification process extremely important to be quick and accurate. However, sometimes the tests that are made might be too slow/accurate and in order to make it possible to create less accurate yet quicker test this option is available. Some input modules (although not nearly all of them) may support this option that skips the more detailed tests and accepts more rudementary validation that a file is what it says it is. [FALSE]

log_file:
    The file that the tool writes it's output to. [STDOUT]

raw:
    A boolean (0/1) that flags whether or not the tool uses the output mechanism that the output modules provide. If this is set to false the tool will operate as usual, but if true the tool will return the RAW timestamp object instead of a formatted one, as is done in the case of an output module being used. [FALSE]

append:
    A boolean value (0/1) that indicates if we want to append to the output file or to overwrite it. [FALSE]

detailed_time:
    Boolean (0/1) This is a bit of a misnamer. However, some input modules to tend to give excessive details in its message/description and even provide additional timetamps that may or may not be pertinent in every case. This option was added to the tool so that these perhaps too verbose messages/details wouldn't be introduced into the tool unless wanted/needed. This means that $FN timestamps are skipped in the $MFT module, loaded drivers are not printed in the prefetch one, etc. [FALSE]

hostname:
    A string that contains the hostname of the image/host the files are extracted from. Some input modules have the capability to extract the hostname, as does some pre-processors. This variable can however be set to override that and to make sure the hostname is printed on every event. [unknown]

preprocess:
    A boolean (0/1) that defines if we should run pre-processors before the start of the run. [FALSE]

When all values have been assigned the routine will go over each assigned variable and call a verification routine on them to verify that the variable is valid and that the supplied value of it is also valid.

When all this is done the routine will assign some other values that are used by the module, such as the OS of the computer using the tool, etc. It will also assign the value of 1 (TRUE) to the variable is_valid, indicating that we have properly set up the module and that this instance is a valid instance of Log2Timeline.

  "is_valid"
A simple subroutine that checks if the variable valid is set or not

Returns:

An integer, 1 if this is a valid l2t instance (self->{'valid'} is set), otherwise 0.

  "_run_preprocess"
A private method (not part of the public API).

The notion of a pre-processor is something that is run prior to the real execution of the tool in order to collect information from the image. Each pre-processor can then either choose to simply output the result of this finding or save it in the class variable that can then be used by other input/output modules to give more context around events.

This routine starts by finding all the available modules that are available in the pre-processing directory. Then it will run each one of those to gather the necessary information and update the settings of the tool.

  "_build_exclusions"
A private method (not part of the public API).

A simple routine that examines the exclusion list passed to the tool and converts it into a hash The exclusion list is a string list, separated with commas (,), containing file names or parts of filenames that should be excluded from the recursive scanner.

The routine simply reads the class variable 'exclusions' and builds a hash called 'exclude_list' that contains all the patterns found in the exclusion list.

Returns:

An integer that inticates whether or not the exclusion building has been successful.

  "version"
A very simple routine that only returns the current version of the tool.

Returns:

A string that contains the version of the tool.

  "_set_timezone"
A private method (not part of the public API).

This routine is used to check if a string representing a timezone is a valid timezone that is accepted by the DateTime library.

Since there are potentially two time zones defined by the end user, both the one of the suspect system/files and of the desired output there is a switch indicating which one we are testing. There is no difference between the two tests, this switch was simply introduced to make debugging information more concise, that is the difference is simply in the text used in the debug dialog.

The test that is performed is simply to load the DateTime library with the supplied timezone. If it successfully loads up it is considered to be a valid timezone string.

If the timezone that was selected is 'local' then the extracted name of said timezone is pulled from the DateTime object created (all the 'local' magic occurs within the DateTime library).

It is possible to define a long name for the timezone (e.g 'Australia/Sydney') so the DateTime library is checked to see if there is a short name for that particular timezone, and if so that is also returned (and used in the output instead of the longer one).

Args:

tz_test: A string of the timezone supplied to the tool and needs to be verified before being used.

tz_mode: Boolean variable that defines if we are testing an input or output timezone.

Returns:

A list containing two variables, tz_ret a string representing the timezone (might be long) and a shorter version of that timezone name.

  "_verify"
A private method (not part of the public API).

Since we are accepting values from the user of the tool, or from a front-end that cannot be trusted we need to validate that each attribute is correctly formed. This is not just as an attempt to verify user inputted data for security purposes, this is also put here to prevent the tool from crashing in later stages due to a bug in one of the parameters.

For each attribute/parameter of the tool that can be defined through the API it's value has to be validated. An attribute is not assigned a value unless this validation returns a true value.

The validation can be very simple, or comprehensive, depending on several factors (one being not completing the implementation).

The routine has a list of all accepted attributes, and if one is passed to the tool that the validation routine does not recognize it is deemed as an invalid attribute and therefore not saved/assigned.

Args:

attr: A string containing the name of the attribute that needs to be validated.

val: The value of said attribute.

Returns:

A boolean value, 0 if the attribute was not valid, 1 if it was deemed valid.

  "get_timezone_list"
This is a simple sub routine that pulls out all names of supported timezones of the DateTime library and puts them in a list.

Ithen sorts that list alphabetically and surrounds it with a banner that gets returned for output.

Returns:

A string containing a list of all available timezones that the DateTime library supports.

  "start"
This is one of the main sub routines, the glue that holds all together. When all values have been assigned to the module and processing can be started this is the routine that starts it all.

The routine starts by checking if it should do a recursive search or simply look at a single file.

It will then invoke various internal/protected sub routines that verify and load up needed functionality. Examples of the magic that occurs in this routine are; initiating pre-processing, loading input and output modules, figuring out what the temporary directory is, calculating the clock offset, assigning timezones, building exclusions.

When all that preparation is done the routine will either call a function to parse the file or initiate the recursive scan of a mount point/directory.

  "get_out_footer"
This subroutine can be called to retrieve the footer of a output file.

This is designed for a front-end to be able to append to an output file even though the output file has a footer.

The problem this routine tries to solve is that if a file has already been created to store the timestamp and that particular format contains a footer, simply appending to it will not cut it. That will brake the format.

The purpose of this routine is to invoke the desired output module and retrieve the footer that it will output and return that to the front-end that then can remove the footer from the previous file before starting to output new data.

Returns:

The footer (raw, mostly strings, but could be something else) that the output module produces.

  "_parse_dir"
A private method (not part of the public API).

This is the recursive method/routine/scanner of the engine. When the tool encounters a directory and it is in a recursive mode it will use this recursive method to go through every possible file in the supplied directory, and if it stumples upon a directory it will call itself with that directory as the root (and thus a recursive method is born).

It is here that the exclusion list is honored. For each file/directory that is found within the supplied directory the path is compared to the entries found inside the exclusion list. If a match is found, that particular file is not tested.

The logic in this method is simple:

1   List up all files within the supplied directory.

    a   Check against exclusion list, if not there continue.

    b   Try to parse (if this is a file or a directory).

    c   If this is a directory then call self again, this time with the current directory as the root one.

2   Done.

Returns:

An integer with the value of 1 (true) if successful.

  "_parse_file"
A private method (not part of the public API).

This sub routine reads the class variable 'file' that contains the path name to a file that needs to be parsed. It accepts only a single string to either a file or a directory.

It will then test to see if this is a file, and then open it or a directory and then open that.

Then the routine will go over each input module that has been loaded up in the tool and attempt to parse the file/directory using that module. It will provide the input module with the necessary information it needs, such as:

fh: A filehandle to the file that needs to be parsed.

name: Name of the file (the full path).

It will then attempt to verify the module can parse the file, and if it successfully validates it will check to see if the module returns a single timestamp object or a one object per line read (variable 'multi-line').

The routine will then either collect that single container and go over each entry therein or call the get_time until all timestamp objects have been collected.

For each output object gathered the tool will check if it should return the raw object (as defined in the 'raw' class variable) or process the output with an output module.

When the routine has completed parsing the file it will close it.

Returns:

An integer, 0 if an error occured or 1 if the operation was successful.

  "_process_timestamp"
A private method (not part of the public API).

After each timestamp object has been extracted from the parsing engine it is passed through this sub routine.

What is essentially does is processing of the timestamp object. It adds some values into it and fixes/adjusts others.

For instance, this routine replaces all backslashes with forward slashed in the description field, it adds the text description field passes as an argument to the timestamp object, it includes information about the directory the tool was called from, includes hostname, etc.

The sub routine also injects other values into the timestamp object, such as the inode value of the file and if the calculate parameter is used it calculates a MD5 sum for all the files passed to the tool and adds that to the timestamp object.

Finally this routine is also responsible for adjusting the timestamps according to the value of the time offset passed as a parameter. That is if we need to adjust all the timestamps because of a faulty clock on the system, that time difference is added or subtracted from the timestamp in this routine.

Args:

t_line: A timestamp object (which is a reference to a hash, hence not a need to return it back).

Returns:

Boolean value(0/1): 0 if the timestamp object is not clearly defined, 1 if successful.

  "_open_file"
A private method (not part of the public API).

A simple sub routine that is responsible for opening up a file and assigning the filehandle to the $self->{'fh'} variable that is used in the tool.

  "_close_file"
A private method (not part of the public API).

A simple sub routine that has only one task, and that is to close the open filehandle.

  <_open_dir>
A private method (not part of the public API).

A simple sub routine that opens up a directory and gives back an open handle to that directory.

No arguments passed to it nor returned (only uses $self)

  <_close_dir>
A private method (not part of the public API).

A simple sub routine that closes a filehandle to a directory.

No arguments passed to it nor returned (only uses $self)

  "_input_exists"
A private method (not part of the public API).

Determine if an input module exists or not. This sub routine takes as an input a list of input modules. This list can consist of a single input module, a single reference to a list file, or it may be a more complex list containing both modules, lists and negative modules (list of modules that should be excluded).

Args:

in: A string that contains a list of modules/list files (comma separated).

Returns:

A string depending on whether or not that input module exists. That is the module goes over the entires list and if it finds at least one module that does not exist on the filesyste it will save the content of that particular variable in the return value. Otherwise the value of 0 is returned.

  "_output_exists"
A private method (not part of the public API).

Check for the existance of the output module. That is this routine checks to see if the output module chosen by the user exists or not.

Args:

out: A boolean value (0/1) indicating whether ot not the file exists on the system or not.

  "get_timezone"
A small sub subroutine that simply returns back the value of the current timezone used by the tool.

Returns:

A string containing the timezone that the tool is using.

  "_load_input_list"
A private method (not part of the public API).

A routine that opens up a list file reads in all it's content and stores it in the input_list class attribute.

If a list file is passed to the tool as a parameter this routine will use that to open up the file and read it in, line-by-line to get a list of all the input module it should load (the input lists contain a list of modules they want to use.

=head3: Args:

list: A string containing a listfile that needs to be parsed.

  "_load_input_module"
A private method (not part of the public API).

# either remove or add a module to the input list

  "_load_input"
A private method (not part of the public API).

  "_load_output"
A private method (not part of the public API).

  "_get_module_help"
A private method (not part of the public API).

  "_format_sort"
A private method (not part of the public API).

A sorting 'algorithm' for input modules.

The problem with some of the input modules is that there might be two input modules that are capable of parsing the same file. And since the tool stops processing each file when a match is found you might end up parsing a file using a module that is not really suited to do so.

The most prelevant example is the exif module that is capable of extracting a generic metadata from vast amount of different types of files. There might be other modules that are specifically written to parse that particular file, which do a lot better job of extracting relevant data from it. This routine is therefore written to lower the priority of these more generic modules so that they do not parse files before the more specific ones do.

Currently the following modules do have lower priority associated to them:

exif
generic_linux

  "_calc_offset"
A private method (not part of the public API).

A sub routine that takes the offset value that is given to the API and converts it into an integer that is used to balance of the timestamps read.

The offset can be one of each values:

+ int: numbers of seconds (eg. 52 or -12)

+ string: An int with appended character indicating the unit of the int. Accepted values are h, m or s that correspond to hours, minutes and seconds. Examples: 52s or 1h (n.b. it is not possible to use 4h2m1s to represent the time in more granularity, it is only possible to use one string, making the int option most useful since offset rarely comes in whole hours.

No arguments are needed since the routine only uses and sets class variables.

AUTHOR
Kristinn Gudjonsson <kristinn (a t) log2timeline ( d o t ) net> is the original author of the program.

COPYRIGHT
The tool is released under GPL so anyone can contribute to the tool and examine the source code. Copyright 2009-2012.

SEE ALSO
Documentation for each input module follows the name of Log2t::input::MODULE and for output modules Log2t::output::MODULE

log2timeline, Log2t::Time, Log2t::BinRead, Log2t::Common, Log2t::Network, Log2t::Numbers, Log2t::Win, Log2t::WinReg

POD ERRORS
Hey! The above document had some coding errors, which are explained below:

Around line 182:
    Unknown directive: =over4

Around line 184:
    '=item' outside of any '=over'