Libdar API - Tutorial
Author: Denis Corbin
Description: Tutorial for using libdar library

| # LIBDAR # APPLICATION INTERFACE # TUTORIAL for API version 5.x.x #

## Presentation The Libdar library has been built from source code originally located directly in the dar command line application. Libdar provides a complete abstraction layer for handling Disk ARchive (dar)'s archives. The general operations provided are: archive creation, file extraction, archive listing, archive testing, archive comparison, catalogue isolation archive merging dar_manager database manipulations Note that Disk ARchive and libdar have been released under the Gnu General Public License (GPL). All code linked to libdar (statically or dynamically), must also be covered by the GPL. This tutorial will show you how to use the libdar API. As dar since its release 2.0.0 also uses this API, looking at it's code may also provide a good illustration. The file src/dar_suite/dar.cpp is the primary consumer of the libdar API. The sample codes provided here is solely illustrative and is not guaranteed to compile. More detailed API documentation is contained in the source code and can be compiled to the doc/html directory using Doxygen, which is also provided online.

## Let's Start ### Conventions #### Language Dar and libdar are written in C++, and so is the libdar API. While written in C++, libdar is easily usable by both C and C++ code. Access from other languages can be provided by specific bindings. I would only say that you are welcome to provide the necessary bindings yourself. :-) #### Libdar namespace All libdar symbols are defined under the libdar namespace. You can either add the using namespace libdar; line at the beginning of your source files:

`using namespace libdar;` `get_version(....);`

or, as shown below, you can explicitly use the namespace in front of libdar objects :

``

`libdar::get_version(....);`

#### Exceptions or no Exceptions The library can be used with or without exceptions. For each example we will see a sample code for both ways. To the left is with exceptions, to the right without:

`example code using exceptions` | `example code not using exceptions`

All exceptions used by libdar inherit from the pure virtual class Egeneric. The only method you will need to know about for any exception is the get_message() call, which returns a message string describing the message (in human language). The type of the error is defined by the class of the exception. The possible exception types follow:

`class libdar::Egeneric` | the parent class of all exceptions (a pure virtual class)
`class libdar::Ememory` | memory has been exhausted
`class libdar::Ebug` | signals a bug, which is triggered when reaching some code that should never be executed
`class libdar::Einfinint` | arithmetic error detected when operating on infinint
`class libdar::Elimitint` | a limitint overflow is detected, indicating the maximum value of the limitint has been exceeded
`class libdar::Erange` | signals a range error
`class libdar::Edeci` | signals conversion problem between infinint and string (decimal representation)
`class libdar::Efeature` | a requested feature is not (yet) implemented
`class libdar::Ehardware` | hardware problem is found
`class libdar::Euser_abort` | signals that the user has aborted the operation
`class libdar::Ethread_cancel` | A program has requested the termination of the current thread while libdar was running
`class libdar::Edata` | an error concerning the treated data has been encountered
`class libdar::Escript` | the script executed between slices returned an error code
`class libdar::Elibcall` | signals an error in the arguments given to a libdar call of the API
`class libdar::Ecompilation` | a requested feature has not been activated at compilation time

## 1 - First we *must* initialize libdar by checking the libdar version

`// we'll want to display some messages #include <io.h> // we include this header to access lidbar API #include <dar/libdar.h> // all sample code shown will be inside this // function for simplicity's sake void my_sample_function() { try { libdar::U_I maj, med, min; // first we MUST call get_version() libdar::get_version(maj, med, min); if(maj != libdar::LIBDAR_COMPILE_TIME_MAJOR || med < libdar::LIBDAR_COMPILE_TIME_MEDIUM) throw libdar::Erange("initialization", "we are linking against a wrong libdar"); }` `catch(libdar::Egeneric & e) { std::cout << e.get_message() << std::endl; } }` | ```// we'll want to display some messages #include <io.h> // we include this header to access lidbar API #include <dar/libdar.h> // all sample code shown will be inside this // function for simplicity's sake void my_sample_function() {` `libdar::U_I maj, med, min; libdar::U_16 excode; std::string msg;` `// first we MUST call get_version() libdar::get_version_noexcept(maj, med, min, excode, msg); if(excode != LIBDAR_NOEXCEPT) { std::cout << msg << endl; return; } if(maj != LIBDAR_COMPILE_TIME_MAJOR ||` `med < libdar::LIBDAR_COMPILE_TIME_MEDIUM``) { std::cout <<` `"we are linking against wrong libdar"``<< std::endl; return; }`

The get_version() function must be called for several reasons : you must check that the library you've dynamically linked with is compatible with the features you will be using. The major number must be the same, for no compatibility is assured between two libdar versions of different major numbers. While run-time compatibility is assured between medium numbers, the medium number must be greater or equal to the one used at compilation time to be sure that all the features you want are available in the libdar library you dynamically linked with. Changes between minor versions correspond to bug fixes and is not to imply any API change, thus no constraints are present there (except the presence of more bugs in lower numbers). the get_version() call, as well as returning version information, does important initialization tasks for libdar. If not called first, the libdar library will not initialized properly and its behavior will be unpredictable. Note that you may call get_version() several time if you wish, the secon time only the version information are returned, the libdar library is not reset or the like. Last, if strong encryption support is activated at compilation time, libdar will by default performs libgcrypt initialization when get_version() is called if libgcrypt is not already initialized. You can avoid having dar initializing libgcrypt by calling get_version() with an additional argument set to "false" : ger_version(maj, med, min, false) Note that in multi-thread environment with strong encryption activated in libdar at compilation time, libgcrypt requires to be initialized from the application, thus you *must* avoid having libdar initializing libgcrypt by using the get_version() described just above and proceed from the application to the initialization of libgcrypt before calling get_version(). For more details see libgcrypt documentation. ## 1 bis - We must prepare the end right now! As we saw, libdar used some datastructures (mutex, secured memory, etc.) that need to be released properly before ending the program. It is mandatory to invoke the following function before exiting your program if you invoked get_version() previously. It is a good idea to implement this right now not to forget later:

`` `libdar::close_and_clean()` ``

In particular, closes_and_clean() makes the necessary for memory to be released in the proper order. Not calling close_and_clean() at the end of your program may result in uncaught exception message from libdar at the end of the execution. This depends on the compiler, libc and option activated in libdar at compilation time. ## 2 - Let's see the available features once we have called one of the get_version* function it is possible to access the list of features activated at compilation time:

`void my_sample_function() { // let's continue in the same function` `bool ea = libdar::compile_time::ea(); bool largefile = libdar::compile_time::largefile(); bool nodump = libdar::compile_time::nodump(); bool special_alloc = libdar::compile_time::special_alloc(); U_I bits = libdar::compile_time::bits(); // bits is equal to zero for infinint, // else it is equal to 32 or 64 depending on // the compilation mode used. bool thread = libdar::compile_time::thread_safe(); bool libz = libdar::compile_time::libz(); bool libbz2 = libdar::compile_time::libbz2(); bool liblzo = libdar::compile_time::liblzo(); bool libcrypto = libdar::compile_time::libgcrypt(); bool furtive_read = libdar::compile_time::furtive_read`(); `` } | `` `// here there is no difference because no exceptions // are thrown by the get_compile_time_feature() // function`

You can do what you want with the resulting values. It's possible to display the available libdar features or to terminate if you don't find a desired feature. However, verifying that features are available is not strictly necessary because libdar will tell you if an operation you call requires a feature that has not been activated at compilation time, by throwing an Ecompile exception (or returning the LIBDAR_ECOMPILATION error code if you are not using exceptions). ## 3 -User interaction ### The generic user_interaction class To be able to report messages to the user and prompt for feedback a special class called user_interaction has been introduced. Simply put, user_interaction is a virtual class which you can derive to provide user interaction (a GUI's graphical interaction, for example). There are four methods whose prototypes you must override: void pause (const std::string &message); this method is called by libdar when the library needs a yes or no ("continue" or "kill") answer to a question, which is provided by the string message. The question posed by pause() must be answered by returning normally (= "true") or throwing a Euser_abort exception if the user refused the proposition. Don't worry about throwing an exception in your code; it will be trapped by libdar if you don't want to manage exceptions, and are using libdar in the "no exception" method. But if you really don't want to throw exception from your code see next: bool pause2(const std::string &message); This is an alternative method to pause() as seen above. In place of defining a pause() method in your inherited class, you can redefine the pause2() method. The only difference with pause() is that the user answer to the question is returned by a boolean value, your code does no more have to throw a Euser_abort exception to say "no". Note that you must not redefine both pause() and pause2(). void inherited_warning (const std::string &message); libdar calls this protected method (through the public method named warning()) to display an informational message to the user. It is not always a warning as the name suggests, but sometimes just normal information. In API 3.0.x this method did not exist, but the public warning() method itself was pure virtual and thus needed to be overwritten. Today, the warning() method is no more pure virtual nor it is even virtual, so the user defined implementation of message display has to be done in the inherited_warning() method. std::string get_string (const std::string &message, bool echo); This call is used to get an arbitrary answer from the user. This is mainly used to get a password from the user (when no password has been supplied for an encrypted archive), the echo argument indicates if the user response should be displayed back on the screen (again, very useful for handling password input). If echo is set to "false" the implementation of get_string() should hide the characters typed by the user. user_interaction * clone () const; A deep copy operation must be implemented here. This is because libdar stores the reference to the user_interaction class as a pointer but may want to keep a complete internal copy at some point. A simple implementation of this method should be something like this (even if you don't want to use exceptions):

````

`` `user_interaction *my_own_class::clone() const { my_own_class *ret = new my_own_class(*this); if(ret == NULL) throw Ememory("user_interaction_callback::clone"); else return ret; }`

### The callback interaction class An inherited class from user_interaction called user_interaction_callback provides an implementation of the user interaction based on callback functions. This allows you to replace the three interactions methods (pause, warning and get_string) by three normal functions of your choice, which must be given to the user_interaction_callback's constructor. The clone() method is implemented internally, leaving only the three callback functions to be implemented. Look at dar's command line code for a practical example. dar's user interaction code is implemented using an instance of user_interaction_callback and three static functions in the module dar_suite/shell_interaction.cpp Pay attention to the contextual value present in the arguments of these callback functions :

````

`` `// our own callback functions. // for the illustration of what these 'context' arguments // can be used for we will imagine the situation where // multiple windows or multiple threads may each one use // libdar, but all share the same callback functions. typedef class t_window_type t_win; // this is an arbitrary type that here we will say // points to a graphical window object wrapped in a C++ // class. // Note that the method show() wait_for_click() and so on // attributed to the t_win class are absolutely // imaginary. Any link to an existing class is a pure // coincidence... void warning_callback(const std::string &x, void *context) { (t_win *)(context)->show(x); } bool answer_callback(const std::string &x, void *context) { click_type ret;` `(t_win *)(context)->show(x); ret = (t_win *)(context)->wait_for_click(); return ret == click_OK;` `} std::string string_callback(const std::string &x, bool echo, void *context) { (t_win *)(context)->show(x); if(!echo) (t_win *)(context)->set_hide_typed_char(); (t_win *)(context)->wait_for_click(); return (t_win *)(context)->read_text(); } ---------8<-------8<-------8<------- // So now each window can have its user_interaction object based on the same // user_interaction_callback object pointing to the same functions. // user_interaction_callback objects can be shared among different window objects libdar::user_interaction_callback dialog = libdar::user_interaction_callback(&warning_callback, &answer_callback, &string_callback, (void *)get_current_windows_id()); // just the "context" argument changes, and will be passed as is from the constructor to the callback // functions`

## 4 - Masks Mask are used to define which files will be considered and which will not. Libdar implements masks as several classes that all inherit from a virtual class that defines the way masks are used. This root class is the class mask and provides the is_covered() method which libdar uses to determine which files are considered. There are many different basic masks classes you can use to build fairly complex masks:

class libdar::mask | the generic class, parent of all masks (a pure virtual class)
class libdar::bool_mask | boolean mask, either always true or false, it matches either all files or no files at all
class libdar::simple_mask | matches as done by the shell on the command lines (see "man 7 glob")
class libdar::regular_mask | matches regular expressions (see "man 7 regex")
class libdar::not_mask | negation of another mask
class libdar::et_mask | makes an *AND* operator between two or more masks
class libdar::ou_mask | makes the *OR* operator between two or more masks
class lbdar::simple_path_mask | string matches if it is subdirectory of mask or is a directory that contains the specified path itself
class libdar::same_path_mask | matches if the string is exactly the given mask (no wild card expression)
class libdar::exclude_dir_mask | matches if string is the given string or a sub directory of it
class libdar::mask_list | matches a list of files defined in a given file

Let's play with some masks :

````

`` `// all files will be elected by this mask` `libdar::bool_mask m1 = true; // all file that match the glob expession "A*~" will match. // the second argument of the constructor tell if the match is case sensitive so here // any file beginning by 'A' or by 'a' and ending by '~' will be selected by this mask libdar::simple_mask m2 = libdar::simple_mask(std::string("A*~"), false); // m3 is the negation if m2. This mask will thus match // any file that does not begin by 'A' or 'a' and also finish by '~' libdar::not_mask m3 = m2; // this mask matches any file that is a subdirectory of "/home/joe" // and any directory that contains /home/joe, meaning // "/", "/home", "/jome/joe" and any subdirectory are matched. // here, the second argument is also case sensitivity (so // "/HoMe" will not be selected by this mask. libdar::simple_path_mask m4 = simple_path_mask("/home/joe", true); // now let's do some more complex things:` `// m5 will now match only files that are selected by both m2 AND m4` `libdar::et_mask m5; m5.add_mask(m2); m5.add_mask(m4); // we can make more silly things like this, where m5 will select files // that match m2 AND m4 AND m3. But m3 = not m2 so now m5 will never // match any file... m5.add_mask(m3); // but we could do the same with an "ou_mask" and would get a silly // counterpart of m1 (a mask that matches any files) libdar::ou_mask m6; m6.add_mask(m2); m6.add_mask(m4); m6.add_mask(m3); // lastly, the NOT, AND and OR operation can be used recursively.` `// Frankly, it's possible to have masks reference each other!` `libdar::not_mask m7 = m6; m6.add_mask(m7);`

Now that you've seen the power of these masks, you should know that in libdar there are three masks that are required: The first mask is used against the names of all files except directories. It is applied solely to the names themselves (not the file path). This mask may be any combination of the masks seen previously; it will only be applied to socket, named pipes, symbolic links, char or block devices, plain files, but again not to directories. This way you can filter by file type for save, restore, list, compare, compress, and other library operations. The second mask is applied to any file including directories, including the path part of the filename. So with it you can prune directories, or in any other way restrict the operation to a particular subdirectory, as well as to a particular plain file for example. Important note about this second mask: what your own mask will be compared to by libdar is the filesystem root (as defined under the argument "fs_root" of the same call you will give your own mask to) plus the current file being proceeded: Assuming you choose for example tmp/A as argument to fs_root (which argument is present when creating an archive, for example), your mask will be used against strings like "tmp/A/some/file" . This is true up to libdar version 3.0.x (alias release 2.2.x). Instead, since libdar 4.0.0 the fs_root argument is expended to an absolute path, so if in the previous example, your current directory was /var your masks will be used against strings like "/var/tmp/A/some/file". Of course there is no difference between these two libdar revisions when the fs_root argument is an absolute path [this change was necessary to support masks based on a list of files] An exception is the test operation, which has no fs_root argument (because the operation is not relative to an existing filesystem), however the subtree argument exist to receive a mask for comparing the path of file to include or exclude from the test operation. In this case the situation is as if the fs_root was set to the value "<ROOT>". For example, masks will be compared to <ROOT>/some/file when performing an archive test operation. The third mask --- which is not always needed --- concerns Extended Attributes (EA). It is applied to the full EA name in the form <domain>.<name> where <domain> is any string value like but not limited to the usual "user" or "system" domains. ## ## 5 - Let's create a simple archive Now that we have seen masks and exceptions let's start the real thing: All the operations on archives are handled by the archive class which is defined in libdar/archive.hpp. Each operation requires some mandatory parameters and some optional parameters. Optional parameters are gathered in a archive_option auxilliary class, which default constructor set them to default values. We will see a bit further how to set these options, but for now let's keep the things simple:

`// creating an archive is simple; it is just // a matter of calling the "create" constructor // of the archive class. It may be used for full or // differential archives. We'll see an example of // of differential archives later. // note that while this example uses a pointer to store // my_arch, it is perhaps better practice to use a plain // stack object. In your code, use an object instead of // a pointer to an object under normal circumstances. libdar::user_interaction_callback dialog = libdar::user_interaction_callback(ptr1, ptr2, ptr3); // where ptr1, ptr2 and ptr3 are three callback // functions. libdar::statistics ret; // we will see this structure a bit further libdar::archive *my_arch = new libdar::archive(dialog, "/home", // saving all under this "root" "/tmp", // where the slices will go "my_archive", // the basename of the slices "dar", // the extension used for slices archive_options_create(), // default options &ret); // this value is returned by libdar // if you don't want to have statistics of the // operation you can set this parameter to NULL` | ```// creating an archive is simple; it is just // a matter of calling the "create" constructor // of the archive class. It may be used for full or // differential archives. We'll see an example of // of differential archives later. // note that while this example uses a pointer to store // my_arch, it is perhaps better practice to use a plain // stack object. In your code, use an object instead of // a pointer to an object under normal circumstances.` `libdar::user_interaction_callback dialog = libdar::user_interaction_callback(ptr1, ptr2, ptr3); // where ptr1, ptr2 and ptr3 are three callback // functions. libdar::statistics ret;` `// we will see this structure a bit further` `U_16 exception, std::string except_msg;` `libdar::archive *my_arch = libdar::create_archive_noexcept(dialog,``"/home", // saving all under this "root" "/tmp", // where the slices will go "my_archive", // the basename of the slices "dar", // dar's slice extensions archive_options_create(), // default options``` `&ret, // this value is returned by libdar` `// if you don't want to have statistics of the // operation you can set this parameter to NULL` `exception, // this gives the status of the call except_msg); // and in case of error the cause. if(exception != LIBDAR_NOEXCEPT) std::cout << "an error occurred: " << except_msg << std::endl;`

When creating an archive, the created archive object can be used only as reference for an isolation or for a differential backups. You cannot use it for restoration, listing, or comparison, because the underlying file descriptors are opened in write only mode. An implementation which uses file descriptors in read-write access is not possible and is not a good idea anyway. Why? Because, for example, if you want to test the newly created archive, using the newly created object would make the test rely on information stored in virtual memory (the archive contents, the data location of a file, etc.), not on the file archive itself. If some corruption occurred in the file you would not notice it. So to totally complete the archive creation we must destroy the archive object we have just created, which will also close any file descriptors used by the object :

`delete my_arch;` | `libdar::close_archive_noexcept(my_arch, exception, except_msg);` `if(exception != LIBDAR_NOEXCEPT) std::cout << "an error occurred: " << except_msg << std::endl;`

### Optional Arguments: Back on the optional arguments. The archive constructor used above to create an archive uses an argument of type "archive_option_create". In the above example, we called the constructor of this class directly withing the argument list of the constructor. Thsi has for effect to built a anonymous temporary object of this class. Such a "just borned" object has all the necessary options set to the default values inside it (like default masks for example) to correspond to the default options. If you want to use non default options like compression, slicing, encryption , file filtering and so on, you must change the options to your need thanks to the appropriate method provided by the archive_options_create class. Assuming we want to make a compressed an sliced archive we would use the following code:

`// we define an options object to be able to use // non default options: libdar::archive_options_create options;` `// so now we can modify only the option we wish to options.set_slicing(1024); // 1024 bytes, well that's small options.set_compression(bzip2); // default is no compression options.set_compression_level(6); // default is 9` `libdar::statistics ret; // we will see this structure a bit further libdar::archive *my_arch = new libdar::archive(dialog, "/home", // saving all under this "root" "/tmp", // where the slices will go "my_archive", // the basename of the slices "dar", // the extension used for slices options, // the object we modified above &ret); // this value is returned by libdar // if you don't want to have statistics of the // operation you can set this parameter to NULL` | ```` `// we define an options object to be able to use // non default options: libdar::archive_options_create options;` `// so now we can modify only the option we wish to options.set_slicing(1024);` `// 1024 bytes, well that's small` `options.set_compression(bzip2); // default is no compression options.set_compression_level(6); // default is 9` `libdar::statistics ret;` `// we will see this structure a bit further` `U_16 exception, std::string except_msg;` `libdar::archive *my_arch = libdar::create_archive_noexcept(dialog,``"/home", // saving all under this "root" "/tmp", // where the slices will go "my_archive", // the basename of the slices "dar", // dar's slice extensions options, // the object we modified above``` `&ret, // this value is returned by libdar` `// if you don't want to have statistics of the // operation you can set this parameter to NULL` `exception, // this gives the status of the call except_msg); // and in case of error the cause. if(exception != LIBDAR_NOEXCEPT) std::cout << "an error occurred: " << except_msg << std::endl;`

In the same way, each other operation (diff, testing, extraction, merging, ...) has a specific class that gathers the options parameters. These classes are defined in the file libdar/archive_options.hpp you are welcome to refer to for a complete up to date list of available options. The advantage of this class is to not break ascendant compatibility of the API when new features get added, while it also improve readability of your code. This way, the major current number '5' of the API should stay for a longer time than previous numbers, as for thoses each new feature implementation broke the ascendant compatibility by adding an new argument to an API call. ## 6 - Testing the archive we have created So, as explained previously, we must create a new archive object but this time with the "read" constructor:

`
`

`my_arch = new` `libdar::archive(dialog, "/tmp", // where is the archive "my_archive", // slice name "dar", // dar's archive extensions archive_options_read()); // default options` | `my_arch =` `libdar::open_archive_noexcept(``dialog, "/tmp", // where is the archive "my_archive", // slice name "dar", // dar's archive extensions archive_options_read(), // default options` `exception,// this gives the status of the call except_msg); // and in case of error the // cause of the error i` `f(exception != LIBDAR_NOEXCEPT) std::cout << "an error occurred: " << except_msg << std::endl;` ``

Now that we have opened the archive we can perform any operation on it. Let's thus start by testing the archive coherence:

`// for the exercice, we will change the default options: archive_options_test options; options.clear(); // this set back all options to default // here this is not required as the object has just bee // created, however it is used here for illustration that // you can recycle an archive_option_* object. options.set_info_details(true); // to have a verbose output ret = my_arch->op_test(dialog, options; // the non default options set above` `NULL); // we don't want a progressive report``` | `// for the exercice, we will change the default options: archive_options_test options; options.clear(); // this set back all options to default // here this is not required as the object has just bee // created, however it is used here for illustration that // you can recycle an archive_option_* object. options.set_info_details(true); // to have a verbose output` `ret = libdar::op_test_noexcept(``dialog, my_arch, // the archive to test` `options, // the non default options set above NULL, // we don't want a progressive report` `exception,// this gives the status of the call except_msg); // and in case of error the // cause of the error` `i` `f(exception != LIBDAR_NOEXCEPT) std::cout << "an error occurred: " << except_msg << std::endl;`

We have tested the archive, but have not yet seen the libdar::statistics variable. It can be used when creating an archive as well as when testing it. This object reports the number of files treated, as well as the number files with errors and the type of error. You can have a look at the API reference guide concerning the archive class methods, for more information about the uses of these different fields. Here is an example, which relies on the class deci to display the value of an infinint variable:

````

`// we need the class deci to display the value of an infinint:` ```#include "deci.hpp" std::cout << std::string("Number of file treated :") << libdar::deci(ret.treated).human() << std::endl; // or much simpler (but totally equivalent): std::cout << std::string("Number of file treated :") libdar::<< ret.treated << std::endl;`

Note that the use of the class deci may throw exceptions (in case of lack of memory, for example), and there is actually no wrapper available to trap the exceptions that may be thrown by the class deci. So you have to protect the code using a try {} catch {} statement. You may have noticed that we used NULL as argument for "progressive_report". This argument must either receive NULL as argument or the address of a real allocated statistics object. This object will be updated by the libdar call and if multi-threaded support is enabled it will let a concurrent thread able to reading its value to display the current number of file treated, for example. Note that there is a little overhead passing a variable to progressive_report, due to the mutex that need be used to avoid one reading data while it is updated by another thread. Follows a example of use of this progressive report feature:

````

`// we need a variable that will be visible by two threads:` `libdar::statistics report; // and we need store the libdar call returned value libdar::statistics final_result; // we spawn a first task with a libdar call passing &report as argument to "progressive_report" final_result = some_call_to_be_defined_to_call_op_test_in_another_tread(..., &report); // doing a endless loop (assuming the current thread will be signaled or interrupted once the // previously libdar call running in another thread will end) while(true) { sleep(1); // updating the display each second some_function_to_update_the_display_with(report); }`

## ## 7 - listing archive contents The simple way:

`my_arch->op_listing(dialog, archive_option_listing()); // default options` | `libdar::op_test_listing(``dialog, archive_options_listing(), // default options``` `exception,// this gives the status of the call except_msg); // and in case of error the // cause of the error` `i` `f(exception != LIBDAR_NOEXCEPT) std::cout << "an error occurred: " << except_msg << endl;`

By default the library will complete the listing by calling the warning() method of the dialog object one time for each file listed. The warning text will consist of a string for each file with the relevant information in columns that would need to be parsed if individual information was desired. This may not be appropriate for you and as such there is another way to get listing information. This requires a simple reimplementation of the user_interaction object. The user_interaction class has a listing() method which provides separate arguments for each piece of information that can be displayed: filename, permission, user, group, file size, last modification date, if the file is a directory if the file has children or is an empty dir file type flag about saved data / saved EA / compression used Technical note: You may notice that file type is not explicitly given as a parameter in the listing method. File type is available as the first byte of the permissions string. This is standard POSIX stuff except for an extension: "h" for files hard linked several times (it has been removed after release 2.3.0 / API 4.0.0). See man 2 stat for more information about POSIX permissions. Note however that the last arguments of this call, let you easily know whether a file is a directory or not and whether it is empty or not. In the user_interaction class (a virtual class), the listing() method is not a pure virtual method, so you are not obliged to overwrite it, but it has just an empty implementation so it does nothing. You understand now that, by default, this method is not used. To activate it, you must call set_use_listing(true) protected method and of course you will have to overwrite the listing() method to have a less silly behavior:

````

`// here follows the definition of our own implementation of // of a user_interaction class class my_user_interaction : public user_interaction { public : // the inherited pure virtual methods we must define // as seen at the beginning of this tutorial: void pause(const std::string & message);` `void warning(const std::string & message); std::string get_string(const std::string & message, bool echo); user_interaction *clone() const; // we can overwrite this method to have splitted fields for listing: void listing(const std::string & flag, const std::string & perm, const std::string & uid, const std::string & gid, const std::string & size, const std::string & date, const std::string & filename, bool is_dir, bool has_children); // but it will not get used by libdar unless we call the protected method set_use_listing() // for example this can be done in the class constructor : my_user_interaction() { set_use_listing(true); }; };`

Now assuming we have implemented the listing() method in my_user_interaction class, calling op_listing() exactly as we did before, only replacing the dialog object by one of the my_user_interaction class. Then this listing() method will be called for each file to be listed, in place of the warning() method. As seen at the beginning of this tutorial, there is a child class of user_interaction based on callback functions which is called user_interaction_callback. The listing() method must also be activated here. This is done automatically when you give a callback function to the object, thanks to the set_listing_callback() method :

````

`` `// our mandatory callback functions: void warning_callback(const std::string &x, void *context) { .... } bool answer_callback(const std::string &x, void *context) { .... } std::string string_callback(const std::string &x, bool echo, void *context) { .... } // let's build a user_interaction_callback object: libdar::user_interaction_callback dialog = libdar::user_interaction_callback(&warning_callback, &answer_callback, &string_callback, NULL); // at this point our dialog object is perfectly operational for listing // but libdar will call the warning_callback function to list the archive // contents // a new callback function for listing : void listing_callback(const std::string & flag, const std::string & perm, const std::string & uid, const std::string & gid, const std::string & size, const std::string & date, const std::string & filename, bool is_dir, bool has_children, void *context) { .... } dialog.set_listing_callback(&listing_callback); // now libdar will call the listing_callback function when we // use this dialog object for listing the archive contents.`

Last point about listing, if you examin the definition of the archive_option_listing class in file libdar/archive_options.hpp, you will notice the the set_list_mode() method. It may receive either normal, tree or xml (normal being the default). normal, produces (if the listing() method of the given user_interaction object is not overwritten) a listing like tar would do. tree produces a tree like directory listing (this was the original listing format in dar version 1.0.0) xml, produces a XML output as described in doc/dar-catalog-1.0.dtd Note that for these two last formats (tree and xml) the listing() is never used, so even if you provide a object which listing() method is overwritten, the archive::op_listing() method will still use the warning() method of this user_interaction object to report the archive contents. ## 7 bis - Dynamic archive contents listing Well, in the previous chapter, we saw how to list the archive contents. You can imagine that when you have a huge archive this op_listing() call may take a long time to complete and produces a long output. If your application uses some graphical components and you want to have a more interesting way for listing the archive contents, you would maybe like to have just the first level of the directory tree and let the user open the subdirectories and list their contents when needed, having a sort of iterative archive listing. This would avoid having to wait for the long listing to complete as well as it would avoid having to allocate memory for all this graphical components representing each directories and files, entries that will most of the time would not be read by the user. First step, we need to use the listing() method of the user_interaction() as seen above. Second step, we have to call the get_children_of() method of a given archive class, instead of the op_listing() method. In the following example, we will use the user_interaction_callback class, but you can use your own inherited class from user_interaction, and its listing() class.

````

`` `// our mandatory callback functions: void warning_callback(const std::string &x, void *context) { .... } bool answer_callback(const std::string &x, void *context) { .... } std::string string_callback(const std::string &x, bool echo, void *context) { .... } // Now the callback function implementing the listing() method of class user_interaction` `void listing_callback(const std::string & flag, const std::string & perm, const std::string & uid, const std::string & gid, const std::string & size, const std::string & date, const std::string & filename, bool is_dir, bool has_children, void *context) { .... } // Now that our callback functions are ready, let's create a user_interaction callback object named "dialog"` `libdar::user_interaction_callback dialog = libdar::user_interaction_callback(&warning_callback, &answer_callback, &string_callback, NULL); // now we must assign the listing_callback() function to "dialog". dialog.set_listing_callback(&listing_callback); // Let's open an archive: archive some_archive = archive(dialog, "/some/dir", "basename", "dar", archive_option_read()); // we are reading a new archive, but we could have created one instead...` `// now, instead of calling op_listing() method of some_archive giving our dialog object as argument, we can rather call: some_archive.get_children_of(dialog, ""); // the second argument is the directory of which we want to know the subdirectories and subfiles. Here "" means the // root of the archive (we cannot use absolute path here). // get_chidren_of() method will call listing()'s dialog method (here our listing_callback() function through the // user_interaction_callback implementation) for each entry of the root directory. // let suppose that thanks to listing_callback() during the previous call to get_chidren_of() we know that the entry // "var" exists (filename == var) and is a directory (is_dir == true) and has some children (has_children == true), // suppose the user want to know what is inside this directory, we would then only have to call: some_archive.get_children_of(dialog, "var"); // assuming through listing_callback we know that a subdirectory tmp exist and is not empty, assuming that the user // want to know what is in it: some_archive.get_children_of(dialog, "var/tmp"); // and so on.`

## ## 8 - comparing with filesystem We can compare file in an archive with the filesystem by calling the op_diff method of the class archive.

`ret = my_arch->op_diff(dialog, "/home", // what directory to take // as root we shall // compare the archive // contents to archive_options_diff(), // default options` `NULL); // we don't use progessive report` `` | `ret =` `libdar::op_diff_noexcept(dialog, my_arch, // the archive to use "/home", // what directory to take // as root we shall // compare the archive // contents to` `archive_options_diff(), // default options NULL, // we don't use progressive report` `files` `exception, // this gives the // status of the call except_msg); // and in case of // error the cause of the // error` `i` `f(exception != LIBDAR_NOEXCEPT) std::cout << "an error occurred: " << except_msg << std::endl;` ``

Simple, no? Just a note about the set_what_to_check() method argument of the archive_options_diff class. It may take several values: cf_inode_type (default value): a file is considered as changed if its inode type has changed (directory/plain file/symbolic link/ ...) cf_mtime : permission change is ignored, as well as ownership change cf_ignore_owner : ownership change is ignored cf_all : all fields denoting a content's file change triggers a file changed status (ownership, permission, dates, inode type). ## 9 - restoring files Restoration of files is done by calling the op_extract method of class archive.

`ret = my_arch->op_extract(dialog, "/tmp", // where to restore files to archive_options_extract(), // default options` `NULL);// no progressive report used` | `ret = libdar::op_extract_noexcept(``dialog, my_arch, // the archive to test` `"/tmp", // where to restore files to` `archive_options_extract(), // default options``` ```exception,// this gives the status of the call except_msg); // and in case of error the // cause of the error` `i` `f(exception != LIBDAR_NOEXCEPT) std::cout << "an error occurred: " << except_msg << std::endl;`

Here as we used default options, we restore all the files stored in the archive in the directory /tmp (we also restore there the directory structure stored in the archive), but we could also make a flat restoration (ignore directory structure), as well as restore only some of the files. By default too, dar asks user confirmation before overwriting a file. You can change these options and many others using the methods the class archive_options_extract. We will here restore all that is under usr/lib and only files which filename ends with ".a", we want libdar to skip file that would lead to overwriting an existing file and also have libdar display files that have been skipped from the restoration.

`archive_options_extract options; options.set_selection(simple_mask("*.a", true)); options.set_subtree(simple_path_mask("usr/lib", true)); options.set_allow_over(false); options.set_display_skipped(true); ret = my_arch->op_extract(dialog, "/tmp", // where to restore files to options, // non default options set just above` `NULL);// no progressive report used` | ```archive_options_extract options; options.set_selection(simple_mask("*.a", true)); options.set_subtree(simple_path_mask("usr/lib", true)); options.set_allow_over(false); options.set_display_skipped(true);` `ret = libdar::op_extract_noexcept(``dialog, my_arch, // the archive to test` `"/tmp", // where to restore files to` `options, // non default options set just above NULL,``// no progressive report used` ```exception,// this gives the status of the call except_msg); // and in case of error the // cause of the error`

last point about optional parameter concerns the set_what_to_check method. It serves two roles here: Which field are to be ignored when looking whether a file is more recent than the one in the filesystem (if this feature is enabled) Which field to avoid restoring (for example, when not having root privileges, avoid restoring ownership may be interesting instead of having a plethora of failure to restore ownership messages). ## ## 10 - Isolating the Catalogue OK, I know, catalogue is not an English word (one would rather write catalog), but that's the name of the C++ class used in libdar, so we will keep using it here. Note that you don't have to directly access this class (if you really don't like French). Isolating the catalogue creates a new archive that only contains the list of files and their attributes (ownership, dates, size, etc.), but no data and no EA are stored in it. It is very similar to the same archive one gets if one makes a differential backup of a filesystem that has not changed since the creation of a reference archive. The usage is very similar to the archive creation, but it uses a different constructor that has less arguments:

`archive_options_isolate options; // we just want to have a compressed isolated catalogue options.set_compression(gzip); libdar::archive *my_cat = new libdar::archive(dialog, "/tmp", // where the extracted // catalogue is saved my_arch, // the archive of reference // is the one we have been // playing with previously "my_catalogue", // slice name "dar", // file extension options) // non default options set above` | ```archive_options_isolate options; // we just want to have a compressed isolated catalogue options.set_compression(gzip);` `libdar::archive *my_cat = libdar::op_isolate_noexcept(dialog,` `"/tmp", // where is saved the // extracted catalogue my_arch, // the archive of reference // is the one we have been // playing with previously "my_catalogue", // slice name "dar", // file extension options, // non default options set above` ```exception, // this gives the status // of the call except_msg); // and in case of error the // cause of the error` `i` `f(exception != LIBDAR_NOEXCEPT) std::cout << "an error occurred: " << except_msg << std::endl;`

Now we have two archive objects. my_arch is a read-only object created by the "read" constructor. You can do any operations with it, like file restoration, file comparison, archive testing, as we have done in the previous sections. The second archive object is my_cat which is a write only object. It can only be used as a reference for another backup (a differential backup) or as a reference for a subsequent catalogue isolation (which would just clone the already isolated catalogue object here). Note that once closed (object destruction) you can re-open the isolated catalogue and use it as a read-only object (you can then test its integrity as seen previously). So for now we will just destroy the extracted catalogue object, so that all its file descriptors are closed:

`delete my_cat;` | ```close_archive_noexcept (my_cat, exception, except_msg);` `if(exception != LIBDAR_NOEXCEPT) std::cout << "an error occurred: " << except_msg << std::endl;``` ``

and we keep the my_arch object for our last operation: ## 11 - creating a differential backup This operation is the same as the first one we did (archive creation). We will just provide the archive of reference as an optional parameter. If we had not destroyed my_cat above, we could have used it in place of my_arch for exactly the same result.

`archive_options_create options; // we provide here the reference to an // existing archive object, this implies that // the archive will be a differential backup options.set_reference(my_arch); // as we are now used to options, we will set a more // complex set of other options: options.set_selection(not_mask(simple_mask("*~"))); options.set_empty_dir(true); options.set_compression(bzip2); options.set_compr_mask(not_mask(simple_mask("*.bz2"))); options.set_cache_directory_tagging(true); options.set_slice_permission("0600"); options.set_slice_user_ownership("root"); options.set_slice_group_ownership("bin"); options.set_crypto_algo(crypto_blowfish); // if not specified with set_crypto_pass() the password // will be asked interactively to the user options.set_slicing(100000000, 20480); libdar::archive *my_other_arch = new libdar::archive(dialog, "/home", // saving all under this "root" "/tmp", // where the slices will go "my_archive", // the basename of the slices "dar", // dar's slice extensions options, // the optional parameter as defined above``` `NULL); // no progressive report` | `archive_options_create options; // we provide here the reference to an // existing archive object, this implies that // the archive will be a differential backup options.set_reference(my_arch); // as we are now used to options, we will set a more // complex set of other options: options.set_selection(not_mask(simple_mask("*~"))); options.set_empty_dir(true); options.set_compression(bzip2); options.set_compr_mask(not_mask(simple_mask("*.bz2"))); options.set_cache_directory_tagging(true); options.set_slice_permission("0600"); options.set_slice_user_ownership("root"); options.set_slice_group_ownership("bin"); options.set_crypto_algo(crypto_blowfish); // if not specified with set_crypto_pass() the password // will be asked interactively to the user options.set_slicing(100000000, 20480);``` `` `libdar::archive *my_other_arch = libdar::create_archive_noexcept(dialog,``"/home", // saving all under this "root" "/tmp", // where the slices will go` `my_arch, // differential backup` `"my_archive", // the basename of the slices "dar", // dar's slice extensions` `options, // the optional parameter as defined above``` `exception, // thisgives the status of the call except_msg); // and in case of error the cause. if(exception != LIBDAR_NOEXCEPT) std::cout << "an error occurred: " << except_msg << std::endl;`

As previously, my_other_arch is a write only object that we won't need anymore. So we destroy it:

`delete my_other_arch;` | `libdar::close_archive_noexcept(my_other_arch, exception, except_msg);` `if(exception != LIBDAR_NOEXCEPT) std::cout << "an error occurred: " << except_msg << std::endl;`

We are at the end of this first part of the tutorial, where we have seen the general way to manipulate dar archives like dar command-line does. But we still have an object we need to destroy to cleanly release the memory used:

`delete my_arch;` | `libdar::close_archive_noexcept(my_arch, exception, except_msg);` `if(exception != LIBDAR_NOEXCEPT) std::cout << "an error occurred: " << except_msg << std::endl;`

For more detailed information about the API you can build the API documentation from the source code using Doxygen or get it online from dar home page or mirror site. ## 12 - Compilation & Linking ### Compilation All the symbols found in the libdar API are defined via <dar/libdar.h> so you should only need to include this header.

````

``> cat my_prog.cpp #include <dar/libdar.h> main() { libdar::get_version(...); ... } > gcc -c my_prog.cpp

### ### Linking Of course, you need to link your program with libdar. This is done by adding -ldar plus other library libdar can be built to use like libz, libbzip2, liblzo or libgcrypt :```` ``

```` > gcc -ldar -lz -lbzip2 -llzo -lgcrypt my_prog.o -o my_prog

### Libdar's different flavors Well, all the compilation and linking steps described above assume you have a "full" libdar library. Beside the full (alias infinint) libdar flavor, libdar also comes in 32 and 64 bits versions. In these last ones, in place of internally relying on a special type (which is a C++ class called infinint) to handle arbitrary large integers, libdar32 relies on 32 bits integers and libdar64 relies on 64 bits integers (there are limitations which are described in doc/LIMITATIONS). But all these libdar version (infinint, 32bits, 64bits) have the same interface and must be used the same way, except for compilation and linking. These different libdar versions can coexist on the same system, they share the same include files. But the LIBDAR_MODE macro must be set to 32 or 64 when compiling for linking with libdar32 or libdar64 respectively. The LIBDAR_MODE macro defines the way the "class infinint" type is implemented in libdar, and thus changes the way the libdar headers files are interpreted by the compiler.

````

``> cat my_prog.cpp #include <dar/libdar.h> main() { libdar::get_version(...); ... } > gcc -c -DLIBDAR_MODE=32 my_prog.cpp ```` > gcc -ldar32 my_prog.o -o my_prog

and replace 32 by 64 to link with libdar64. Note that libdar*.pc files are installed in the $(PREFIX)/lib/pkgconfig file that should simplify (depending on the point of view) all these operations. For example, if you have all different flavors of libdar installed, the $(PREFIX)/lib/pkgconfig dir will contain (among other files) the three following ones: libdar.pc libdar32.pc libdar64.pc Thus, if you want to build your application with libdar32 for example, you will have to call (assuming you have pkg-config installed)

``> gcc `pkg-config --cflags libdar32` -c my_prog.cpp ```` > gcc `pkg-config --libs libdar32` my_prog.o -o my_prog

## 13 - Aborting an Operation If the POSIX thread support is available, libdar will be built in a thread-safe manner, thus you may have several thread using libdar calls at the same time. You may then wish to interrupt a given thread. But aborting a thread form the outside (like sending it a KILL signal) will most of the time let some memory allocated or even worse can lead to dead-lock situation, when the killed thread was in a critical section and had not got the opportunity to release a mutex. For that reason, libdar proposes a set of calls to abort any processing libdar call which is ran by a given thread.

````

`` // next is the thread ID in which we want to have lidbar call canceled // here for simplicity we don't describe the way the ID has been obtained pthread_t thread_id = 161720; // the most simple call is : libdar::cancel_thread(thread_id); // this will make any libdar call in this thread be canceled immediately // but you can use something a bit more interesting: libdar::cancel_thread(thread_id, false); // this second argument is true for immediate cancellation, // of false for a delayed cancellation, in which case libdar aborts the operation // but produces something usable, for example, if you were backing up something // you get a real usable archive which only contains files saved so far, in place // of having a broken archive which miss a catalogue at the end. Note that this // delayed cancellation needs a bit more time to complete, depending on the // size of the archive under process.

As seen above, cancellation can be very simple. What now succeeds when you ask for a cancellation this way? Well, an exception of type Ethread_cancel is thrown. All along his path, memory is released and mutex are freed. Last, the exception appears to the libdar caller. So, you can catch it to define a specific comportment. And if you don't want to use exceptions a special returned code is used.

`try {` `libdar::archive *my_arch = new libdar::archive(...); ... } catch(libdar::Ethread_cancel & e) { ... do something when thread has been canceled; }` `` | ```` `U_16 ex; std::string msg;` `archive *my_arch = libdar::open_archive_noexcept(...,ex,msg); switch(ex) { case ... .... break; case LIBDAR_THREAD_CANCEL: ... do something when thread has been canceled break; case ... }`

Some helper routines are available to know the cancellation status for a particular thread or to abort a cancellation process if it has not yet been engaged.

````

pthread_t tid; // how to know if the thread tid is under cancellation process ? if(libdar::cancel_status(tid)) cout << "thread cancellation is under progress for thread : " << tid << endl; else cout << "no thread cancellation is under progress for thread : " << endl; // how to cancel a pending thread cancellation ? if(libdar::cancel_clear(tid)) cout << "pending thread cancellation has been reset, thread " << tid << " has not been canceled" << endl; else cout << "too late, could not avoid thread cancellation for thread "<< tid << endl;

Last point, back to the Ethread_cancel exception, this class has two methods you may find useful, when you catch it:

````

``try { ... some libdar calls } catch(libdar::Ethread_cancel & e) { if(e.immediate_cancel()) cout << "cancel_thread() has been called with "true" as second argument" << endl; else cout << "cancel_thread() has been called with "false" as second argument" << endl; U64 flag = e.get_flag(); ... do something with the flag variable... } // what is this flag stored in this exception ? // You must consider that the complete definition of cancel_thread() is the following: // void cancel_thread(pthread_t tid, bool immediate = true, U_64 flag = 0); // thus, any argument given in third is passed to the thrown Ethread_cancel exception, // value which can be retrieved thanks to its get_flag() method. The value given to this // flag is not used by libdar itself, it is a facility for user program to have the possibility // to include additional information about the thread cancellation. // supposing the thread cancellation has been invoked by : libdar::cancel_thread(thread_id, true, 19); // then the flag variable in the catch() statement above would have received // the value 19.

A last and important point about multi-threaded environment: An object like any other variable cannot be modified or read (with the use of its methods) without precaution from several threads at the same time. Care must be taken to avoid this situation, and the use of Posix mutex is recommanded in your program if you plan to let an archive object be accessed by more than one thread. See the FAQ for more about this point. ## 14 - Dar_manager API For more about dar_manager, please read the man page where are described in detail the available features. Note that for dar_manager there is not a "without exception" flavor, your program must be able to handle exceptions, which by the way are the same as the ones described above. To get dar_manager features you need to use the class database which is defined in the libdar/database.hpp header file so you first need to include that file. Most of the methods of the database class do use options. For the same reason as previously seen for archive manipulation, these options are passed thanks to a container class. These container classes for options used by the database class are defined in the libdar/database_options.hpp file. Let's see the different method of the class database : ### Database object construction Two constructor are available:

````

`#include <dar/database.hpp>` `void my_sample_function(user_interaction & dialog) { database base; // we have created an empty database (no archive in it) called "base" database other = database(dialog, "/tmp/existing_base.dmd", database_open_options()); // we have created a database object called "other" which contains // (in RAM) all information that were contained in the // database file "/tmp/existing_base.dmd" // I will explain below the last argument database_open_option opt; opt.set_partial(true); database other2 = database(dialog, "/tmp/existing_base.dmd", opt); // we have created a database object called "other2" which differs // from "other" in the option we used. While "other" is a fully loaded // database, "other2" is a partial database. This notion is explained // below }`

So far, this is not much complicated. You can build an empty database from nothing, or load a database to memory from a file using the second constructor. As you can see over the filename to give in this later constructor, we need a user_interaction object to be able to inform the user of any problem that could be met, and an object of class database_open_options. This last object contains options to use for this call (options are set to their default unless modified explicitely). Currently, the only available option is the "partial" option which is a boolean argument: In all the available methods for class database, some require to load the whole database in the memory while some other only require the database header. Loading just the database header is much faster than loading the whole database, of course, and as you guess it requires much less memory. While you can perform any operation with a full loaded database, only a subset of available method will be available with a partially loaded database. If you try a method that requires a completely loaded database, you will get an exception if the object you use has been loaded with "true" as last argument (called "partial") of the constructor, and of course an empty database (built with the first constructor) is a completely loaded database, so you don't have restriction in using a new database object. But now let's see the available method for that class: ### Database's methods First we will see methods that work with both partially and completely loaded databases: dump(...) : it is used to write back the database to a file. change_name() : change the basename of the archive which index is given in argument set_path() : change the path to the archive which index is given in argument set_options() : change the default options to always pass to dar when performing restoration set_dar_path() : specify the path to dar (use empty string to rely on the PATH variable) show_contents() : list the archives used to build the database get_options() : list the options that will be passed to dar (as defined with the set_options() method) get_dar_path() : return the path to dar (or empty string if relying on the PATH variable) Now let's see the database methods that only work with completely loaded databases: add_archive() : add an archive to the database remove_archive() : remove an archive from the database set_permutation() : change archive relative order within the database show_files() : list the files which are present in the given archive show_version() : list the archive where the given file is saved show_most_recent_stats() : compute statistics about the location of most recent file versions restore() : restore a set of given files given in argument. Well, you might now say that as description this is a bit light for a tutorial, yes. In fact these call are really very simple to use, you can find a complete description in the reference documentation of the API. This documentation is built if doxygen is available and is put under doc/html after calling make in the source package. It is also available from dar's homepage.

## Thanks I would like to thank Wesley Leggette and Johnathan Burchill for having given their feedback and having done grammar corrections to this document. Out of this document, I would also like to thanks them a second time for their work around dar and libdar (Johnathan is the author of kdar, the KDE front-end for dar). Regards, Denis Corbin.