QuickStart Guide - Importing Data

## 3.3 Importing Data

Of the data formats that Data Explorer can import, the General Array format is likely to be the most useful to a majority of users. Other formats are discussed in detail in Appendix B. "Importing Data: File Formats" in IBM Visualization Data Explorer User's Guide.

The General Array format uses a simple header file to describe data characteristics such as grid dimensions, data type, and layout. The two examples here use the Data Explorer Data Prompter to create such header files for two simple data files. The Data Prompter is designed specifically for importing data in General Array format. (For details of format and the Data Prompter, see Chapter 5. "Importing Data".)

### Example 1

This example illustrates the importation of a simple data set consisting of a single variable on a 5 × 5 × 5 grid.

1. First enter:

       dx -prompter

   You can also access the Data Prompter by choosing `Import Data` from the Data Explorer Startup window.

   When the initial dialog box appears (see Figure 15), choose Grid or Scattered File, and then type the path name `/usr/local/dx/samples/tutorial/external.data`. into the `Data file name` field at the top of the dialog.

2. Press the `Describe Data` button to bring up a window which allows you to describe the data.
3. You can now view the file by clicking on the ellipsis button (`...`) to the right of the file-name field and selecting `Browser` from the pull-down menu. The `File Browser` window appears. (If necessary, move this window so that you can view it and the Data Prompter window at the same time.) It is also possible to browse the data file from the initial prompter window once the file name has been entered.
4. The first three lines of the file may look like data, but they are header information, so click on the `Header` toggle button and then on `# of bytes` to the right (now activated).
5. Select `# of lines` and enter the value "3" in the associated field.
6. The first line of the header gives the grid dimensions as `5 5 5`, so enter these numbers in the first three `Grid size` fields. Note what happens in the `Grid positions` fields as each number is entered. (You can use the tab key or the mouse to move the cursor from one field to the next.)
7. For `Data format`, ensure that `ASCII (text)` is selected.
8. Next you need to specify whether the data is in row or column majority order. In this particular file, the data is in row majority, so select that button.
9. Position the cursor in the first `origin, delta` field, hold down the left mouse button, and drag the cursor over the numbers there to highlight the field before releasing the mouse button. The default values and the highlight bar will disappear as soon as you start typing.
10. The origin of the grid is [1.0, 3.0, 2.0] (second line of the header) and the corresponding deltas are .5, .3, and .8, respectively (third line). So enter the values:

              1, .5
              3, .3
              2, .8

    in the three `origin, delta` fields.
11. On the right side of the prompter window, you can change the name of the data variable (`field0` by default) and specify the data type and structure. For this example, the data are floating-point scalar, so you do not need to change the settings.
12. You can now save the header file you have defined. Select `Save As...` from the `File` pull-down menu. Save the file under any name you choose.

    Note: The data can now be imported by specifying this file name to an `Import` module. However, if the extension is not "general," you must specify "general" in the "format" parameter field of the `Import` configuration dialog box (see "Controlling Inputs: Configuration Dialog Boxes").

13. In the initial Data Prompter window, note that the `Test Import` and `Visualize Data` buttons are now enabled. First choose `Test Import`. A window appears, displaying a description of the imported data. Now choose `Test Import`. A general purpose visualization program will be run on this data set. To view the program, choose Open Visual Program Editor from the Windows menu of the Image window. You can also experiment with changing the interactor settings in the control panel.

### Example 2

In this example, you will use the initial dialog box to customize the Data Prompter before importing a data file that contains scattered data values for two variables. The organization of the file is:

        x, y, data1, data2
        x, y, data1, data2
        . . .

where `x, y` define the positions (or locations) of the data. (See also "For Future Reference" later in this example.)

1. First, invoke the Data Prompter (as in Example 1, Step 1). When the initial dialog box appears, click on the `Grid or Scattered File` button for scattered data. Then click on the `Grid Type` button for scattered data (farthest right).
2. Use a stepper button to set `Number of variables` to "2."
3. The data positions are specified in the data file itself, so activate the `Positions in data file` toggle.
4. The data positions are 2-dimensional (x,y), so use a stepper button to set `Dimension` to "2."
5. Verify that the `Single time step` toggle is activated.
6. Set `Data Organization` to `Columnar`.
7. Click on `Describe Data`. The simplified prompter window appears.
8. Position the mouse cursor in the `Data file` field at the top left and type in the path name `/usr/local/dx/samples/tutorial/spreadsheet.data`.

   You can view the file by choosing `Browser` (as in Example 1).

9. There is no header in this file. Set `# of Points` to "49."
10. To save the header file you have defined, select `Save As...` in the `File` pull-down menu. Save the file under any name you choose.
11. The `Test Import` and `Visualize Data` buttons in the initial Data Prompter window are now enabled. Choose `Visualize Data` to see a visualization of this scattered data.

Many other examples of using the General Array format can be found in 5.1 , "General Array Importer". Data Prompter options are described in 5.4 , "Data Prompter".

For Future Reference
It is important to note that the top-to-bottom order of items in the `Field list` (right side of window) is the same as the left-to-right order of items in the data file itself:
    locations     x, y
    Field0        data1
    Field1        data2
    ...           ...
("locations" is a General Array reserved word used to indicate when numbers in a data file are to be interpreted as "positions"). The order of the field list can be changed (with the `Move field` stepper buttons) to agree with the order in the data file. If other information (e.g., descriptive text) is interspersed among the data values, you must use the layout options available in the full Data Prompter (see Layout and "layout").

[ OpenDX Home at IBM | OpenDX.org ]