Users interested in analysis of large data files will find that Sophia, our UNIX host, is the best place to get the job done. No other platform can offer comparable performance. Users have access to Sophia from all our Novell networks, as well as via modem.

TSP can be run in two ways: interactively and batch mode. The latter is more efficient, and this document will discuss it in detail. Interactive mode is useful for data exploration, and will be discussed in brief at the end of this document.

Note: UNIX is case sensitive, so pico test.tsp is not the same as Pico Test.Tsp


Using TSP in Batch Mode

In batch mode you save a string of TSP commands, one per line, in a text file. Later you submit the batch file and the series of commands is executed, one after another. On Sophia, you can use the pico text editor to create this file.

To Create a Batch File:

1. At the % prompt, type: pico test.tsp <enter>

   This will invoke Pico, and you can begin typing TSP commands. A typical batch file looks something like:

   options limprn=80,leftmg=5,indent=0;

   in "data/tsp/nlsy";

   msd wage93;

   graph wage93 xpr;

   olsq wage93 c xpr hideg93 female blk hsp;

   end;

2. To save your changes, hold down the ctrl key and press o.
   
   
3. To exit Pico, hold down the ctrl key and type x.

Note: You should use the file extension .tsp for all TSP batch files. TSP expects batch files to have this extension, and will look for such files by default.


Some Common Batch Commands

There are some commonly used commands you may want to use when using TSP in batch mode.

If you want to print your output on a laser printer, you will want to set the width to 80 columns. The first line of your batch program would be something like:

options limprn=80,leftmg=5,indent=0;

Before you can do any data manipulations or statistical analysis, you must reference a data file. You can name a file on your own account, or any publicly readable file on someone else's account. If you plan to do a lot of data analysis, it is a good idea to create a data directory on your account. The document An Introduction to UNIX at Smith College includes instructions on how to create a directory. If you have a directory called data, you might include a file reference like this:

in "data/nlsy";

The command in is used with TSP databanks (TSP system files). If your data are in a text file with variables or series in columns and cases in rows, you would use the read command, as in:

read (file="data/nlsy.dat") var1 var2 var3 var4;

Note: The data reference must include the name of all of the variables (or series) in the data text file. Since TSP stores variable information in the databank, you do not need to name the variables with the in command.

TSP file references can use the ~ abbreviation. For example, to get to the NLSY databank in a directory on Tim Shortell's account, you would type:

in "~tshortel/data/eco280/nlsy.tlb";

You must supply the file extension when referencing data files in TSP.

Once you have referenced a data file, you are ready for data manipulation or analysis. Commands should end with a semicolon. If a command wraps onto a second line, indent it. File references should be enclosed in quotations.

If you do any data manipulation and want to save the changes, you should include an out command. This should precede the data manipulations, as in:

out "data/newdata.tlb";

logx=log(x);

ysq=y**2;

Note: If this is the same file that was referenced in the in or read command, the out command will write over the original version. It will include all of the variables in the original file and those created after the out command in the current job.

You can read in and write out multiple files. The number of files that can be held in working memory depends on the size of the files and the amount of workspace TSP allocates when it runs. For large files, you can explicitly ask for additional memory. The default memory size is 4 MB. The options command can specify the size of working memory, in MB. It must be the first line of the file. For example,

options memory=6;

This will allow TSP to allocate 6 MB of space for working memory.

The out command saves all variables created after it, until the next out command or the end of the file. To save all the variables in a file into a new file, the out command should come before the in command. Additional variables can be created and stored. To stop marking variables to be saved, use the out command with no file specification, as in:

out "data/data2.tlb";

in "data/data1.tlb";

x_1=x(1);

ysqrt=sqrt(y);

out "data/data3.tlb";

z=1;

q=sin(p);

out;

r=abs(s);

In this example, the variables x_1 and ysqrt are saved in data2.tlb but not data3.tlb. Variables z and q are included in data3.tlb but not data2.tlb. The variable r is not saved at all.


To Submit a Batch File:

Once you have finished creating your batch file, you must submit in TSP for it to run.

At the system prompt, type: tsp test &

Press Enter.

This command tells TSP to process the batch file test.tsp and send the output to test.out. The last character, &, tells UNIX to run the job in the background. This way, you can go on working while the file is processed. Of course, Sophia is so fast that it will probably return the output before you would have the opportunity to get started on anything else!

The system will prompt you when the job is finished.

{sophia} stats/tsp % TSP output will be in: test.out

Done tsp test

{sophia} stats/tsp %

Use Pico to view and/or edit the output file, as in:

pico test.out <enter>

to examine the results of test.tsp. Unless you specify otherwise in the tsp command, the output will be sent to the current working directory.


To Print Files:

Once you are satisfied that your results are in order, you can send the output file to a printer. Many networked printers have a queue on Sophia. To see the available printers in the Jahnige Center, type:

printers jahnige <enter>

This will produce the names of all the print queues for Jahnige Center printers:

Printer Name: jahnige_hp

Aliases: j_hp

Description: HP LaserJet in Jahnige Resource Center (Wright Hall)

Printer Name: jahnige_ps

Aliases: jahnige_postscript, j_ps

Description: Postscript printer in Jahnige Resource Center (Wright Hall)

Printer Name: jahnige_epson

Aliases: jahnige, j

Description: Epson printer in Jahnige Resource Center (Wright Hall)

You could specify Bass, Seelye, or faculty (for faculty printers), and so forth.

You specify a particular printer with the -P switch; for example, to print to the Jahnige Center postscript printer, you would type:

lpr -Pj_ps example.out <enter>

You can also print batch files. You should avoid printing large data files.


TSP Naming Conventions

TSP uses the following naming conventions. Databanks, or system files - binary files with data and formats combined - have the extension .tlb. Syntax files, or batch files, have the extension .tsp. Output files have the extension .out.

UNIX is case sensitive, so test.tlb and Test.tlb are different files. If you receive the following error message,

NOTE: TSP databank not found: /home/x1/tshortel/data/nlsy.tlb

Check to make sure that the file reference matches the name of the file exactly. You can check the file names with the ls (list) command:

ls *.tlb <enter>

This will display all of the files with the extension .tlb in the current directory.

Unlike DOS/Windows, filenames may be longer than 8 characters. The following are valid file names:

ProductData0.tlb ProductData3.tlb Exports96.tlb
ProductData1.tlb ProductData4.tlb Exports2Ireland.tlb
ProductData2.tlb ProductData5.tlb
 

Using TSP in Interactive Mode

When running an interactive session of TSP, commands are executed as they are issued, one at a time, and the results written to the screen. This is useful for data exploration.

To Invoke TSP:

At the system prompt, type tsp <enter>

The system will respond:

 Enter batch filename [or press Enter for interactive]:

Press Enter to get into interactive mode. You will see the TSP logo: 

TSP Version 4.3A

05/06/96) Sun/Sol f2c 4MB

Copyright (C) 1996 TSP International

ALL RIGHTS RESERVED

09/13/96 6:04 PM

In case of questions or problems, see your local TSP

consultant or send a description of the problem and the

associated TSP output to:

TSP International

P.O. Box 61015, Station A

Palo Alto, CA 94306

USA

Enter TSP statements:

1 ?

The prompt will change from the system prompt (%) to the TSP prompt (?). You can now enter any valid TSP commands.

When you hit the enter key, after typing a command and the semicolon, TSP immediately executes the command and writes the output to the.


To Save Output in a File:

Often, when running an interactive session, you will want to keep a copy of the results in a file as well as view them on screen.

At the system prompt, type output filename to save the output in a file, as in:

? output "output/session1.out";

TSP will respond:

TSP output will be in: output/session1.out

To start, you will need to reference a data file, with the in or read command.

When you are finished with your interactive session, type end;

The system will respond by returning to the system prompt.


Getting Additional Help

A full set of TSP users guides are available in the Jahnige Center. Users can also consult the Jahnige Center statistician, x3043, for assistance with TSP on Sophia.