Previous       Contents       Next

nifxref: Cross-Reference Report

This program post-processes the output of nifpan, and is a tool to help cross-reference two different snapshots of the same packet sequence, as viewed from two different hosts. The nifpan text files being used must have been generated with -v2 or -v3 (ie. detail levels 2 or 3).

The input to nifxref is typically generated by running nifpan against niftap log files created on two communicating machines. The two nifpan text files thus created will therefore contain incoming and outgoing views of the same packets, ie. for a given packet, one of the text files will see it being transmitted, and the other will see it being received.
To run nifxref, you arbitrarily nominate one of the nifpan text files as an Input and the other as a Reference and nifxref will use the latter to rewrite the former to its standard output, such that for any Input packet which also appears in the Reference file, its entry is replaced by a cross-reference to the corresponding event in the Reference file.

Command-Line Syntax
nifxref [-a|-s|-d host[:port]] [-u] [-f] [-e event-id] input-file ref-file
nifxref -h

-a Only IP packets whose source or destination address matches the address spec will be cross-referenced against possible matches in the reference-file (ie. other input-file events will be transcribed as is, whether a match exists for them or not). The port spec is optional, and if present, is delimited from the host part with a colon.
The host address and TCP/UDP port must be specified in the same format (and case) as in the input-file, eg. if the text file contained numeric IP addresses, and the -s option specified a symbolic host name, then nothing would match.
 
-s Similiar to -a, except only the source address of input-file packets is compared to the address argument. This is therefore a more restrictive filter.
 
-d Same as -s, except this specifies a destination address.
 
-u This specifies that any port spec in the address filter refers to UDP. Otherwise, the port spec is only used to match TCP packets.
This is meaningless, if no address filter is specified.
 
-e This specifies the event number in the reference-file at which to start looking for matches. By default, we scan the Reference file from its beginning (ie. Evt=1).

Background: nifxref uses the 16-bit IP datagram-id to correlate packets in the Input and Reference files, but this can wrap around within as little as 10 seconds during sustained bursts (eg. FTP'ing a large file over a LAN).
Therefore, if the Reference file commences at a significantly earlier time than the Input file, and records a large number of packets, then it's possible that this earlier traffic will contain spurious matches for Input events.

Hence this switch can be used to guide nifxref to the (approximate) area of interest in the Reference file, for the first Input packets which ought to have matches. Once nifxref locks onto the first true match in the Reference file (guided by the -e switch if necessary), it uses the observed clock skew to select the correct match for all subsequent Input packets, even if multiple Reference packets have the same Datagram ID.
 
-f This switch asserts that none of the Input packets will be fragmented, when observed in the Reference file.
This condition is generally true anyway (and always true over a LAN), and asserting it helps to minimise false matches in the Reference file, due to the same datagram-numbering wraparound discussed in the -e switch above.
In this case, the false-match scenario being avoided is one where no genuine match exists for an Input event, but as nifxref scans further down the Reference file, it finds another matching datagram ID within the permissible time window. Asserting no fragmentation allows us to minimise such false matches by tighten the criteria for a match, by checking datagram size as well as ID.
In practical terms, this switch can be used as an additional safeguard in combination with -e, to weed out false matches caused by jitters in the IP datagram numbering (which do sometimes occur). Provided datagrams are numbered incrementally by the IP software, the -e switch should be all nifxref needs to identify false positives ... but you can never be sure that datagrams were numbered as expected.
 
-h Display a usage synopsis.
 
input-file This is the nifpan text file which is going to get rewritten on our standard output, with some of its events replaced by a reference to the reference-file.
 
ref-file This is the nifpan text file which is used to cross-reference events in the input-file, so it determines which events in the input file will get rewritten, or which will simply be transcribed without change.

In the absence of an address filter (ie. the -a,-s,-d switches) every packet in the input-file is cross-referenced against the reference-file, to see if a match exists.

Note that nifxref is only capable of cross-referencing IP packets. Other events are always written out to standard-output unchanged.

Example Session

 Assume that niftap has been run on two communicating machines, Alpha and Beta, generating the log files alpha.log and beta.log respectively.
The niftap log files are then processed by nifpan -v2 (or -v3) to generate alpha.txt and beta.txt.

$ nifpan < alpha.log > alpha.txt
$ nifpan < beta.log > beta.txt

We then run nifxref to rewrite events in alpha.txt as references to events in beta.txt, ie. alpha.txt is the input file, and beta.txt is the reference file.
We restrict the affected events in alpha.txt to those that represent incoming packets from Beta.

$ nifxref -s beta alpha.txt beta.txt > alpha.ref

A fragment of alpha.ref might now look like this:

- Evt=1 (490.719164): ETH; IP: epsilon->alpha , id=18504.0; UDP: 1016->NFSD; data=100 [RPC C=1001614387, nfs.2.GetAttr]
- Evt=2 (490.721217): ETH; IP: alpha->epsilon , id=8031.0; UDP: N FSD->1016; data=96 [RPC R=1001614387, status=0,0]
- Evt=3 (664.994255): Ref=17
- Evt=4 (665.124329): ETH; IP: alpha->beta , id=57635.0; UDP: NFSD->127 ; data=96 [RPC R=140196425, status=0,0]
- Evt=5 (665.126762): Ref=18
From this, you can see that events 1 and 2 refer to packets exchanged between Alpha and a third machine, and so are not affected, ie. they are rewritten in an unchanged form.
Event 4 is a packet sent to Beta and so is also unaffected, due to the source filter in the nifxref command-line.
Events 3 and 5 must have referred to packets that originated on Beta and were seen by Alpha (whether they were addressed to it or not), so they have been rewritten as references to events 17 and 18 respectively, in beta.txt.

If packets were fragmented en route, then one outgoing packet in the reference file can result in multiple Ref rewrites, so these entries are distinguished from each other by specifying their datagram offset, and the length of their payload data. This info would be appended to the Ref= text in the output file, in the following format, with the Reference event's offset and data-length values shown in parentheses.
Ref=37 - FRAG: off=0 (1500), data=2000 (500)

You might now want to rewrite beta.txt as well, to complete the picture.
$ nifxref -s alpha beta.txt alpha.txt > beta.ref

This extended example shows how nifxref can cut out the hours of painstaking labour that would have otherwise been required to manually correlate related events in alpha.log and beta.log with each other, an effort which would have to be repeated after each new niftap packet capture session. Using nifxref instead will quickly highlight lost, duplicated and corrupted packets.

Previous       Contents       Next