Home     About PeterBlum.com     Policies     Download a Licensed Product     Newsletter
Traffic Monitor  
Current Release: 1.0.1  Release History  |  Get this Update
Jump to: Technical Overview | Download

TrafficMonitor is a framework that generates a web traffic log file much like IIS does but gives you complete control to what goes into the logs. It eliminates unwanted records, provides more recognizable data, and reduces the size of each record. In addition, it can record any event you want into the log, such as when a button is clicked or an ad is displayed. It can output data in delimited and W3C Extended File Format or you can subclass to provide output in other formats including directly writing log entries into a database.

This system is only a part to a complete traffic reporting system. It creates the log data. You still need a traffic reporting program like Microsoft Commerce Edition 2000 or WebTrends to turn the data into a reports.

The IIS Logs have long been used for traffic reporting. They can collect data about every message passed into the web server. This has advantages such as being able to collect data on static pages that don't run scripts, such as raw HTML files.
However, IIS Log files contain many things that make extra work when using them for traffic reporting.
  • Numerous records that don't relate to pages. Examples: internal files with 'js' and 'exe'. If you fail to filter these out, they will greatly increase the size of your web reporting data storage and slow down reports.
  • Real pages that don't apply to your reporting needs. Example: pages used only by your internal staff may need to be omitted if you are only tracking external user visits
  • Pages designed for your external users may still need to be omitted when hit by internal staff to eliminate the discrepancy. If you could detect the source IP Address or the login attributes are associated with internal staff, you could filter it out.
  • Pages designed for your external users may not be of any value on a traffic report. Example: Login page.
  • Events other than page loads should be able to track themselves. Examples include Ad Placement, Ad Clicks, Poll submissions, and other form submissions. IIS cannot capture this kind of data. However, the logs could easily store it if the event could generate it.
  • Page names are paths. Marketing departments who use these reports relate to pages better by their onscreen name instead of the page path. Need some way to translate \dir\dir2\productInf2.aspx to "Product Info".
  • Suppose you have a single web page that displays different database records. Users want to know the user friendly name of that record instead of an ID found in the URL parameters. They also want to treat each record as a separate hit on that page. For example, the URL may be \dir\dir2\productInf2.aspx?ID=1234 but users want to see the onscreen name: "Product Info: Super Widget"
  • Logged in users are not tracked when using custom security models. Thus traffic reporting by username are unavailable.
  • When tracking reports by users, their login name is hard to relate to the actual user. Report users want to see the real person or company. For example, login "JKL2234" could be replaced by "Homer Simpson, Burns Power Co."
  • One referrer may have several different URLs or domains. Need a way to map to a common URL to allow Referrer reporting to group them together.
  • Pages contain parameterized data in cookies. Instead of keeping the entire cookies list on every log entry (which can take up space), only selected cookies can be kept. If those you want to keep are actually made into URL parameters, it simplifies things for some traffic report products.
  • If your server has multiple sites that share a domain, you may want to segregate log entries into separate databases. IIS puts it all into a common file.

The TrafficMonitor framework generates logging information that:
  • Eliminates unwanted hits
  • Provides more recognizable data
  • Cleans up some of the data (like referrers)
  • Reduces the shear volume of data on each hit

Technical Overview

The TrafficMonitor is a framework of three main classes:
  • TrafficMonitor - Your pages interact with this class to add entries to the log. It has methods to add page and non-page events (like ad clicks).
  • TrafficLog - packages data about one entry for the log. Includes fields like page name, time, referrer and URL parameters. Can be subclassed to track additional elements.
  • TrafficOutput - collects the TrafficLog entries and writes them out. This abstract class is subclassed to define the format for writing. This package includes subclasses for W3C Extended Format and delimited text files. You can subclass for other text file formats and to connect to a database.
In the following sections, you will see how to install TrafficMonitor into your code. These examples show the simplest cases.

Setup requires creating the TrafficMonitor and TrafficOutput subclass in your Global.asax file:
protected void Application_Start(Object sender, EventArgs e) 
{ // Create the W3C Extended Format TrafficOutput class
   W3CExtendedFileTrafficOutput vTrafficOutput = 
      new W3CExtendedFileTrafficOutput("C:\\inetpub\\wwwroot\\MySite\\LogFiles");
   // Initialize various properties of vTrafficOutput here
   
   TrafficMonitor vTrafficMonitor = new TrafficMonitor(vTrafficOutput);
   TrafficMonitor.fCurrent = vTrafficMonitor;   // make it global
   // Initialize various properties of vTrafficMonitor here.
}  // Application_Start()
Pages record themselves into the log. This allows the page to filter itself out and customize the information such as providing a descriptive page title.
using TrafficMon;
private void Page_Load(object sender, System.EventArgs e)
{
   TrafficMonitor.fCurrent.RecordPage(this, "");
}
Events record themselves into the log this way:
using TrafficMon;
private void AnyEventHandler([event parameters])
{
  TrafficMonitor.fCurrent.RecordEvent(this.Request, 
      "An_Event_Name", "", "", null);
}

Download

Product Traffic Monitor
Version 1.0 (Current release: 1.0.1)
Price Free
Restricted Usage Read the License Agreement
Source Code Included
Language C#
Compatibility Microsoft .Net 1.0 and 1.1

Download this file