Monitoring your websites progress

Monitoring your websites progress

In the physical world when someone walks into your organisation and looks around, asks questions and gets involved with whatever you are promoting, you are well aware of it. Online all this takes place with you knowing very little about it. Just a ghostly visitor crosses your path.

You can get acquainted with these apperitions by using a web stats package to analyse your logfiles.

Web stats; how is it possible?

Whenever information is requested on your website, the server needs to know who to return it to. The communication protocol that the web is built upon (HTTP, hypertext transfer protocol) states that whoever requests information should also send some other extra information. Two very useful ones for monitoring your website are what program (User-Agent) is requesting the information and who told them about the page they are requesting (Referer).

By logging what, when, who and how the visitor came to your website you can build up an impressive overview of activity on your website.

    Why use web stats?
  • Find problem areas creating errors
  • Track spam bots
  • Track hacking attempts
  • Track search engine referrals
  • Search terms used to find your site
  • Help make decisions on redesigns and new content

Early planning

Plan early if you are not using a standard server (Apache, IIS etc..) or if your site uses plug-ins such as Flash.

If you are hosting multiple websites make sure that your server logs each virtual host to separate log files, having to split up log files at a latter date is an annoyance and can add in errors.

Try and archive an adequate amount of data, this will vary depending on each project, I find a years worth come in very useful.

When deciding to analyse your log files decide what you want to know. Most programs will generate a mass of charts and statistics which can quickly become over whelming or lead you on a long winded browsing process with few revelations. What is it that you want to know? overall figures for that month? how many computers accessed a specific page last week? What search terms were used on search engines to find your site?

A cautionary example (hits, pages & robots)

Great, so you?ve just launched your new project and the webstats report shows 500,000 hits in your first week.

On closer inspection you find there are only three distinct IP addresses and that for each page request there are 30 other requests for images and linked files (javascript and css). Looking deeper you find that one of the scripts that generates your page links is self referencing and the two search engines have been drilling down into your site creating page after page. 500,000 hits equals one visitor (that was you) and 10 real page views (the ones you checked on Monday).