100% CPU Usage

Good day,

Although I still have some cleanup to do on the "tracking" script, I would estimate the "logging" required code is ranging in the 40kb, not 5Kb as you suggested.

Perhaps you looked only at the top part of the script where the LogFile() function is, but you need a little more than that, since you need to be able to set the class properties, and some other functions that are called from LogFile().

Globally, you're absolutely right, the script need to be "splitted" according to the usage, keeping the logging and reporting as two separate items/sections.

The very first day I tried MetaTraffic, with the supplied configuration, I noticed a significant increase in the time required to load each tracked page.

I then modified my scripts to accomodate that "split" along with a major modification to the database structure. Really worthwhile!

The site I'm testing MT on is not really busy, not to say very quiet, but regardless, even if I'm using MsAccess as my backend, the response time is more than acceptable.

With an average of 12,000 hits a day, I sincerly hope you're using MsSQL as your backend cuz I sincerely doubt MsAccess would be able to handle the peek times adequately.

As of now, I never had more than 7 users at once on the site, so this is not really an issue for me. You should consider to track this information on your end too, and if you have multiple periods where you have 15+ users at once, I would strongly suggest you to move over to MsSQL, if this is not already done.

So two major items to consider ; splitting the scripts to minimize the load time, then evaluating the concurrency level.

Regards.

I created a very small class file that only does logging. It's about 2-3Kb. I also created a SQL stored procedure to insert the log data into the database. Though I can't say I've stress-tested it, I can see that my pages are only taking an additional 10-20 miliseconds to execute compared to the time they took before adding the logging code.

I only picked up the script the other day out of curiosity and I think I will stick to mainstream stats logging because of the linitations but I liked the code and am using it on a small school site (20 pages per day!) with great results. Thanks for the advice.
Frank

This is definately something that has been taken into consideration for version 2.0 of MetaTraffic. The class file has now been split into two: one for logging, one for reporting. The logging class is done and functional (although not fully tested) and is 22kb in size. The logging code has been optimized significantly, although there are now more DB inserts because of the normalization on the database. On a Sql Server test platform, I have seen significant improvements in logging performance and db space utilizations. However, I have yet to test extensively with Access or mySql.

Regards,
~Chad

Good day,

If we limit the logging script to strictly log, no more no less, then indeed, this will result in a very small script.

For instance, determining the OS, Browser Brand and the browser type, is done with only one piece of information, the UserAgent.

If we want to optimize the logging process, we can move that information seggregation in the reporting script, although there will be a need to determine if there is a real need to refresh the information prior to reporting.

In that case, with proper normalization, you may have 1000 different useragent strings, covering 100,000 records in the pagelog table, therefore, updating 1000 records is much faster than 100,000.

In fact PageLog has a pointer to the UserAgents table, which in turn has pointers to the Browsers, OS and Browser types.

Depending how busy the server is, on the test site I have it takes roughly 5 seconds to update the 780 useragents and related tables.

Also, I "included" some external maintenance scripts, which are directly accessible from the reporting listbox when logged on as an administrator. At times we may need a one shot deal thing, and that approach suit my needs perfectly for that matter.

Anyhow, good too know that you still alive Chad. You were awefully quiet during the last few weeks/months.

Cheers.

Been busy coding. I assume people still want version 2? :)

~Chad

Most definitely ! :D

Absolutely!!!

Don't get discouraged Chad. I know I am still looking forward to and patiently inticipating version 2.

TechStud ,
[url="http://www.techstud.com/?MetaSun"]www.TechStud.com[/url]

quote:Originally posted by fbridge
I have installed the software and CPU usage rose from 30% to 100% solid and the site slowed to a crawl. I get about 12000 page views per day which equates to about 1 page every 4-5 seconds.

Frank

Frank,

Are you running on a particularly busy or slow server? I've tested at rates of 20,000 hits an hour with no observable slowdown on a dual Xeon 2.8. You mention that your server is typically at 30%. That's quite high for a static web server, and should correspond to the load at peak usage of a dynamic server. Perhaps there are other processes you can turn off? Sometimes an interfering process doesn't need to be consuming much CPU to affect the web service (for example, if it's doing much with the hard drive or moving info in RAM), so when your web service needs the CPU it has to wait for the data bus to clear.

You might also consider using the javascript method of logging, as that is done after the web page has beed delivered.

PLease keep in mind that these are things to check after you have trimmed the logging class down to the bare essentials if you still have performance problems.

Looking forward to V2! :)

Haha - you would laugh if I told you it was running on a 200MHz PII 128MB RAM using Access as the backend. Just shows how good MY coding is! Recently upgraded now.

Any news for V2?

Frank