ChangeLog for ModLogAn Last added: $Date: 2004/08/27 20:10:00 $ -- TODO ** for 0.8.x * o update the documentation for the splitter which is a dawn cool feature but noone really knows how it works * + enhance the splitter (combining state files) * o (o/modlogan) add a report for delete files for the wuftp parser * + use the login field * o create 'all' subdirectorys used by the splitter * o check http://usableweb.com/topics/000649-0-0.html (SwahaNaga@aol.com) * o fix singular/plural -> 20 von 87 Fehlende Datei / Kaputter Link -- 0.8.13 - - input - clf add support for negative timezone if format is specified - glue-code disable debug code - output - modlogan fix segfault -- 0.8.12 - - bugs - symlink points to it self (Thomas Seifert) - global - reduced memory usage by 20% (ostborn) - the data-libs are now compiled staticly to the binary - input - clf added %D (chrullrich) -- 0.8.11 - - output - template - added 'full reports' with all entries (ostborn) - added a new 'hosts by traffic' report (ostborn) - modlogan - use vhost-name from the splitter instead of the hostname set by the configfile for the output (ostborn) patch by Tomasz Lemiech - input - clf - tightend the regex the check the format of the timestamp before evaluating it (ostborn) - cp_ims_smtp, cp_ims_login - added parsers for some special smtp-servers (ostborn) - glue-code - added largefile support to mio (ostborn) - fixed the long-standing 'i-forgot-my-history' bug (ostborn) patch and full analisys done by Tomasz Lemiech - updated gettext to 0.12.1 -- 0.8.10 - 19.09.2003 09:52 - glue-code - fixed an endless loop in combination with read-ahead-limit (ostborn) - input - mod_log_sql - added a new plugin on request of DESC: parser for mod_log_sql (http://www.grubbybaby.com/mod_log_sql/) (ostborn) - ippl - added support for ipmod (courtesy of Ed Tanzer (miham) - clf - fixed ip/hostname handling in format handling (ostborn) - fixed segfault in the url-parser (ostborn) - postfix - added 'year' to specify the year of syslog-based logfiles (ostborn) - added handling of wrap-arounds at the end of the year (ostborn) - msiis - fixed the 'a-space-in-front-of-the-regex' bug reported by wladimir. (ostborn) - various plugins - fixed non-critical mem-leaks which would be closed at the end of the programm anyway (ostborn) - processor - web - fixed seg-fault in process_searchengines() (ostborn) - output - template - improved internal memory handling a little bit and closed 2 mem-leaks (ostborn) - rewrote the parser for template variables to improve the performance (ostborn) - using a small cache for a often used template should improve the memory-usage and the performance (ostborn) - i18n - fixed several untranslateable messages (miham) - created new locale (hu) (miham) - configure - improved the check for bind_textdomain_codeset() (ostborn) -- 0.8.9 - 08.07.2003 10:40 - output - modlogan - minor XML-conformance patch (ostborn) [Robin Ericsson] - text - minor reorganization, added ippl data support (miham) - processor - ippl (new) - DESC: processor plugin for IPPL data (miham) - glue-code - mio - improved handling of broken (binary) files (ostborn) [Malte John] - resolver - fixed compilation (ostborn) -- 0.8.8 - 17.05.2003 15:43 - processor - web - fixed is_page() (ostborn) - glue-code - moved the resolver code to the buffer-functions (ostborn) [patch from Michael Coulter ] - input - ippl (new) - DESC: input plugin for IP Protocol Logger (miham) -- 0.8.7 - 01.05.2003 18:12 - processor - web - fixed broken page and visit count (ostborn) - glue code - warn user about broken modlogan.statefile (ostborn) -- 0.8.6 - 18.04.2003 20:26 - glue-code - add the buffer-function from lighttpd (ostborn) (reduces memory usage by recycling old buffers) - removed usage of pcre_get_substring_list in substitude() (ostborn) - mrecord is using buffers now and mrecord_reset() just sets the used size of the buffer to zero instead of freeing the buffer (ostborn) - input - flow, elmeg, hicom - not supported anymore (ostborn) - all plugins - added gzip/bzip2 support (ostborn) - improved buffer handling (ostborn) - msmedia, msiis - fixed default time/date handling (ostborn) - rsync - (??) - processor - web - reduced memory fragmentation/usage (ostborn) - output - template - reduced memory fragmentation/usage (ostborn) - sybase_guj, tkcontrol - not supported anymore (ostborn) - documentation - added an example of a modlogan setup (erich) -- 0.8.5 - 14.01.2003 11:43 - glue-code - fixed bzlib detection (ostborn) - minor patches to clean the build under cygwin (ostborn) - if a line is longer than 4096 bytes the internal buffers are resized (ostborn) - output - template - fixed the mail-reports to used the 'new' template scheme introduced some releases ago (ostborn) - input - qmail - added gzip support (ostborn) - datatype - match - handle broken regexes (ostborn) -- 0.8.4 - 14.01.2003 12:42 - glue-code - fixed the directory check to handle the 'I'm root' case (ostborn) - fixed the mio-wrapper for lines longer than 1024 bytes (current limit is 4096 bytes for a single line) (ostborn) - fixed a seg-fault in url_decode() (ostborn) - datatypes - traffic, visit, hist, location - those datatypes became pointers in mdata (ostborn) -> results in less memory usage (ostborn) - string - added this lightweight datatype (ostborn) - all plugins - code cleanups and simplifications (ostborn) - added the changes introduces by the mdata-change (ostborn) -- 0.8.3 - 04.01.2003 18:01 - glue-code - reduced the memory-consumption of url_decode and url_encode (ostborn) - reduced the memory-fragmentation caused by mstate_characters (ostborn) - added a IO-wrapper which transparently handles uncompressed, gzipped and bzipped logs (ostborn) - added support for setting config-options over the command-line interface -> -o
:= (ostborn) - input - clf - added gzip support (ostborn) - referrer is including the get-parameters now (ostborn) - ported to mio - squid - ported to mio -- 0.8.2 - 27.12.2002 01:32 - global - fixed spec file (ostborn) - fixed the checks for the right of the statefile-directory (ostborn) - added a man-page (blindman) - processor - web - added localizer support (ostborn) - output - template - added 3 localizer-based reports (ostborn) ( include = output_template_localizer ) - input - realserver - fixed segfault in the url-parser (ostborn) - clf - added support for %T (duration) and %a (remote ip) (ostborn) - rsync - added parser -- 0.8.1 - 17.11.2002 10:59 - global - add ./doc/Changes-0.8.0.txt to the distribution (ostborn) - fixed the resolver (ostborn) - added strndup() and memrchr() fallbacks (le_zas) - fixed issues concerning timezone handling (le_zas) - input - clf - fixed format= handling (ostborn) -- 0.8.0 - 05.11.2002 12:43 - global, datatypes - rewrote the xml-parser for state and history (ostborn) - faster, less code, easier to understand - rewrote the xml-naming-scheme (ostborn) - the xml is now wellformed and VALID (http://jan.kneschke.de/projects/modlogan/mla.state.dtd) - automatic update of older statefile (0.7.x) - seperated statefile-dir from outputfile-dir (ostborn) - every output-plugins has "outputdir" - [global]-outputdir was replaced by "statedir" - added global-variables (ostborn) - var(hostname, jk.123.org) - drowngraded to gettext 0.10.40, automake 1.5x (ostborn) - added support for expat as the new default xml-parser (ostborn) - added the option -u the run modlogan under a given UID (le_zas) - added the a sybase_guj output plugin (ostborn) - removed the --enable-build-special option (ostborn) - cleanups and optimizations in misc.c and resolver.c (le_zas) - french translation was updated (le_zas) - updated INSTALL and README (ostborn) - updated modlogan.conf-dist (ostborn) - added russian translation (Ivan Pouzyrevsky (Sandello)) - relaxed the whitespace-handling in the config-file parsing (ostborn, le_zas) - optimized mhash_insert_sorted() (ostborn) - optimized strmatch and substitute (ostborn) - input - clf - rewrote the LogFormat parser to be more flexible (ostborn) - we can now handle mod_gzip logs - stricter matching for protocol part of request (ostborn/le_zas) - absolute URIs requests are more elegantly handled (ostborn/le_zas) - added timezone handling (ostborn) - should fix the 'my logs only contains records for august, but the report puts them in september' bug-reports - rewrote the internal parsers for timestamp, url and referrer to use plain ANSI-C instead of a simple regexes (ostborn) - doubles to speed of the plugin (7000 rec/s -> 13000 rec/s) - processor - web - added hide_hostmask (ostborn) - re-added host-count for unresolved ips (ostborn) - replaced the whitelist-like handling of the searchengines by a blacklist approach (ostborn) - handle the "not-yet-handled-visits" before the output-generation is started (ostborn) -> processor_web has to be loaded be any output-plugin is loaded !! - the regex for the default extension is now a little bit more stricter (ostborn) - better check of response codes to determine ressource existence (le_zas) - referrer report was fixed (searchengines were ignored) (ostborn/le_zas) - visit duration report was fixed (abnormally long duration) (ostborn/le_zas) - two new options to ease group.searchengines.conf updates: log_ungrouped_searchengines (0|1) log_ungrouped_searchengines_file (absolute or relative to statedir filename, default to ungrouped_searchengines.txt) (le_zas) - searchstrings are now url-decoded (%3A -> ":", + -> " ") (le_zas) - new option hidebrokenlinksref that permit to hide broken links by referrer match against regexps (ie. hidebrokenlinksref = "^.?$" will hide all broken links with a referrer having zero or only one char, typically "-") (le_zas) - output - all - added "outputdir" (ostborn) - output_modlogan can be used in parallel with output_template (ostborn) - template - added "flat_menu" as template option (drzoom) - added "cell_class_xxx" for each table cell element (drzoom) - added "cell_tags_xxx" for each table cell element (drzoom) - changed the behavoir of table generation producing more different celltypes (left||right)(header|line|hline|gline|footer) (drzoom) - changed the templates to use this new features (drzoom) - (internal) move the subpath-handling to the wrapper-function (ostborn) - fixed drawing of pies for small angles. (le_zas) - added replacement of the TITLE variable with the menutitle. (drzoom) - added a 'users' and 'traffic by users' report (ostborn) - Hits -> Visits in entry/exit pages reports (le_zas) - reduced the number of allocations in the template engine for more speed and less memory-usage/fragmentation (ostborn) - modlogan - fixed drawing of pies for small angles. (le_zas) - % column was added in response codes report (le_zas) - new boolean configuration option to show month fullname at top of menu, named showmonthinmenu (le_zas) - new boolean configuration option to show a grid in monthly graph, named showmonthlygraphgrid (le_zas) - new RGB configuration option to set color of grids, named grid (le_zas) - Hits -> Visits in entry/exit pages reports (le_zas) - Visit paths report is now shorter, / -> / -> / is now represented as 3x / (le_zas) - check for 'modlogan.css' BEFORE the logs are parsed (ostborn) - Added a new option named showmonthlymaxima which activates highlighting of max. values in history report, using the tinym modlogan.css class. - Some colors were added to summary report. See td.(hits|visits|...) classes in modlogan.css-dist file. -- 0.7.18 - 09.08.2002 21:15 - global - pcre and libxml are now detected properly even on RedHats (ostborn) - fixed another gzprintf() seg-fault in mstate_write (ostborn) - handled whitespaces in config-files more generously (ostborn) - output - template - fixed a seg-fault if no menuentries are specified (ostborn) - modlogan - fixed the month naming the picture for low-traffic sites which don't generate data for each month (ostborn) - input - clf - tightened the hostname-rule from (.+) to ((?i)[-.a-z0-9]+) (ostborn) - datatype - visited - fixed traffic handling for values > 4GB (ostborn) -- 0.7.17 - 05.08.2002 19:09 - input - (all) - fixed the flapping hour problem by initializing the "struct tm" properly (reported by Guy- @ #modlogan) - ostborn - clf - minor speed improvements - zas - processor - web - re-enabled the visit-reports for the logs without a useragent (reported by sascha schumann) - ostborn - minor speed improvements - zas - major speed improvements by rewriting the visit-handling - ostborn - fixed broken int2hex conversion - ostborn - output - modlogan - fixed inconsistency between XML-header and META-tags (reported by dan ohnesorg) - ostborn - added NOINDEX to the meta-tags - zas - fixed maxgraphedmonths > 12 - zas - template - fixed maxgraphedmonths > 12 - zas - etc - group.extension.conf - fixed a broken line (reported by dan ohnesorg) - ostborn - datatype - visit - autoupdate older statefiles (0.7.x) to current format - ostborn - webhist, mailhist - fixed the lost index-page bug by setting the internal datatype correctly (reported by several users) - ostborn -- 0.7.16 - 29.07.2002 20:17 - glue code (global) - main o speed improvements in the case that the resolver is used and a record is skipped - ostborn o warn the user if enable_resolver = 1 and modlogan isn't compiled with resolver support - ostborn o actually got the error-detection for loading plugins working - ostborn - output - template o fixed a unresolved symbol - ostborn - processor - web o added max_hits_per_visit - ostborn -- 0.7.15 - 14.07.2002 22:22 - glue code (global) - main o added the read_ahead_limit again (ostborn) o fixed various possible seg-faults (ostborn) o fixed a off-by-one allocation in the resolver-code (ostborn) - input - postfix o first official announement (bruggie) -- 0.7.14 - 10.07.2002 13:43 - glue code (global) - main o fixed seg-fault in dns-resolver (ostborn) o fixed a abnormal exit if a new month didn't contained any data (ostborn) o check if all dlopen headers are installed as libtool doesn't seem to do it for us (ostborn) - input - qmail o added tai64 support (ostborn) - config - modlogan.def.conf o added missing 'maxgraphedmonths' switch for output_modlogan for the variable months patch of dkl which was applied in 0.7.13 (ostborn) -- 0.7.13 - 01.07.2002 22:06 - translation o updated german translation - erich o updated french translation (Laurent MONIN) - ostborn - glue code (global) o downgraded to autoconf 2.52 too again o fixed seg-fault while writing state-files (gzprintf) - ostborn o new mhash sorting method: by quotient count/vcount - erich o added more '-lz' to the gd check for dumb ld's - ostborn o generalized the history structure a little bit - ostborn o added a VERSION check to all the plugins - ostborn o moved the resolver code from the processor/web and the input plugins to get_next_record - ostborn o seperated requesting hostname and requesting ip in the record datastructure (preparations for the localizer) - ostborn o updated to autoconf 2.53, automake 1.6.1, gettext 0.11.1 o added history support for the mail reports - ostborn o fixed non-chronological month in the history (reported by Robert Claeson) - ostborn - doc/etc o updated the INSTALL and README file - ostborn - datatype - visit o fixed seg-fault (Andreas Greve) - ostborn - output - template o added new reports "web_user_countries_visits", "web_user_view_duration_visits" and "web_user_view_duration_average" - erich o "pages" and "files" were swapped in overview - erich o added percent column to traffic of mail output - erich o added four new reports to mail output - erich o fixed the "bars out of box" error in the index-page (Laurent MONIN) - ostborn o made the number of days in the graph variable (Bradley Grainger) - ostborn - modlogan o fixed the "bars out of box" error in the index-page (Laurent MONIN) - ostborn o added some missing translations (Laurent MONIN) - ostborn o set a maximum width for the image of the Visit duration and visit path length (Laurent MONIN) - ostborn o added the colors of the coloums to the yearly summary (Laurent MONIN) - ostborn o made the number of days in the graph variable (Bradley Grainger) - ostborn - webalizer o made the number of days in the graph variable (Bradley Grainger) - ostborn - input - clf o removed the sorter from the clf plugin and add a "working" version for all plugins (bug reported by Laurent MONIN) - ostborn o moved the resolver to main.c - ostborn - msiis o moved the resolver to main.c - ostborn - msmedia o moved the resolver to main.c - ostborn - processor - web o fixed the "first character is missing in the referrers" bug (Jürgen Metelski) - ostborn -- 0.7.12 - 04.01.2002 16:28 - glue code (global) o added protection against .. attack against the splitter - ostborn o saved some allocation/deallocations by using mrecord_reset() - ostborn o modlogan doesn't want to run as root from now on - ostborn o the throughput is now calculated by rec/user-time - ostborn o added "show_available_config_options" option - ostborn o saved some more allocations by using mrecord_move() - ostborn - output - template o removed some unnessesary allocations and copies - ostborn - small speed boost, less memory fragmentation, less memory usage o added "show_available_reports_and_die" option - ostborn o added 8 new mail reports - ostborn - doc/etc o added some menu-structs and report-defaults for the mail reports - ostborn - processor - mail o added a number-cruncher for the the virus informations and two for the domain parts for the mail-addresses - ostborn -- 0.7.11 - 01.01.2002 13:58 - datatypes - visit o sublists (->hits) were not parsed correctly - ostborn o mlist_init was called far too often without any good reason - ostborn - input - clf o added a dummy '%a' format handler - Johannes Erdfelt - output - template o fixed alignment of the traffic column - ostborn o handle the case that the menu-structure can't be parsed correctly (die) - ostborn o handle the case that conf->menu == NULL correctly (no menu) - ostborn o a long is too short if you want to accumulate traffic - ostborn - fixed show_mhash accordingly o added new report "extensions by traffic" - erich o added three new reports sorted by visits - erich - process - web o added 'ignorehostmask' to ignore whole networks (192.168.2.0/24) - ostborn - glue code (global) o a long is too short if you want to accumulate traffic - ostborn - fixed mdata_get_vcount and m(hash|list)_sumup accourdingly o stat the history file before starting the xml-parser - ostborn - fixes a seg-fault in libxml on Solaris o mhash can now resize themselves if they notices that the contained list becomes to large - ostborn - should give us some speed and save some memory at the other side o dropped 'restore_internal_state_old' as it was deprecated anyway - ostborn o added 'mhash_unfold_sorted_limited_vcount' - erich o moved from mhash_insert to mhash_insert_sorted for speed reason - ostborn o if no *.la files are available try to load the .so directly - ostborn - fixes a problem with freebsd - doc/etc o added the new reports to the default config - erich -- 0.7.10 - 27.12.2001 19:38 - output - template o fixed seg-fault for long urls - erich o new experimental report 'requested urls sorted by traffic' - erich o calculate percent value for visits column - erich o fixed a some huge memory-leaks with the help of dmalloc - ostborn - most of them in the template code - smaller ones in the picture generation code o fixed a possible seg-fault in generate_history - ostborn - free(fn); was called twice on the same pointer - modlogan o fixed a possible sprintf bug caused by a incorrect format string (%li instead of %.0f) - ostborn - input - qmailscanner o added a scanner for the qmail-scanner quarantine logs - ostborn - global o added code for different sorting (by visits) - erich -- 0.7.9 - 25.12.2001 17:00 - output - template o fixed 'splitby' handling - the subpath was ignored o fixed a segfault - allocated too less bytes for the filenames in the 'copy-section' o removed 'pages_suffix' o added 'filename_pattern' and 'index_filename' instead - the filename_pattern replaces the pages_suffic as it gives the user the control over the filenames o fixed the 'bar is out of picture' bug o added a snow theme - modlogan o fixed the 'bar is out of picture' bug - webalizer o reactived the plugin o fixed some minor issues in the output - input - clf o fixes for the sorter of the input record - the first inserted record was always returned as the last record as the 'record_list' was handled completly wrong instead of taking the first element out of the sorted list the last one was used. -- 0.7.8 - 23.12.2001 17:10 - output - template o added some reports for the web-section o added the first 'theme' named 'basic' o added a second theme called 'modern' o added a clever menu generation scheme - input - bsdftpd o finished parser - isdn4bsd o added a basic parser - global o changed the spec file according to suggestions by - - - ... o added a check for pcre/pcre.h - fix for redhats o updated czech translation - by o updated to gettext 0.10.37 - fixes make distcheck o make distcheck and rpm-building works now again o fixed the return values for the config-value parsing - invalid values result in a abort now -- 0.7.7 - 06.11.2001 11:29 - global o requiring autoconf 2.50 at least (ostborn) o added polish translation by Piotr Szczepanski (ostborn) - output - modlogan o fixed missing symbols (ostborn) -- 0.7.6 - 24.10.2001 20:23 - input - clf o combined the tree regex for parse_pcre - speedup of 20% (ostborn) - sendmail o extrace DSN numbers (ostborn) - qmail o extract DSN numbers (ostborn) - processor - web o added a hostname cache for the groupings - speedup of 200% for smaller datasets (ostborn) - output - modlogan o rewrote the sorting algorithm to be more stable for greater datasets (mergesort instead of selection-sort) - libltdl o update configure-script (ostborn) - global o added a performance measurement infrastructure (ostborn) -- 0.7.5 - 19.10.2001 01:30 - global, datatypes o replaced the DOM parser by a SAX parser (ostborn) o added more exploit urls -> ./doc/group.url.conf (ostborn) o the config path is now set correctly in the modlogan.conf-dist by default (Alexander Leidinger) o fixed typos (Damien Clermonté) o updated to libltdl 1.4.2 - fixes for sol7-64bit (ostborn) o fixed vhost support (Damien Clermonté) - output - template o finished the template engine (ostborn) - input - clf o better handling of return values (Damien Clermonté ) - realserver o fixed missing symbols (ostborn) -- 0.7.4 - 23.09.2001 22:33 - global o fixed distribution (ostborn) o added switch for profiling [--enable-profiling] (ostborn) -- 0.7.3 - 21.09.2001 19:25 - datatypes - brokenlink, visit o decode the serialized strings (ostborn) - output - modlogan o improved lines graph to display only ~ 95% of the data this hides exteme long visits of search engines (erich) o fixed the extension graphic (ostborn) - doc o group.extensions.conf modified to ignore no-longer-hidden search strings (erich) - global o more than one includepath can be specified (ostborn) -- 0.7.2 - output - modlogan o added average page views for month and per page (lohgan) o the graph for visit duration and path length uses lines now (ostborn) o added a pie for the extensions and status codes (ostborn) o minor cosmetics (ostborn) - input - clf o better useragent detection (ostborn) o the requested URL is returned completly now (ostborn) - netscape o the requested URL is returned completly now (ostborn) - processor - web o fixed the hits == visits issue for logs without useragent (ostborn) - global o added support for multiple input plugins (ostborn) -- 0.7.1 - 09.09.2001 21:15 - output - modlogan o huge speed-up for the visit reports (ostborn) o new report for page view duration (lohgan) - input - pureftpd o new parser (ostborn) - clf o fixed format= handling (ostborn) o handle return values correctly - bsdftpd o new parser for tranfser logs of freebsd ftpd's (ostborn) - msmedia o finished the parser (ostborn) o extract duration field (lohgan) - realserver o extract duration field (lohgan) o parse user agent (lohgan) - qtss o extract duration field (lohgan) - proccess - web o calculate/sum view duration for all log formats (lohgan) - datatypes - mlogrec_web_extclf o 'duration' field for streaming input plugins (lohgan) - mstate_web o 'views' hash for view duration reports (lohgan) - global o dropped the DOM support from the XML writing code (ostborn) o modlogan dies if a configoption can't be parsed correctly (ostborn) o for configure of libltdl is rebuild (ostborn) -- 0.7.0 - 05.09.2001 11:40 - global o switched from -ldl to -lltdl (shouldn't harm anyone) (ostborn) o added $_number syntax for lowercasing, $^number for uppercasing in substitution patterns (for grouping searchstrings etc.) this syntax might change soon. (erich) o german translation updated (erich) o made new month with no data non-fatal (in splits) (erich) o renamed modlogan.searchengines to match.searchengines.conf [alexander@leidinger.net] (ostborn) o match.* and group.* are now installed in /etc/modlogan/ [alexander@leidinger.net] (ostborn) o made libintl and libxml support working for *BSD and Windows [alexander@leidinger.net] (ostborn) - output - modlogan o fixed seg-fault for small datasets (erich) o fixed bug for month with no data at all, but history in splitter (erich) o fixed yearly summary (position and year where wrong) (ostborn) - input - realserver o integrated a complete parser for all realserver logfile types (lohgan) - datatypes o key is now saved url-encoded, so non-ascii chars work (erich) - sublist o fixed readcode to insert the list instead of just ignoring it (erich) -- 0.7.0rc8 - - datatypes - sublist o fixed write code (wrong key for "sublist") (eRich @ #modlogan) - brokenlist o fixed missing init in setdata (eRich @ #modlogan) - visit o fixed init data.visit.hits in setdata (found by nachfalke @ #modlogan) - output - modlogan o seperated the graphic code for bars and pies o added graphes for visit path length and visit duration - input - clf o fixed extraction of requested url for 'format=' (found by nachfalke @ #modlogan) - global o text-nodes are ignored in mstate_read (eRich @ #modlogan) -- 0.7.0rc7 - 30.08.2001 00:49 - output - modlogan o more xhtml 1.0 patches o added vhost graph (patch by Alex Petrov) o enabled extensions again o visits length path displays now the right values - text o made this output plugin a destination for the mail processor - processor - web o fixed a seg-fault in is_visit (reported by Stefan Knoblich) o files are treated as hits if the input data is comming from the ftp parser (az@rada.ftc.ru (requested at #modlogan)) o set a md5 hash instead of the quite long key for visits (eRich@#modlogan) (code is taken from rfc 1321) o added massvhost support (patch by Alex Petrov) o the include path is now used for the search engine file too (requested eRich@#modlogan) - mail o a new processor for mail-like input data (qmail, sendmail, ...) - global o added some xmlFreeDoc's for critical situations o new debug output handling o fixed return handling in mdatatypes_from_xml o fixed the buildsystem for libgd 2.x o fixed the skript that generates ./doc/plugin-options.txt - input - qmail o finished the plugin the generate usefull 'records' - sendmail o added a really basic sendmail parser. far from complete. - msiis o added some more 'known' fieldtypes (provided by Jesper G. Jensen) -- 0.7.0rc6 - 20.08.2001 01:24 - output - modlogan o removed the 'floating slice' from the country pie o removed slice with 0% from the pie o removed all 'bgcolor' and replaced them by CSS o fixed some cosmetics in the output (new colours, CSS) - processor - web o new searchengines are now writting into the file 'searchstrings' in the output-directory o added visit-path's, visit duration, visit path length o fixed visit support -- 0.7.0rc5 - 18.08.2001 00:18 - global + new searchengines o fixed the sample.def include dir to .../etc (reported by Sascha.Hemmerling@ision.net) + added 'compress_mode' to set the compression level - input - squid o Fixed the upcomming timewrap (997 912 373 -> 1 000 000 000) (reported by Sascha.Hemmerling@ision.net) - output - modlogan o Fixed a sprintf which doesn't wanted to display a NULL-pointer (reported by ) o finished the indention + added headers for 'internal errors' - datatype - brokenlink o fixed a seg-fault caused by a uninitialized variable (reported by ) - processor - web o fixed the countries report by inserting the right value into the hash -- 0.7.0rc4 - global o fixed the distribution again, all *.conf's were missing (reported by Sascha.Hemmerling@ision.net) -- 0.7.0rc3 - global + the *.conf files are now installed into the .../etc dir (reported by Sascha.Hemmerling@ision.net) + added match.*.conf to the distribution (reported by Sascha.Hemmerling@ision.net) -- 0.7.0rc2 - global + some Makefile.in's were missing (reported by hugo @ #php.de) -- 0.7.0rc1 - global + two new countries + various new searchengines o applied modlogan-tld.patch o fixes a memory leaks in - mconfig.c - mplugins.c - mstate.c o fixed dmalloc support o fixed state file and history handling - datatypes o fixed various memory leaks - input - netscape + added better errorhandling/reporting - shoutcast, ipchains + added timestamp handler - clf + added a fix for broken request -> "GET

senfnase rulez

HTTP/1.0" (reported and working fix provided by Alex Petrov ) - qtss o finished the parser (need testers) - output o enable output generation again - modlogan o fixed a memory leak + added group url by extension (.tar.gz, ... ) + beautified the HTML + pushed the HTML into the direction of XHTML 1.0 + added traffic col. to requested urls and extension + added a totals line to the index page + added a year total to the index page - processor - web + added group url by extension (.tar.gz, ... ) (groupextension, hideextension) -- 0.7.0pre3 - global + added a new configure switch which hides the very special input plugins (--enable-special) + enabled the output plugins again + added better progress feedback + removed the -t hack, every plugin HAS TO provide a timestamp. otherwise main() will die + added better errorreporting. modlogan is now going down hard if necessary. - input + added a plugin which parses the flow-packets directly (flowraw) + prepared msmedia server parser based on the msiis parser -- 0.7.0pre2 - Tue Jun 12 16:57:03 2001 - global o rewrote the plugin handling o rewrote the configfile handling o the datatype code in mdatatype.c has been rewritten and divided into several files under ./src/datatype/ o wrote a script to create the skeleton of a new datatype (./doc/create_datatype.sh) o the state code has been rewritten to write XML instead of the old internal format o some defines have been grouped to enums (./src/mdatatype.h) o the code is now doxygen'ed o patched gettext.m4 to support windows, freebsd and others who use the internal libintl () o updated to autoconf 2.50, libtool 1.4 and automake 1.4d o many small patches to the configure.in by Alexander Leidinger - output - modlogan o fixed translation of the index page - input - viruswall + added a new plugin skeleton for viruswall logs - msiis + added resolver support (Stephen Woodbridge ) - qmail + added the parser for qmail log via syslog - processor - web + new searchengines by Alexander Leidinger and Marcel Telka + seperated the grouphost and groupua from the configfiles -- 0.6.5pl1 - output - modlogan + added error handling for CSS writing - template - remove dependencies to pictures.h - global - "starting at" -> "Ignoring before" -- Sat Mar 31 17:03:38 CEST 2001 - 0.6.5 - output - modlogan o only display the bottom header if the table has more then 10 entries o the averages are used to create the index graph o the index page has not handled the white table background right + the active menu field is now highlighted o made HTML color tripple uppercase o a % was missing in the bookmarked pages report - null (new) + dummy plugin, if now output is needed - processor - accounting (new) + processor for traffic based accountings - web o fixed the subsitute function if tagged count below ref count - input - qtss [Quicktime Streaming Server] + updated the format documentation, taken from www.apple.com (but not functional yet) - ipchains + finished the plugin - netscape + added the dynamic filter generation - flow (new) [flow-tools] + a new parser for the output of the flow-tools o timestamp is still missing - global o rewrote the hash functions to use less memory and to prepare a dynamic hash function o rewrote the configfile parser to use regular expressions o reversed the meaning of disable_resolver to enable_resolver the resolver is now disabled by default o rewrote the configfile read code to support EOF at the end of a config option + added some new searchengines and reordered the whole file + reading statefiles from a directories with a depth > 1 -- Sat Feb 10 15:59:09 MET 2001 o fixed the web processor for wuftpd records + added a automatic test system [not distributed] o fixed a div-by-zero in the tkcontrol output plugin + added a script to generate plugins (./doc/create_plugin.sh) + added some meta info to the plugins to generate a overview which plugins is doing what o changed the output of the searchengine detection for easier integration into the database o updated the isdnlog parser to support version 3.2 o fix msiis to take the date from the header if no date is specified + new input plugins: qtss, ipchains + new processor plugin: firewall (incomplete) + add a warning messages at the end of the configure run. o guarded the most mallocs with asserts. o fixed a small bug for the apache-format string support o fixed urlescape for chars > 127 + if you set max... to -1 it now means 'show me all' + apache customlog support improved: format %V added + records can now be ignore based on host or url + added support to group and hide robots * released 0.6.4 -- Thu Jan 11 19:06:09 CET 2001 + fixed the visit read code to support multiple visits from the same host o the skeleton driver is compiled and installed anymore + added debug switch for the resolver -> debug_resolver + the modlogan output plugins now display the table headers at the bottom too o fixed the resolver handling for those plugin which don't request the ip + the resolver can now be disabled -> disable_resolver o the last timestamp is now init'ed correctly o fixed the process_timer to display the right times + enhanced the searchengine grouping o fixed a segfault in the state code. + added a charset selector to the modlogan output -> cont_charset, cont_language * released 0.6.3 -- Sun Dec 24 11:30:30 CET 2000 o updated faq o updated german translation o fixed a check for the dynamic version libadns if it is not installed o fixed a segfault in is_visit (processor - web) o fixed the makefile for the modlogan output plugin * released 0.6.2 -- Sat Dec 23 12:29:47 CET 2000 o fixed the restore code for the state files + added convert script for the new state-file format -> ./doc/convert_to_060.pl + update the checks for the dynamic version of modlogan + release 0.6.1 (devel) -- Fri Dec 22 19:00:44 CET 2000 + the handling of the following option has changed for the static modlogan - inputplugin - processorplugin - outputplugin if theses options aren't reflecting the hardlinked plugins modlogan dies gracefully o fixed the visit counting for multiple users from the same host (firewall, proxy) o rewrote the timing code for the thoughput calculations + added ip resolver + the modlogan output plugin got a 'percent' coloumn o 'mlist_insert' has now a friend -> 'mlist_append'. this one fixes many problems related to 'grouping' and 'hiding' - searchengine grouping is back - group... + new searchengines o circumvented a bug in gzprintf which can't handle block longer than 4096 bytes without a buffer overflow + added more grouping/hiding options o fixed a seg-fault if 'dont_escape_entiries' was enabled o removed several small memory leaks o the country pie uses 'better' colours * released 0.6.0 (devel) -- Thu Nov 16 23:32:40 CET 2000 + updated the FAQ o fixed a bug in the parser for the searchengine definitions. - lines regex starting with [...] where interpreted as section head o circumvented another bug in the searchengine handling that was introduced in one of the last release with the searchengine grouping - some engines were hidden by engines with the same group name -> "Lycos" currently the groupname has to be unique + added 'hidebrokenlinks' o the path to the plugins is now hardcoded at compile time. this should make the packaging process a lot easier. + added a check the the needed shared libraries o fixed the pagecounting - added pagetype="/$" to the default configfile NOTE: don't compare the pagecount from previous versions with the pagecount from this version !! * release 0.5.7 - stable -- Sun Oct 22 23:59:16 CEST 2000 o the state for the first month wasn't written if the splitter was used. fixed that. + added two new countries o fixed page counting and other releated bug o more fixes for OS/2 * released 0.5.6 - stable -- Sun Oct 15 18:13:40 CEST 2000 o fixed the country pie for some special cases + rewrote the state-file reader to use PCRE instead using the non-existent string processing functions of C + the grouping and hiding directives are now using PCRE rather than the weak strmatch-function o suppressed the debugmessages of the splitter + added grouping of searchengines o fixed the incremental support which was broken since 0.5.0 o fixed broken image generation on OS/2 o small fix to compile on platforms where libintl.h (and others) are not installed system-wide o fixed memory leaks in the input plugins: squid, wuftpd + added a skeleton input plugin as a start point for developers + added a input plugin for shoutcast logs o fixes for the splitter - the replacement of the backreferences was incomplete - if a stream was empty, modlogan crashed with a SIGFPE in the output plugins * released 0.5.5 -- Thu Sep 21 23:42:10 CEST 2000 o the two fields srv_host and srv_port were not copied correctly in the clf input sorter o fixed the wrong averages on the input page o added splitter support for the webalizer output plugin o removed the debian directory again on request of the packager o added dont_escape_entities and dont_cut_urls switches on request of o added a whole bunch of new searchengine definitions submitted by and o added a italian translation by o implemented the group_url rule which was already configurable but not working * released 0.5.4 -- Mon Sep 4 23:05:21 CEST 2000 o the native squid parser mixed up the values for req_method, req_url and xfersize. fixed that + added debian support o if the configfile contained a wrong option the reported linenumber was somtimes wrong. fixed. + added more default values to the sample file o cleaned up the tkcontrol output plugin o fixed the telecom processor which wasn't updated to the changes which were introduced with the splitter o fixed the 'native squid' parser to be more relaxed on the input + added the 'd'elete flag to the wuftp input plugin + added a realserver logfile parser o fixed multiple typos noted by * released 0.5.3 -- Sun Aug 13 20:23:37 CEST 2000 + added new searchengine definitions from + added a input plugin for the native squid logfiles o removed a crash if the url is empty o removed some warnings o removed a linker error for red hats in combination with libpcre (patch submitted by ) + added a hide_host option * released 0.5.2 -- Fri Aug 4 11:16:49 CEST 2000 o the input wasn't parse if no format string was specified for (input-clf:format) + added some new searchengines submitted by * released 0.5.1 -- Wed Aug 2 22:45:51 CEST 2000 + added support for the apache logfile definition to the clf input plugin + updated the documentation (./doc/manual.txt) o rearranged the mainloop in main.c for to support the multiple output streams + added a logfile splitter for the web processor + added a update po file for the czech translation from + included a new spec-file from + added support generating a reports every N lines + improved the support for proftpd + added checks for the output directory o fixed a seg-fault in the clean-up code. + added more output to the tkcontrol output plugin * released 0.5.0 -- Sun Jul 23 15:08:51 CEST 2000 o fixed a bug in the modlogan output plugin which caused reports to appear on the wrong page (reported by ) + you can now set the background color of the body (output_modlogan:body) + the modlogan output plugin has two new configfile options to set the html header/footer (output_modlogan:htmlheader, output_modlogan:htmlfooter) o updated the configfile documentation -> ./doc/plugin-options.txt + added some preperations for the logfile splitter + added a tkcontrol output plugin for the telecom processor o fixed a endless loop in main.c o fixed minor problems in combination with libintl * released 0.4.5 -- Thu Jul 20 16:51:39 CEST 2000 o added fixes for the configure script submitted by o fixed several small glitches in the configure-script (most of them came up in combination with failed detections of libgd) o fixed for freebsd * released 0.4.4 -- Mon Jul 17 01:43:27 CEST 2000 o updated my internal build system from make 3.76.1 to 3.78.1 after some users complained about make errors. fixed the corresponding errors. * released 0.4.3 -- Fri Jul 14 17:10:33 CEST 2000 o updated the installation guide to covers the locations of the packaged libraries o rewrote the serializer o changed the state of libgd from 'required' to 'highly recommend'. if libgd can't be found the graphics are disabled o rewrote some configure tests for libgd * released 0.4.2 -- Thu Jul 6 20:17:26 CEST 2000 o fixed some typos in the sample.def.conf + added a default for the url completition + added support for non-web logfiles (hicom116, elmeg, isdnlog) + added a very basic telecom processor + fixed some symbol clashes in the text output plugin + the values which were set before the default config file was read, were reset again. fixed. * released 0.4.1 -- Sat Jul 1 17:30:36 CEST 2000 + added support for a default configfile + added a sorter to the clf input plugin o more config options are checked if their values are valid * released 0.4.0 -- Wed Jun 28 22:39:36 CEST 2000 + the msiis inputplugin is now parsing the '#Fields' line and generates the parser according to this definition + added strings for translation + added the spanish translation from + the modlogan output plugin expands every request string to a full URL now * released 0.3.7 -- Sun Jun 18 16:58:26 CEST 2000 o the last fix for the inputplugin wasn't complete. the request protocol wasn't seperated correctly * released 0.3.6 -- Sun Jun 18 14:45:20 CEST 2000 o fixed the clf-inputplugin for malformed url-fields + added new searchengines * released 0.3.5 -- Sun Jun 18 13:05:51 CEST 2000 + added some missing config-option to the ./doc/sample.conf + added semi-incremental mode (requested by ) -> new default * bought a new monitor. bigger and better than the old one. * released 0.3.4 -- Sat Jun 17 13:54:10 CEST 2000 + small documentation updates -> ./doc/plugin-options.txt + the dynamic version wasn't even compiling. fixed that. + moved the last few no-global config-option to the right section (searchengines -> [processor], inputfile -> [input]) + created a small (non public) test-environment * released 0.3.3 -- Sat Jun 17 00:27:39 CEST 2000 * my monitor has died and the old 14" tube is hurting my eyes. (jk) + the configure script is now telling you if a required lib isn't found + added a msiis inputplugin for the logfiles from MS IIS 5.0 + added Czech translation from Dan Ohnesorg * released 0.3.2 -- Wed Jun 14 22:25:46 CEST 2000 + added the missing modlogan.searchengines o fixed the commandline option handling for platforms without getopt_long o typo fixes o updated the German translation * released 0.3.1 -- Tue Jun 13 23:36:10 CEST 2000 + rewrote the searchengine handling. new searchengines are now easier to detect and adding more searchengines shouldn't drop the speed that much. o fixed some seg-fault in the processing plugin caused by unchecked return values o typo fixes sent in by various people o rewrote the TODO to cover a look into the future of modlogan - moved the todo list to TODO + added a basic man-page (with the help of help2man) + every month which is finished produces a unique state-file which can be used later for a rerun o the clf-input plugins hadn't seperated the get_vars from the URL + major cleanup for the split up of the processing plugin + moved ./src/process.c to ./src/processor/web/process.c + rearranged ./src/mrecord.[ch] + changed the input plugins to the new infrastructure + moved all processor specific config option from [global] to [processor] * released 0.3.0 -- Sat Jun 10 17:57:04 CEST 2000 + fixed some small memory leaks in the history file handling and state file reading + fixed a huge memory leak in clf plugin + fixed some small memory leaks in the image creation of the output plugin * released 0.2.2 -- Sat Jun 10 15:25:46 CEST 2000 + fixed the regex for webserver logfiles without useragent/referrer + added support for squid logs * released 0.2.1 -- Fri Jun 2 22:03:33 CEST 2000 + added perl-regex matching o the wuftpd plugin is back again + updated the documentation for the new debug options * released 0.2.0 -- Thu Jun 1 22:14:21 CEST 2000 + added a spec file (provided by ) o fixed the country pie for huge pieces (jk) + added support for gzipped state and history files (jk) o fixed the error handling the input plugins (jk) o fixed 'last hit' time for broken links in output/modlogan/generate.c (jk) + added 'last hit' time to server errors in output/modlogan/generate.c (jk) + added support for gzipped inpt file for the clf-input plugin (jk) + every plugin is now able to set some default values (jk) -- Mon May 29 22:44:05 CEST 2000 o fixed a segmentation fault in mdatatypes.c (jk) o completed the list of countries (jk) o fixed the country pie in output/modlogan/pictures_countries.c (jk) * released 0.1.1 -- Mon May 8 23:41:22 CEST 2000 + added many new searchengines (jk) + added host count (jk) + extented the broken link section by 1. referrer 2. last time of request (jk) o fixed the german translation for the solaris 'gettext'-package + the modlogan module has 2 new modes: onepage, seppage (jk) + s/non-plugin/static/; s/plugin/dynamic/ in the docs where it was neccessary (jk) + added startup output (linked modules for the static version) (jk) + stripping index.* from the entry/exit pages (jk) o fixed urltolower in process.c - the protocol wasn't 'lower'ed (jk) + added some country strings (jk) + added numbers to every report in output/modlogan and output/webalizer (jk) + added the 'country pie' to output/modlogan and output/webalizer (jk) o fixed some small glitches for the plugin version (jk) o state->indexed_pages wasn't written to the state file - fixed (jk) * released 0.1.0 -- Mon Apr 24 01:11:59 CEST 2000 + finished the german translation (jk) + wrote some more documentation (jk) + added the modlogan output plugin (jk) + added a working 'text' output plugin (jk) + added a 'null' input plugin (jk) + rearranged the configure script. staticly linked plugins can be selected at configuration time (--with-output=..., --with-input=...) (jk) o hosts (requested url, referrer, hostsname) are now handled caseinsensitive (jk) * relesed 0.0.6 -- Fri Apr 21 19:37:06 CEST 2000 o fixed small glitches in pictures_month.c (jk) + added averages in the month-overview (jk) o rewrote the config-file handling, this is simplifying the option handling a lot (jk) + added 'broken links' grouping (jk) o fixed strmatch for short strings (jk) o fixed history writing (jk) o fixed many possible crashes in combination with grouping (jk) + added -lz if -lpng is used (jk) * released 0.0.5 -- Thu Apr 20 01:53:36 CEST 2000 + added the history file handling (jk) + added incremental mode again (jk) + added some new country codes (jk) * released 0.0.4 and announced it at freshmeat.net -- Tue Apr 18 23:39:29 CEST 2000 - removed the old changelog. It can be found on the webpages for further reference. the changelog will only contain changes that have been done on the code. (jk) + added more countries strings (jk) o fixed some small memory leak in mconfig.c with the help of dmalloc. one leaks is still in input/clf/parse.c because dlclose crashes in mplugin_free(); (jk) * tested modlogan against all logs at www.kneschke.de. It didn't crash and produced it logs. 437959128 bytes (1815199 records). the memory consuption went up to 3.5 Mb only. (jk)