From jm@jmason.org Wed Sep 18 12:57:54 2002 Return-Path: Delivered-To: yyyy@example.com Received: by example.com (Postfix, from userid 500) id 4376516F1C; Wed, 18 Sep 2002 12:57:54 +0100 (IST) Received: from example.com (localhost [127.0.0.1]) by jmason.org (Postfix) with ESMTP id 40A2CF7B1; Wed, 18 Sep 2002 12:57:54 +0100 (IST) To: Matt Kettler Cc: yyyy@example.com (Justin Mason), SpamAssassin-devel@lists.sourceforge.net Subject: Re: [SAdev] phew! In-Reply-To: Message from Matt Kettler of "Wed, 18 Sep 2002 02:04:29 EDT." <5.1.1.6.0.20020918014722.00a99b20@mail.comcast.net> From: yyyy@example.com (Justin Mason) X-GPG-Key-Fingerprint: 0A48 2D8B 0B52 A87D 0E8A 6ADD 4137 1B50 6E58 EF0A X-Habeas-Swe-1: winter into spring X-Habeas-Swe-2: brightly anticipated X-Habeas-Swe-3: like Habeas SWE (tm) X-Habeas-Swe-4: Copyright 2002 Habeas (tm) X-Habeas-Swe-5: Sender Warranted Email (SWE) (tm). The sender of this X-Habeas-Swe-6: email in exchange for a license for this Habeas X-Habeas-Swe-7: warrant mark warrants that this is a Habeas Compliant X-Habeas-Swe-8: Message (HCM) and not spam. Please report use of this X-Habeas-Swe-9: mark in spam to . Date: Wed, 18 Sep 2002 12:57:49 +0100 Sender: yyyy@example.com Message-Id: <20020918115754.4376516F1C@example.com> X-Spam-Status: No, hits=-6.4 required=7.0 tests=AWL,HABEAS_SWE,IN_REP_TO,QUOTED_EMAIL_TEXT version=2.50-cvs X-Spam-Level: Matt Kettler said: > Ok, first, the important stuff. Happy birthday Justin (a lil late, but > oh well) cheers! > a 13% miss ratio on the spam corpus at 5.0 seems awfully high, although > that nice low FP percentage is quite nice, as is the narrow-in of average > FP/FN scores compared to 2.40. As Dan said -- it's a hard corpus, made harder without the spamtrap data. Also -- and this is an important point -- those measurements can't be directly compared, because I changed the methodology. In 2.40 the scores were evolved on the entire corpus, then evaluated using that corpus; ie. there was no "blind" testing, and the scores could overfit and still provide good statistics. In 2.42, they're evaluated "blind", on a totally unseen set of messages, so those figures would be a lot more accurate for real-world use. --j.