[dmarc-discuss] Discrepancies between ISPs' aggregate reports

Franck Martin fmartin at linkedin.com
Thu Jun 21 10:27:39 PDT 2012

May be we should have by default the UTC day for sending reports. At least this is what I'm doing, because that's the way my logs are stored. So very convenient for me.

But in general servers internal clocks are set to UTC and logs and other critical data are stored in UTC time. So this could be the path of least resistance.

This could be a SHOULD in the spec.

From: Jan-Pieter Cornet [johnpc at xs4all.net]
Sent: Thursday, June 21, 2012 1:46 AM
To: Dave Hensley
Cc: Franck Martin; dmarc-discuss at dmarc.org
Subject: Re: [dmarc-discuss] Discrepancies between ISPs' aggregate reports

On 2012-6-12 22:07 , Dave Hensley wrote:
>>> Finally, it seems that Google and Yahoo agree about the date range of
>>> the reports (12:00:00am to 11:59:59pm UTC). But I've received reports
>> >from 126.com that go from 4:00:00pm UTC to 3:59:59pm UTC, and the
>>> report that I received today from xs4all.nl goes from 12:58:12pm UTC
>>> to 1:00:04pm UTC the following day, which isn't even a 24-hour period.

XS4ALL does not synchronize reports. Whenever a first message arrives from a domain, it puts a "FIRST DATA" timestamp in the database. At each run of the aggregate collector, if your reporting interval (usually 1 day) has passed, an aggregate report is sent over all data collected for your domain. This does mean you usually get reports over periods that are slightly longer than your reporting interval...

I now realize that it's a bit sloppy to use such a non-uniform period. I'll see what we can do about that, by aligning the 'FIRST DATA' on aggregate collector runs, making sure you get nice 24-hour periods in the aggregate reports (or whatever reporting period requested, as long as it's a multiple of the aggregator runs).

I believe the xs4all reports are within the specifications of the (draft) RFC. What's the opinion of this list, would it be better to enforce strict 24-hour periods where possible? Or would it be preferable to align all reports on the same interval? (as google and yahoo seem to do).

Franck Martin <fmartin at linkedin.com> wrote:
>> I don't think you will be able to ask the report senders to sync on a
>> specific time. Can't you interpolate based on the time to fit a 24 hours
>> window?

Dave Hensley continued:
> I'm not sure how interpolation could be accomplished... can you please
> explain your suggestion a bit further?
> What I've got right now is a report from, say, Yahoo, which tells me
> that sometime between 12:00:00am and 11:59:59pm, the IP address
> sent 5 invalid messages to @yahoo.com users using my
> domain name. And then I've got another report from, say, 126.com that
> tells me that sometime between 4:00:00pm and 3:59:59pm, the same IP
> address sent 3 invalid messages to @126.com users using
> my domain name.
> But if I don't get a timestamp for each individual message, how can I
> calculate the total number of invalid messages that were sent during a
> particular 24-hour window?

You can obviously not get precise numbers, but you can split reports that do not fit 'your' reporting window. In this case, the netease report starts 16 hours later, and extends 16 hours into the next day. So you only count 8 hours for 'yesterday', and the other 16 hours for 'today', and distribute the numbers proportionally over those periods. So sent 1 invalid message to @126.com 'yesterday', and 2 invalid messages 'today'.

In this case, the numbers happen to come out as round numbers. That's obviously not always the case with varying reporting intervals. You can either round to integers or just deal with fractions. When graphing or averaging the results, you might as well use a fractional amount of messages.

Does this seem workable for you?

Jan-Pieter Cornet <johnpc at xs4all.net>
Systeembeheer XS4ALL Internet bv
Internet: www.xs4all.nl
Contact: www.xs4all.nl/contact

More information about the dmarc-discuss mailing list