A proposal for an HTTP stream capture mechanism

[Jean-Christophe Collet]

Max (Weijun) Wang wrote:
> Hi Jessie
>
> This is very useful.
>
> What kind of info are dumped? Content and/or headers?

What is dumped is the entire stream, byte for byte, captured at the TCP 
socket level. So, includes request, headers and body. Both ways.

This is supposed to complement the traditional logging of events that is 
being extended as well.
So HttpURLConnection is getting some new logging entries as well as an 
optional Formatter for these entries.I attached an example of the 
created log (independent of the capture mentioned earlier).

What the dump now permits is easy comparison between the raw data (from 
the capture rules) and how it was interpreted by HttpURLConnection (from 
the logging).

>
> Can you also capture the connection info? Sometimes I need to know if 
> a new connection is created or the old one is reused.

This is now part of the "normal" log.

>
> The \. in sun\.com looks a little cumbersome. I know .hgignore 
> supports several different pattern matching mechanisms, probably worth 
> a look?

The idea is to use regular expressions to allow for maximum flexibility 
AND simpler implementation.

>
> Thanks
> Max
>
>
> On Apr 23, 2009, at 10:02 PM, Jean-Christophe Collet wrote:
>
>> I have been working on adding better logging and debugging tools to 
>> the HTTP implementation.
>> A significant chunk of the bugs that we have to investigate are HTTP 
>> related and being able to monitor the traffic has always been a 
>> problem. Using a packet sniffer is cumbersome and not always possible.
>> So I have been putting together a prototype to make that a lot 
>> easier. The idea is to be able to turn on traffic capture with a 
>> system property and a simple "rule" file.
>>
>> The idea is that you can specify a set of capture rules in a file and 
>> point to that file with the property 'sun.net.http.captureRules'
>> The rule file has a simple syntax. 1 rule per line. A line starting 
>> with a '#' is a comment and is ignored. A rule is a comma separated 
>> pair regular-expression , file pattern.
>> e.g.:
>>
>> http://www\.sun\.com , sun%d.log
>>
>> tells to capture traffic for all URLs containing http://www.sun.com 
>> and to dump that traffic into a file sunxxxxxx.log (the xxxxxx is a 
>> randomly generated number that ensures the created file doesn't 
>> overwrite an existing one. Of course that '%d' is optional. If the 
>> files exist, then every capture is appended to it (which might not be 
>> a good idea in case of multiple parallel threads).
>>
>> So to capture all the traffic into a single file, the rule would be:
>>
>> .* , http.log
>>
>> A usage example becomes:
>>
>> $ java -Dsun.net.http.captureRules=capture-rules GetURL 
>> http://www.sun.com
>>
>> where 'capture-rules' is the file containing my set of rules.
>>
>> I have a first prototype of this working but I'm still refining it. 
>> I'll publish a webrev in a few days.
>>
>> Comments?
>>
>>
>>
>

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: oldlog.out
Url: http://mail.openjdk.java.net/pipermail/net-dev/attachments/20090423/757984fa/oldlog.out