Library enhancement proposal for copying data from/to Reader/Writer InputStream/OutputStream

[Remi Forax]

On 11/21/2014 04:05 AM, Stuart Marks wrote:
> Hi Patrick,
>
> Good to meet you in Antwerp!
>
> On 11/20/14 10:30 AM, Patrick Reinhart wrote:
>>> Am 20.11.2014 um 10:22 schrieb Pavel Rappo <pavel.rappo at oracle.com>:
>>>
>>> There is at least one method in the JDK with similar 
>>> characteristics: java.nio.file.Files#copy(java.io.InputStream, 
>>> java.io.OutputStream)
>>> But, (1) it has a private access (2) even if we made it public I 
>>> doubt java.nio.file.Files would be a good place for it
>>>
>>
>> I would suggest to introduce a separate IOUtil class holding such 
>> utility methods. Additionally to copyTo() and copyFrom() method could 
>> be added later for more intuitive usage. Also the copy method within 
>> the Files method could be replaced with a static reference to the 
>> IOUtil class.
>
> Thanks to Pavel for pointing out the existing copy operations in the 
> nio Files class. I think there's a case for the InputStream => 
> OutputStream copy method to be placed there too, although I admit it 
> is somewhat unusual in that it doesn't actually have anything to do 
> with files.
>
> At my first encounter with the nio.Files class some years ago I saw 
> the following copying methods:
>
>     copy(istream, targetfile)
>     copy(sourcefile, ostream)
>     copy(sourcefile, targetfile)
>
> and I immediately thought, "Where is copy(istream, ostream)?" So to me 
> at least, it makes more sense to be in the Files class than in some 
> newly created IOUtils class. (I'd like to hear further from Alan on 
> this.)
>
> As Pavel pointed out, the logic is also already there in the Files 
> class. Probably the way to proceed would be to rename the existing 
> (private) method to be something like copyInternal() and then create a 
> new, public copy(istream, ostream) method that does argument checking 
> before calling copyInternal().
>
>>> P.S. The thing that confuses me though, is the progress consumer. I 
>>> believe this feature is rarely needed (at least in a way you 
>>> described it).
>>> If you want to do something like this, you should probably go 
>>> asynchronous with full blown solution like what they have in 
>>> javax.swing.SwingWorker.
>>>
>>> -Pavel
>>
>> The method having a IntConsumer I have already discussed with some 
>> other colleagues and they suggested to better use a IntPredicate in 
>> order to have the possibility to interrupt the copy process, without 
>> having to interrupt any threads. Additionally there is still the 
>> possibility to use such a Predicate for counting the total or using 
>> the information for other possibilities.
>
> I'd suggest starting simple with a copy(istream, ostream) operation 
> and considering some kind of interruptible, progress-reporting 
> operation separately. It would seem quite limiting if the *only* 
> progress-reporting operation in the system were the stream copier. 
> We'd want a way to apply such a mechanism to other long-running 
> operations.
>
> I think the progress update reports also need to be decoupled from the 
> actual I/O operations. For example, the current buffer size in 
> nio.Files is 8192 bytes. If the streams are connected to a fast 
> network connection, this will result in thousands of calls to the 
> predicate per second. On the other hand, if the buffer size were 
> raised, and the streams are connected to a slow network connection -- 
> like my home internet connection :-) -- that might result in too few 
> callbacks per second.
>
> How to report progress from a running operation is an interesting 
> problem and it's worthy of considering, but a copying utility doesn't 
> seem like quite the right place for it.

I also think you don't need a specific progress API given that you can 
write an InputStream that act as a proxy on top of the real inputStream 
and delegate the number of read bytes to a specific lambda.

>
> s'marks
>

cheers,
Rémi