vector write/read

[Tigran Mkrtchyan]

Alan Bateman wrote:
> Tigran Mkrtchyan wrote:
>> Hi,
>>
>>
>> just want to ask why vector operations are not a part to nio?
>>
>> we are working on distributed storage and our users  often know where 
>> data is (files are like small databases - header and records). We have 
>> our own protocol for it and extensions to posix IO api:
>>
>> typedef struct {
>>     off64_t offset;
>>     int len;
>>     char *buf;
>> } iovec2;
>>
>> extern int     dc_readv2(int, iovec2 *, int);
>>
>> Basically what it does is you give an array of buffers and for each 
>> buffer you provide an offset. It's very helpful when there is a lot of 
>> randomly distributed small records. If file system do not support it 
>> you can always fall back to 'classic' reads internally, but provide 
>> only classic API force other people to find some custom solutions.
> Right, vector I/O is not currently supported, and for your example, 
> would require a sequence of reads (or you could use a thread pool to get 
> some parallelism or use AsynchronousFileChannel). I haven't seen many 
> requests for this but it may be worth considering. Off-hand, I don't 
> know if this is an area that has been standarized (the closest that 
> comes to mind is lio_listio in the real-time extensions when used in 
> wait mode).


NFSv4 supports it, but while posix API do not have it applications 
unable use it. I do have pure java nfsv4 client which can generate 
request like that. Most of network based protocols will benefit. The 
question is should application issue parallel read to overcome network 
latency or IO subsystem can do it. If backend Channel implementation 
can't do it, you can always fall back to multiple reads. That's exactly 
what we are doing now - vector read over network, multiple reads for 
local file system, but applications still call the same readv2.

Tigran.

> 
> -Alan.

-- 
________________________________________________________________________
Tigran Mkrtchyan                               DESY, IT,
tigran.mkrtchyan at desy.de                       Notkestrasse 85,
Tel: + 49 40 89983946                          Hamburg 22607,
Fax: + 49 40 89984429                          Germany.