Anyone know much about buffering when reading and writing to files or
other streams?
Basically, from what I can tell, the OS may do one level of
buffering, the library may do another (like fread might), and... then
you might want to do your own, also!!!
Well, in my mind, this can be a major source of inefficiency. Why?
because it means data is needlessly copied about.
I think, it would be much more efficient, if I did all the buffering,
myself. This is because, the ElfData plugin, it's splitter functions
actually point to a fraction of the original source, making splitting
faster for larger texts. In addition to this, I am writing some new
functions which mean that I can make a "slidable" ElfData object,
which does not need to get freed or disposed! This will give a truely
zippy experience when reading files, because all the create/dispose
object fat will have been trimmed off. It'll start getting closer to
the speeds you'll see in C.
Or maybe faster, because... of C's sucky memory management, data is
often copied around for you, anyhow. Like these buffers I talked
about. If the FILE* stores it's own buffer... well, then when you can
fread it will copy it's buffer into your data, which is one extra
unnecessary copy step.
So, anyhow, for me to capitalise on all of this, I must really know
how to work nicely with the buffering.
Ideally, I would have some way to tell fread "please don't do any
buffering for me, just read straight off the disk into my own
privately managed buffer, thanks!"
And also I want to ASK the FILE* "please tell me what is the OS's
prefered size for buffering, so that I can most effectively grab
stuff off the disk".
I think I can do the first, via setvbuf. The second, I'm not so sure
about.
Ideally, it'll be a solution that is fast for MacOSX, Linux, and
Windows, too :)
Anyone??
--
http://elfdata.com/plugin/
_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>
Search the archives of this list here:
<http://support.realsoftware.com/listarchives/lists.html>
|