Developer Libraries » IOCipher

updating from email convo:

Hey Abel,

I saw you were the assignee on the Bug report so I thought I would update
you on a work around I'm trying out.

I noticed (somewhat obviously) that the bigger I made the buffer the faster
it would copy the file. I'm not overly familiar w/ the intricacies of I/O,
let alone encrypted I/O, but it made sense that if the performance
increases with fewer writes I'd try to just do one big write. So I tried
copying the file to a ByteArrayOutputStream as an intermediary and then
just writing to the encrypted VFS in one call
(FileOutputStream.write(baos.toByteArray()). This significantly improved
write time to almost as quick as the native file system.

This seems like a viable work around for me at the moment given the small
file sizes I'm dealing with. However, it's probably a bad idea for a
general work around.

-Aaron

from email:

Hey Aaron,

Thanks for the bug report

Each write call to libsqlfs (done by that Posix method pwriteBytes) has
some significant SQL overhead. So, while I don't have the data to
support it yet, my hypothesis is that the SQL overhead is causing poor
performance.

Your workaround reduces the number of calls to libsqlfs, and hence the
SQL overhead. We've talked about buffering small writes in IOCipher,
then flushing them periodically in larger writes to libsqlfs, we'll see..

Another way to implement file copying is using FileChannels. The result
is the same, a single large write is performed rather than a bunch of
smaller writes.
Here's an example snippet https://gist.github.com/3763841

We're still in the process of profiling and optimizing IOCipher and its
underlying components. Your feedback is quite helpful, definitely keep
it coming.

Soon, I hope to write some docs for IOCipher describing how using it
differs from normal Java IO. Using larger buffers is a great example of
the types of tips I hope to include there.

Cheers,

~abel