Exremely long IOCipher write time when copying from SD card
It takes forever (~2 minutes in reality) to copy a file from the SD card to the encrypted VFS. The same operation only takes about a second when I don't use IOCipher. After performing a trace it's clear that the bulk of the time, ~99%, is spent in the Posix.pwriteBytes() method of the IOCipher library. I've attached a sample Android app (VfsTest.zip) and the trace file (IOCipher.trace) to help illustrate the problem as well as a copy of the file I was transferring (it's a pic of Titanaboa from the Smithsonian).
I tested this on a Galaxy Nexus (Google Play store version) and an HTC Incredible 2 running 2.3.7.
#2 Updated by abeluck over 5 years ago
updating from email convo:
I saw you were the assignee on the Bug report so I thought I would update
you on a work around I'm trying out.
I noticed (somewhat obviously) that the bigger I made the buffer the faster
it would copy the file. I'm not overly familiar w/ the intricacies of I/O,
let alone encrypted I/O, but it made sense that if the performance
increases with fewer writes I'd try to just do one big write. So I tried
copying the file to a ByteArrayOutputStream as an intermediary and then
just writing to the encrypted VFS in one call
(FileOutputStream.write(baos.toByteArray()). This significantly improved
write time to almost as quick as the native file system.
This seems like a viable work around for me at the moment given the small
file sizes I'm dealing with. However, it's probably a bad idea for a
general work around.
#3 Updated by abeluck over 5 years ago
Thanks for the bug report
Each write call to libsqlfs (done by that Posix method pwriteBytes) has
some significant SQL overhead. So, while I don't have the data to
support it yet, my hypothesis is that the SQL overhead is causing poor
Your workaround reduces the number of calls to libsqlfs, and hence the
SQL overhead. We've talked about buffering small writes in IOCipher,
then flushing them periodically in larger writes to libsqlfs, we'll see..
Another way to implement file copying is using FileChannels. The result
is the same, a single large write is performed rather than a bunch of
Here's an example snippet https://gist.github.com/3763841
We're still in the process of profiling and optimizing IOCipher and its
underlying components. Your feedback is quite helpful, definitely keep
Soon, I hope to write some docs for IOCipher describing how using it
differs from normal Java IO. Using larger buffers is a great example of
the types of tips I hope to include there.