THIS DOCUMENT IS OUTDATED AND WILL BE REPLACED BY A MORE UPDATED VERSION AS SOON AS POSSIBLE.
After going through the source code for firefox for andoroid which can be found at http://hg.mozilla.org/mozilla-central/ , these were the network related findings that I've made.
The application uses maily 3 layers to work.
1) The core layer, written in C++.
2) A JS layer with some more functional code.
3) A Java layer that makes uses the Android API's and other code used for android.
Out of the 3 layers, 2 of them make network calls.
1) The C++ layer.
There is an advantage of this layer making the network calls as all the calls made by this layer are proxied. They obey the fennec settings for proxy that has currently been set at mozilla-central/mobile/android/app/mobile.js . Hence, we do not need to bother about setting proxy to these connections and calls.
2) The Java layer.
The Java code, the code run by android also has many network calls but the problem is that these calls are not done obeying the tor proxy and hence they are harmful.
List of places where these calls are made are:
a) any class which imports org.apache.* (commons.net.ftp.*, http.*):
build/mobile/sutagent/android/DoCommand.java
build/mobile/sutagent/android/SUTAgentAndroid.java
mobile/android/base/distribution/Distribution.java
mobile/android/base/favicons/LoadFaviconTask.java
mobile/android/base/tests/BaseRobocopTest.java
b) The browser also seems to be using another library for network communications and that lies at mobile/android/thirdparty/ch/boye/httpclientandroidlib
This library seems to have proxy support but the calls made to this library have not been proxied, a list of those are here:
mobile/android/base/background/bagheera/BagheeraClient.java
mobile/android/base/background/bagheera/BagheeraRequestDelegate.java
mobile/android/base/background/bagheera/BoundedByteArrayEntity.java
mobile/android/base/background/bagheera/DeflateHelper.java
mobile/android/base/background/fxa/FxAccountClient10.java
mobile/android/base/background/fxa/FxAccountClient20.java
mobile/android/base/background/fxa/FxAccountClientException.java
mobile/android/base/background/fxa/SkewHandler.java
mobile/android/base/background/fxa/oauth/FxAccountAbstractClient.java
mobile/android/base/background/fxa/oauth/FxAccountAbstractClientException.java
mobile/android/base/background/fxa/oauth/FxAccountOAuthClient10.java
mobile/android/base/background/healthreport/upload/AndroidSubmissionClient.java
mobile/android/base/browserid/verifier/AbstractBrowserIDRemoteVerifierClient.java
mobile/android/base/browserid/verifier/BrowserIDRemoteVerifierClient10.java
mobile/android/base/sync/GlobalSession.java
mobile/android/base/sync/jpake/JPakeClient.java
mobile/android/base/sync/jpake/stage/DeleteChannel.java
mobile/android/base/sync/jpake/stage/GetChannelStage.java
mobile/android/base/sync/jpake/stage/GetRequestStage.java
mobile/android/base/sync/jpake/stage/PutRequestStage.java
mobile/android/base/sync/net/AbstractBearerTokenAuthHeaderProvider.java
mobile/android/base/sync/net/AuthHeaderProvider.java
mobile/android/base/sync/net/BaseResource.java
mobile/android/base/sync/net/BaseResourceDelegate.java
mobile/android/base/sync/net/BasicAuthHeaderProvider.java
mobile/android/base/sync/net/HMACAuthHeaderProvider.java
mobile/android/base/sync/net/HawkAuthHeaderProvider.java
mobile/android/base/sync/net/HttpResponseObserver.java
mobile/android/base/sync/net/MozResponse.java
mobile/android/base/sync/net/Resource.java
mobile/android/base/sync/net/ResourceDelegate.java
mobile/android/base/sync/net/SyncResponse.java
mobile/android/base/sync/net/SyncStorageCollectionRequest.java
mobile/android/base/sync/net/SyncStorageRequest.java
mobile/android/base/sync/net/SyncStorageResponse.java
mobile/android/base/sync/net/TLSSocketFactory.java
mobile/android/base/sync/repositories/Server11RepositorySession.java
mobile/android/base/sync/setup/auth/AuthenticateAccountStage.java
mobile/android/base/sync/setup/auth/EnsureUserExistenceStage.java
mobile/android/base/sync/setup/auth/FetchUserNodeStage.java
mobile/android/base/sync/stage/EnsureClusterURLStage.java
mobile/android/base/sync/stage/SyncClientsEngineStage.java
mobile/android/base/tokenserver/TokenServerClient.java
mobile/android/tests/background/junit3/src/sync/TestUpgradeRequired.java
This is the url for the query,
https://dxr.mozilla.org/mozilla-central/search?q=ch.boye.httpclientandroidlib&case=false&offset=700
NOTE:
Not all these classes make network calls, I will make a shorter list of this in the next few days.
[13:26:08] <n8fr8_> okay amoghbl1, i am here now. let's just do an IRC meeting today.
[13:26:16] <-> You are now known as n8fr8
[13:26:24] <amoghbl1> OK
[13:26:26] <n8fr8> so, I don't see any updates posted the tickets or project: https://dev.guardianproject.info/projects/orfox-private-browser/
[13:26:33] <n8fr8> can you do that today with anything you share here?
[13:26:54] <amoghbl1> I'll update those, haven't kept that up!
[13:27:07] <n8fr8> let's get off to a good start there :)
[13:27:18] <amoghbl1> I've been building the repo, with our mozconfig but on the tbb repo
[13:27:23] <amoghbl1> I have an apk ready
[13:27:43] <amoghbl1> But it seems to crash, I don't know what exactly the problem is, so I've been looking into that mainly
[13:27:56] <n8fr8> okay. can you post it to: https://dev.guardianproject.info/projects/orfox-private-browser/files ?
[13:27:59] <n8fr8> so i can try it on some devices?
[13:28:08] <n8fr8> have you tested on both ANdroid 4 and 5?
[13:28:20] <amoghbl1> Doing that now...
[13:28:29] <amoghbl1> I don't have an android 4 device :/
[13:28:50] <amoghbl1> So just on my 5, that could be the problem actually
[13:29:00] <amoghbl1> Posting the apk now, test it out...
[13:29:06] <n8fr8> there might be one at the office. when you are there, ask hans.
[13:29:23] <n8fr8> otherwise, i have a bunch.
[13:29:32] <n8fr8> can you get stack traces from the crash?
[13:29:38] <amoghbl1> I'm here right now...
[13:30:05] <amoghbl1> Ya, I've got the trace, lemme post that as well
[13:32:03] <n8fr8> post that as bug ticket
[13:32:07] <n8fr8> or a gist i suppose... either way
[13:32:32] <n8fr8> also amoghbl1 i think we can order a moto G for testing on this as well... or some low-cost Android 4.x device
[13:33:10] <amoghbl1> Lemme look around the office for a device, if I don't find one, then we could order it I guess!
[13:35:04] <amoghbl1> These builds are going to take some while as well, about an hour each time :/
[13:35:17] <n8fr8> yeah... that is the worst part
[13:35:25] <n8fr8> doesn't it get faster though once part of it is built?
[13:35:39] <n8fr8> like if you just change a small part of the mozconfig, does it recompile everything?
[13:35:53] <n8fr8> so you are using our old mozconfig?
[13:35:57] <amoghbl1> No, it doesn't recompile everything...
[13:36:19] <n8fr8> i wonder if you should start with the Tor mozconfig
[13:36:25] <n8fr8> and then modify it for mobile?
[13:36:26] <amoghbl1> Yup, the old mozconfig, produced an apk, but I think there was some problem with libraries with that one..
[13:36:33] <amoghbl1> Exactly, that's what I've done now
[13:36:42] <amoghbl1> And it's almost done building...
[13:36:57] <amoghbl1> It's been going on for 50 mins...
[13:38:03] <amoghbl1> If I just wanna write notes about what I'm doing, where should I upload those n8fr8 ?
[13:38:14] <n8fr8> put them on the project wiki
[13:38:23] <n8fr8> https://dev.guardianproject.info/projects/orfox-private-browser/wiki
[13:38:29] <n8fr8> don't attach them... create a new wiki page
[13:38:39] <amoghbl1> ok, I'll write up a short report on week one and put it there?
[13:39:28] <n8fr8> weekly reports should be posted as "News"
[13:39:33] <n8fr8> it is like the project blog
[13:39:51] <n8fr8> for documentation or notes that are less chronogical, and longer put them on the wiki
[13:39:57] <n8fr8> you can link from weekly reports to the wiki
[13:40:02] <n8fr8> i just created a new wiki subsection for you: https://dev.guardianproject.info/projects/orfox-private-browser/wiki/Development_Notes_Summer_2015?parent=Wiki
[13:43:05] <amoghbl1> Did you get the file I just uploaded?
[13:43:36] <amoghbl1> https://dev.guardianproject.info/attachments/download/1581/fennec-31.3.0esr.en-US.android-arm.apk
[13:49:16] <n8fr8> i will get it now
[13:51:51] <n8fr8> amoghbl1: so i am running on moto G with Android 4.4.
[13:51:57] <n8fr8> it started up, and then crashed about 1 second after
[13:52:11] <amoghbl1> Yup, I get the same results
[13:52:49] <krt> n8fr8: gilgamesh came in pretty handy today, so thanks for doing that :)
[13:52:57] <n8fr8> StrictMode policy violation; ~duration=211 ms: android.os.StrictMode$StrictModeDiskReadViolation: policy=31 violation=2
[13:53:12] <n8fr8> krt: cool! we used it over the weekend at the windfarm0 event, and it worked pretty well tehre tooo
[13:53:44] <n8fr8> https://talk.developersquare.net/t/some-pictures-from-todays-event/89/1
[13:55:07] <n8fr8> amoghbl1: https://gist.github.com/n8fr8/82cc1a37c4e52daf4199
[13:55:10] <n8fr8> that is the crash i am seeing
[13:55:11] <n8fr8> strange
[13:55:58] <amoghbl1> Yup, I have the same thing...
[13:56:32] <n8fr8> actually that is not the crash i don't think
[13:56:33] <n8fr8> hmm
[13:56:54] <amoghbl1> OK, I've got an updated apk, built off of a new mozconfig...
[13:56:57] <amoghbl1> Wanna try that out?
[13:56:59] <amoghbl1> Uploading now
[13:57:15] <n8fr8> the crash seems to be later with native code
[13:57:19] <n8fr8> libmozglue
[13:58:17] <n8fr8> https://gist.github.com/n8fr8/fcdb35904600c40ce870
[13:58:25] <n8fr8> sure amoghbl1 i can test
[13:58:37] <amoghbl1> OK, this is weirder, it seems to have produced a 428MB apk :O
[13:58:59] <n8fr8> uh oh
[13:59:08] <n8fr8> i don't think i want to install that
[13:59:13] <amoghbl1> How is that possible o.O
[13:59:27] <amoghbl1> There's something wrong with the mozconfig I guess!
[13:59:43] <n8fr8> so this is the mozconfig from Tor?
[13:59:54] <n8fr8> did you modify it with the proper mobile config lines?
[14:01:19] <amoghbl1> Its a proper apk apparently, it's installed fine on my phone, an runs too
[14:01:23] <amoghbl1> gimme a sec
[14:02:27] <n8fr8> perhaps debug data in it?
[14:03:10] <amoghbl1> ya, reading logs now...
[14:04:26] <amoghbl1> https://gist.github.com/amoghbl1/05e4ca73cb44b6798140
[14:04:34] <amoghbl1> There's the mozconfig I'm using now
[14:05:04] <amoghbl1> The app doesn't crash, but it loads nothing
[14:08:37] <amoghbl1> Telemetry seems broken, keeps returning null, that's what the log says
[14:09:55] <amoghbl1> you're right n8fr8, the strict mode violation seems to happen here too, but doesn't result in a crash
[14:11:09] <n8fr8> maybe comment out some of the things that are disabled?
[14:11:15] <n8fr8> one at a time, rebuild, then try again?
[14:11:26] <n8fr8> you do have orbot running?
[14:11:46] <amoghbl1> Ya I do...
[14:11:49] <n8fr8> also rememebr... tor browser now has tor binary built in somewhere
[14:12:00] <n8fr8> we need to figure out where that code is, and where the torrc config happens
[14:12:08] <n8fr8> so we can set it the orbot socks port, etc
[14:12:26] <n8fr8> we might need to ask mikeperry about this on tor-dev
[14:12:29] <n8fr8> #tor-dev on oftc
[14:12:56] <amoghbl1> I've marked my gist with a comment
[14:13:02] <amoghbl1> With my error log...
[14:13:06] <amoghbl1> https://gist.github.com/amoghbl1/05e4ca73cb44b6798140
[14:22:59] <amoghbl1> Rebuilding with an updated mozconfig now, let's see what happens then n8fr8, talk to you back in an hour then?
[14:36:50] <n8fr8> sure, amoghbl1 i am just working through email lulz
[15:07:25] <n8fr8> i guess we stumbled into the tor-browser meeting amoghbl1!
[15:07:30] <n8fr8> that is good to know... good timing with our meeting! :)
[16:14:55] <-- baudtack (~baudtack@unaffiliated/baudtack) has left #guardianproject
[16:16:12] <amoghbl1> n8fr8, well, there's some good news, I got a build that is of a decent size but it crashes after a second
[16:42:40] * Topic for #guardianproject is "Help us build things at https://dev.guardianproject.info, all questions welcome here, or email us at support@guardianproject.info"
[16:42:40] * Topic set by _hc!~hans@199.119.112.127 on 2015-04-10 17:48:47 UTC
[16:42:55] * Channel #guardianproject created on 2010-04-22 16:12:37 UTC
[16:45:07] <n8fr8> okay, amoghbl1. let's try to drill down to the specifics of the crash, and share with the tor-browser team once you have something posted as a ticket or written up somewhere
[16:51:15] [gpgithub] [ChatSecureAndroid] n8fr8 pushed 3 new commits to master: https://github.com/guardianproject/ChatSecureAndroid/compare/a8d8cf47b9ac...a7c72d91d460
[16:51:15] [gpgithub] ChatSecureAndroid/master 448a7b2 n8fr8: simplify, clean-up handling of network connectivity state and improve notifications
[16:51:15] [gpgithub] ChatSecureAndroid/master 767cb2a n8fr8: update notificaiton with proper intent
[16:51:15] [gpgithub] ChatSecureAndroid/master a7c72d9 n8fr8: Merge pull request #678 from n8fr8/bug-fix-network-state...
[16:51:15] <-- gpgithub (~gpgithub@192.30.252.46) has left #guardianproject
[16:59:21] <amoghbl1> I got it building and running on 5 n8fr8
[17:03:45] <n8fr8> what changed?
[17:03:46] <amoghbl1> by 5 I mean android 5
[17:04:31] <amoghbl1> I went to the firefox irc and spoke to a person there
[17:04:45] <amoghbl1> I got a patch from there
[17:05:27] <amoghbl1> Lemme write something up
[17:08:27] <n8fr8> great
[17:14:49] <amoghbl1> n8fr8, https://dev.guardianproject.info/news/214
[17:16:00] <n8fr8> can you post a new APK so i can try here?
[17:16:09] <n8fr8> i would also add that mozconfig file into our repo
[17:16:21] <n8fr8> https://github.com/guardianproject/orfoxfennec
[17:16:28] <amoghbl1> Yup, I'm uploading it as we speak
[17:16:55] <n8fr8> great. we need to update the external submodule address as well to point to the tor-browser repo at the proper commit
[17:16:58] <amoghbl1> I'll commit and push it to my repo and open a pull request
[17:17:09] <amoghbl1> Yup, I'll do that as well
1. Background on code/project/build
BUILD-
- The Orfox code is currently built upon the Tor Browser ESR38 release
2. Proposed Changes
- The aim of Orfox is to match the specs of the tor browser bundle for the desktop.
We plan on making it an almost exact replica of the desktop app in order to provide
a similar user experience to people who have been using the desktop browser.
This newly implemented browser would hopefully be a replacement application to
the current Orweb app that we have.
We will match the Tor Browser design and re-use code/patches when at all possible:
https://www.torproject.org/projects/torbrowser/design/
Match Tor Browser Desktop spec:
- Bundling Add-ons: HTTPS Everywhere, No Script, Clean Exit
Android-specific changes:
- remove permissions for GPS, camera, microphone, NFC, contacts, accounts
- remove mozilla sync features (currently blocked but not totally stripped out)
- remove screen casting feature
- remove webrtc android specific code
- added in proxying support at Java layer, not yet supported by Mozilla
- added mobile browsing mode that changes the user agent to a mobile one
3. Build/release process
The build release cycle still has to be decided.
4. Risks, Unknowns, QuestionsFennec and the build instructions can be found at https://wiki.mozilla.org/Mobile/Fennec/Android#Building_Fennec. Once successfully built, this source produces a folder named android_eclipse which is where the main Fennec Browser app code is present.
5. Positives
- The vulnerabilities seen at http://xordern.net/ip-leakage-of-mobile-tor-browsers.html are not seen on Orfox.
What is Orfox?
In the summer of 2014 (https://lists.mayfirst.org/pipermail/guardian-dev/2014-August/003717.html), we announced that the results of work by Amogh Pradeep (https://github.com/amoghbl1), our 2014 Google Summer of Code student, has proven we could build Firefox for Android with some of the settings and configurations from the Tor Browser desktop software. We called this app Orfox, in homage to Orbot and our current Orweb browser. This was a good first step, but we were doing the build on Mozilla's Firefox code repository, and then retrofitting pieces from Tor Browser's code, which wasn't the right way to do things, honestly.
This summer (2015!) we have switched to building the Orfox mobile app directly from the Tor Browser code repository, successfully working through any mobile OS incompatibilities in the security hardening patches added by the Tor Browser team. We also had the additional task of reviewing the Android application code in Firefox, that is not part of Tor Browser, in order to modify and patch it to work inline with the Tor Browser requirements and design document.
As of today, we have a stable alpha release ready for testing, and are rapidly moving towards a public beta in a few weeks. Our plan is to actively encourage users to move from Orweb to Orfox, and stop active development of Orweb, even removing to from the Google Play Store. If users really want to continue using a WebView-based solution and do not need all of the capabilities that Orfox/Tor Browser provides, they can use Lightning Browser (https://github.com/anthonycr/Lightning-Browser), a lightweight, open-source app that offers automatic Orbot (SOCKS) proxying on start-up.
Where can I find all the relevant Orfox project pieces?
How is Orfox different than Tor Browser for desktop?
Orfox is built from the same source code as Tor Browser (which is built upon Firefox), but with a few minor modifications to the privacy enhancing features to make them compatible with Firefox for Android and the Android operating system. In as many ways as possible, we will adhere to the design goals of Tor Browser (https://www.torproject.org/projects/torbrowser/design/), by supporting as much of their actual code as possible, and extending their work into the additional Android components of Firefox for Android.
How is Orfox different than Firefox for Android?
Beyond the core Tor Browser components, Orfox also must ensure all Android-specific code is properly routed through the Tor proxy, and otherwise hardened to protect against data and privacy leaks.
How is Orfox different than Orweb?
This is a new privacy-enhanced browser for Android, based on Mozilla Firefox, configured by default to work with Orbot: Tor for Android.