Bazaar2 Monthly Report - April 2017

Added by hans 8 months ago

April was a big month for us in terms of finishing up some big parts
that are directly visible to users, and easy to demonstrate. The
biggest is the final 0.103 release of the F-Droid app which includes
the complete overhaul of the user experience, which feels simple,
friendly and modern. This is one short step from a big 1.0 release,
once we nail down the last features and get some more testing
completed.

We also launched the first alpha of the new F-Droid Repomaker, a
simple web tool for creating and managing collections of apps and
media, and delivering them to users via F-Droid repositories (aka
“repos”). Try the alpha demo! http://repomaker.grobox.de/

On top of those two launches, there are many other small
accomplishments from this biggest and final development sprint for
Bazaar2.

Objective 1 Simple multi-pronged distribution

Make All Text Translatable

All texts within F-Droid and graphics associated with apps are now
translatable, including all the strings within the app itself, all app
names, summaries, descriptions, video links, recent changes, and
screenshots. With release of F-Droid client 0.103, it will use any
available language. For the F-Droid client app itself, many languages
are completely translated, and many more have reached the functional
level, thanks to the ongoing support from F-Droid community volunteers
and the Localization Lab:

  • 19 over 99%, including Belarusian, Brazilian, Persian, Russian,
    Spanish, Chinese, Turkish
  • 32 over 90%, including Arabic, French, Italian, Romanian, Shona, Ukrainian
  • 45 over 70%, including Burmese, Hungarian, Korean, Simplified Chinese,
    Thai, Vietnamese
  • see all and contribute here:
    https://hosted.weblate.org/projects/f-droid/f-droid/

We have not received any Tibetan translations yet. We will be hiring
translators to finish the Simplified Chinese and Tibetan translations.

For the per-app materials, we are now adding all the translated
materials for all the Guardian Project apps to the Guardian Project
F-Droid Repository, which users can enable with the flip of a switch
in F-Droid. We are also helping app developers to get their
descriptive materials integrated for automatic inclusion in
f-droid.org.

Reproducible Builds

For reproducible builds, we started out by doing mass rebuilds of all
apps in f-droid.org, as shown by https://verification.f-droid.org.
This let us fix the most common issues without getting stuck on a few
hard issues. Now that we have reproducibly built over 300 different
apps, we’re turning to focus on reproducibly building the most
security-sensitive apps. These tend to be the most difficult since
they frequently include “native” C code, which is much harder than
Java to build reproducibly.

Handling Media

While the core tools for adding media files to F-Droid repositories
were created months ago, we turned to focus on one specific use case
in order to polish up the media file support: the F-Droid Privileged
Extension “Over-The-Air (OTA) update”. This is a ZIP file that users
“flash” to their device to install it with elevated privileges. This
file is now built, signed, and released using the full F-Droid stack,
providing a trusted download method for users of any Android ROM to
flash to their device:
https://f-droid.org/repository/browse/?fdid=org.fdroid.fdroid.privileged.ota

That means the whole server-side deliver process is ready to handle
any file you can copy into a folder. The 1.0 release of the F-Droid
client app will fully handle installing common file types so that
media players, etc. will automatically find and play them. As part of
the Curation Tools section, RepoMaker already has some basic support
for handling media, which we are now working on completing and
polishing.

Developer Support

In collaboration with Guardian Project’s Developer Square effort, we
held a workshop on the internet called GLOW2017:
https://devsq.net/glow2017 . The videos are archived and available
for anyone to learn from.

Google Play Integration

When the Bazaar2 project was defined, there were not well known tools
for managing all of the localized files in Google Play. Now there are
two: Fastlane Supply and Triple-T Gradle Play Publisher. Both are
free open source software, so instead of reinventing the wheel, we
instead integrated with those existing tools. fdroidserver now
automatically detects the app store support materials in the app’s
source repo if it is already setup for Fastlane or Triple-T. So there
is now one place to put all of the app store materials (descriptions,
graphics, etc) to publish them to F-Droid and Google Play. Those
descriptions can be easily added to Weblate, Transifex, etc so that
the translations can be automatically synced when they are complete.

Objective 2 Curation Tools for Organizations

RepoMaker has reached a functional level with the core features
implemented. It is currently being developed around the two basic setup
modes: as a hosted web app. Apps can be manually added or automatically
fetched from other F-Droid app repos. RepoMaker can publish the repos
in all the same ways that fdroidserver can, e.g rsync GitHub, Amazon S3,
etc. There is a alpha demo of the multi-user mode for anyone to try:
http://repomaker.grobox.de

You can see demos of a number of key features in Torsten’s RepoMaker
playlist:
https://www.youtube.com/playlist?list=PLts8E5OKFffNMtw0HG3MaDiyfig-sfczT

We also began to build the foundations of the localization support.
This current implementation strategy will also allow for standalone
installations like a desktop app following the web app model like Riot,
Signal, etc.

Objective 3 Modern App Store with Built-in Circumvention

The new user experience is functionally complete and a full release,
v0.103, is now available via the normal release channels. We also
nailed down the full integrated experience using F-Droid Privileged
Extension, which allows for installs without enabling Unknown Sources
and automatic updates in background. It is now well tested and
working solidly on all Android versions. For the past month, we found
and fixed a number of issues specific to Android 7.x.

User Tests

We ran two parallel user tests in Lubbock, Texas and Vienna, Austria
of the new user experience for the F-Droid client app. Overall, we
are happy to say that they confirmed the general approach of the new
design, and users overwhelmingly found it simple to use. There were
two areas where users had difficulty: nearby app swapping and adding
new app repositories. This was not a surprise since, first and
foremost, those are totally new concepts for most mobile users, who
are used to getting everything from one source: Google Play.

The full report is available at:
https://docs.google.com/document/d/1WoyxBLnuYKt7GH2BKW2JnAL9rH-xg7QCvRrAwuRVBGI

Website

The new website is ready for launch, once we complete the secure,
automated deployment procedure. The new website is generated using
Jekyll and consists entirely of flat files with no code running on the
server side. On client-side, Javascript is only required for the
search function. This makes the website work well with Tor Browser,
and makes it easy for anyone to deploy their own app store using
simple cloud file hosting services like Alibaba Cloud, GitHub Pages,
Gitlab Pages, Amazon S3, etc. as well as simple appliance devices like
LibraryBox, FreedomBox, etc. We also began the process of making the
website fully translatable. The staging server is publicly available
here: https://fdroid.gitlab.io/fdroid-website/

Automated Circumvention

The fdroidserver tools for automated “collateral freedom” distribution
are in place. The current options for automatic publishing to mirrors
are: GitHub, Gitlab, Amazon S3, and SSH/rsync for webservers and Tor
Hidden Services. The F-Droid client app is already receiving the
metadata about those mirrors, but it does not yet automatically act on
it. Users can manually subscribe to individual mirrors now. The
Guardian Project app repo is currently setup for all of these types of
mirrors:

As for mirrors of f-droid.org, we launched a third mirror for the main
repo which is in the USA. This will better cover the Americas over
the two European mirrors.

Malware Tools

We added support for two sources of metadata about apps. Fdroidserver
can now automatically upload all new release to
https://androidobservatory.org and https://virustotal.com. These both
provide rich sources of metadata about apps and malware, viewable via
web pages or accessible via an API. They both are based on the SHA256
hash sum as a unique ID, so it is easy to link an APK on a device to
the data on those services. This data will be used to alert the user
to known malware in the new “Updates” tab of F-Droid client.

Objective 4 Partner Deployments

We have two prototype libraries for ensuring that apps have a
reliable, trusted update channel no matter where they were downloaded
from. There are lots of custom versions of this, from Firefox to
Signal. The libraries that we are creating are standardized, free
software libraries. They also integrate with the whole F-Droid
eco-system, using the same tools to manage the server-side as are used
for F-Droid “repos”. This provides the flexibility for app developers
to mix and match the features they need, like direct app updates via a
dedicated app repo, updates via https://f-droid.org, confirmed
reproducible builds of releases, “collatoral freedom” mirrors, etc.

Our first test implementations for these new libraries will be Zom for
the direct updates, and Ripple and Location Privacy for the F-Droid
update channel.

Objective 5 Usability Research on In-country Developers

The results of the survey have been compiled, and the public report is
nearing completion. We ran user tests of the fdroidserver tools in a
handful of locations. We were unable to run the tests in Eastern
Europe as we had hoped.


Comments