News

Bazaar: Bazaar2 Monthly Report - September 2017

Added by hans 2 months ago

  1. Bazaar2 Monthly Report - September 2017

This is the final report for the Bazaar2 project. We have wrapped it up with many launches:

  • the new tool Repomaker is now ready for wider use beyond prototyping
  • Debian 9 "Stretch" and Ubuntu 17.04 “Zesty” is the first release
    that includes the Android SDK complete enough to build apps

The Guardian Project work on F-Droid continues via two new funding
sources. The first is a project with Internews known as "Viento" to
improve the mobile experience on basic devices, and limited
internet. The second is a not yet public project with an organization
to build a new tool for training materials built on top of F-Droid.
We are also at various stages of project negotiation with some
companies who want to build on top of F-Droid.

For our final field test of this project, we set up a Copperhead
device with F-Droid and set to people at the Barys Zvozskau Belarusian
Human Rights House. We also demonstrated the Repomaker and command
line tools for managing custom repositories of apps and media. The
goal of this test was to see whether non-technical users with security
concerns would be willing to use a device maintained by a trusted
administrator who only allowed a small, curated set of apps to be
available on the devices. This prepared device was then passed around
to people in Belarus, Ukraine, and Russia for them to evaluate the
idea. The idea was interesting to them, but most thought they were
well served by Google Play now that it is no longer being blocked.
But most also agreed that if Google was blocked again, like it was in
Crimea after it was annexed, then there would be a lot of interest in
F-Droid.

We hear about a new app based on F-Droid being build by Jembi Health
Systems in South Africa. They have not made much public yet, but you
can follow their development efforts here:

  1. Strengthening the Foundations

One key reason why Guardian Project only works with free software is
because it empowers communities of users to maintain the software that
they find most valuable. On top of that, there are many opportunities
to work with existing free software communities on shared goals.
Combining efforts means the impact of the limited develop resources
can be greatly magnified. Everyone gets more bang for their buck.
One essential aspect of the Bazaar2 funded development effort was to
ensure that, on top of all of the new features added, the F-Droid
community should be able to more easily maintain the codebase. In
wrapping up, there is now a large, established automated test
infrastructure

  1. MD5 Transition Complete

Another example of foundational work was just completed: F-Droid now
fully handles the deprecation of the MD5 algorithm for signing Android
APK files. APK signatures are an essential part of the security of
Android, and the MD5 algorithm has been known to be weak for years
now. Oracle has disabled MD5 for Java JAR signatures, MD5 has been
banned in TLS certificate for a while now, but Google Play has not
blocked or even deprecated it yet.

  1. No Longer Beholden to Oracle

The f-droid.org build infrastructure is based on Oracle VirtualBox, a
virtual machine provider. While it is still free software, Oracle is
a capricious maintainer and changes things as they see fit, even if it
breaks things for many users. They recently dropped long term support
release, causing VirtualBox to be removed from Debian. F-Droid uses
Debian for all its servers. As part of the Bazaar2 project, we built
parallel tools built on community-controlled Linux KVM. This ensures
the future livelihood of the F-Droid project, whatever Oracle might
do. This was a large, undertaking that we did not expect to do 2
years ago. While this work was not originally part of the Bazaar2
Statement of Work, it was essential to keeping the whole project
going, and therefore essential to the goals of the Bazaar2 project.

  1. Weekly Meeting Logs

We have a weekly meeting on IRC mostly focused the developer facing
sides of F-Droid. That happens every Thursday at 11.30 UTC on
#fdroid-dev on FreeNode. The September 2017 logs can be found here:

https://botbot.me/freenode/fdroid-dev/2017-09-07

https://botbot.me/freenode/fdroid-dev/2017-09-14

https://botbot.me/freenode/fdroid-dev/2017-09-21

https://botbot.me/freenode/fdroid-dev/2017-09-28

  1. Following Work Related to this Funding

All related work on F-Droid is tagged using the "bazaar" label:

All related blog posts are tagged with the "bazaar" tag:

  1. Objective 1 Simple multi-pronged distribution
  1. Reproducible Builds

Reproducible builds as a standard publishing method turned out to be a
lot harder than we thought, mostly because of peripheral issues like
handling the virtualization stack (Virtualbox and KVM). One major
sticking point was the need to run virtual machines inside of virtual
machines, since our build infrastructure requires a virtual machine,
and Debian’s reproducible build servers run in KVM. But luckily,
interested in reproducible builds was also a lot higher than we
thought, so our efforts have brought F-Droid a lot of attention and
contributions.

Right now, it is possible to push apps to f-droid.org via the
reproducible build process, but it is difficult and error prone. We
have laid solid foundations for f-droid.org to be entirely based on
reproducible builds. What we have left to do is lots of polishing and
bugfixes.

  1. Make all text translatable

The last piece of the whole F-Droid suite is now fully localizable.
All strings in_fdroidserver_ can now be translated up on Weblate with
the rest of the F-Droid projects, and contributions are streaming in.
The fdroidserver 0.8 already included some localization support, the
next release will include the full support, and all of the
translations.

For tracking the localization work in F-Droid, see the localization
tag in the gitlab tracker:

  1. Objective 2 Curation Tools for Organizations

The Repomaker tutorials are complete, they just need to be deployed
and setup on Weblate for translations:

http://tutorials-fdroid-website-pserwylo.surge.sh/fdroid-website/en/tutorials/add-repo/

https://gitlab.com/fdroid/fdroid-website/issues/112

  1. Objective 3 Modern App Store with Built-in Circumvention

We have been getting quite a bit of feedback about the new automatic
vulnerability prompt. F-Droid 1.0 will prompt the user about any apps
that contain known vulnerabilities via the new Updates tab, which
serves are the notification and action center of the whole user
experience. Mostly, people have been reporting that it is finding
apps that they forgot they had installed. Often, people were a bit
confused by the prompt and asked things like: "the app was working
fine, why is F-Droid prompting me to remove it?" In some of those
cases, the user was using unmaintained browsers like Tint, which
definitely is a high risk activity on the internet. The biggest issue
with the current implementation is that we have no good way for the
user to find out more information about why it was marked, and what
the specific issues are. As we expand this feature to also include
apps marked by humans as vulnerable, we will need to provide an easy
channel for the user to find the whole story, with things like links
to CVE numbers, blog posts, etc.

  1. UX Overhaul

Now that the new UX is widely deployed, we are getting lots of
feedback, both positive and negative. Lots of people want to know why
we made certain decisions in the process. We tried to push that
process to the public as much as possible, so it is mostly documented
in the F-Droid issue tracker:

https://gitlab.com/fdroid/fdroidclient/issues?scope=all&utf8=%E2%9C%93&state=all&milestone_title=0.103%20-%20UX%20Overhaul

It was also nice to get some media coverage of our UX work:

http://www.androidpolice.com/2017/10/19/f-droid-open-source-app-repository-updated-v1-0/

  1. Website
  • Volunteer translators are adding more languages, all website
    translations that are in progress can be seen on the staging site:
    https://staging.f-droid.org
  1. Streamlining Circumvention

One last piece was fixed, deployed, and tested: making nearby swap
co-exist with Tor/proxy support.

  1. Translation

In closing, I want to call out Localization Lab's work as part of this
project. Their ongoing coordination of translators made it possible
to have the large amount of translations that have received. On top of
that, they made it easy to hire translators for focused work on the
high priority languages. Those translators then set to work without
needed any training or setup on the materials, since they were already
familiar with them.

For a nice graphical overview of the progress we have made, where are
charts of the languages and completeness for each of the F-Droid tools
that were made fully translatable. The F-Droid client app has been
translatable for a couple of years, so it has many more languages.
The documentation and blog posts are long form text, so they require a
lot more work to translate.

  1. Objective 4 Partner Deployments

We are in discussions with a potential client to build upon the
"Update Channels" library developed under this Objective. This work
would allow us to expand the possibilities for custom app stores and
media collections, and make the whole process a lot easier to do.

https://f-droid.org/2017/06/01/announcing-new-libraries-f-droid-update-channels.html

  1. Objective 5 Usability Research on In-country Developers

Nothing new to report, this work is complete.

Bazaar: Bazaar2 Monthly Report - August 2017

Added by hans 4 months ago

August was mostly about fixing bugs, polishing up, and waiting for feedback from users. The translators finished translating and reviewing, the last piece of the website was made localizable, the design and layout of the tutorials was finished, and Repomaker was packaged as a standalone desktop app.

@Hans and Carrie were at Sneakercon at Columbia University Journalism’s Brown Institute, a conference all about working with very limited internet access, including working offline, using sneakernets, and nearby, local networking. Carrie talked about the process for designing software to work with the limited internet, battery, and storage space common in most of the world. Hans talked about F-Droid's nearby app swapping, offline support, and decentralization. Overall the audience was quite a bit less technical than most events that we attend, so it was interesting to be talking about this in a different setting. For more info:

http://sneakercon.brown.columbia.edu/

As part of Sneakercon, we had a prototyping brainstorm session with Saycel to explore how to build a custom app store based on F-Droid. Saycel is a community-run telecom in Nicaragua built with free software like OsmocomBB. Mobile and fixed Internet access is available there, but it is very expensive and only available through two multi-national companies with little interest in rural areas. Saycel is able to provide services on their local network, so it can easily and cheaply run its own F-Droid app store. It turns out that one of the core F-Droid devs, @NicoAlt, has recently arrived for a long term stay in Nicaragua and is looking into furthering this idea.

While in New York, Hans also talked with Google about open-sourcing their Play Services/GMS libraries. These libraries have become very difficult to avoid. The use of these libraries is the only thing keeping a number of key apps from being built from source and included in f-droid.org, including Wire, Firefox, Zom, Signal, Mattermost, and more. There is an internal Google issue tracking the progress on this, and work is proceeding. Google’s approach looks to be moving these libraries to the Firebase name, and open-sourcing those new versions. It sounds like Play Services GMS libs will be folded into Firebase. There is some public evidence here:

Some other small achievements:

  1. Weekly Meeting Logs

We have a weekly meeting on IRC mostly focused the developer facing sides of F-Droid. That happens every Thursday at 11.30 UTC on #fdroid-dev on FreeNode. This month’s logs can be found here:

  1. Objective 1 Simple multi-pronged distribution
  1. Make all text translatable

The last piece of the localization of the f-droid.org website was completed, and the hired translators completed their work. We just need to get all the pieces integrated and launched. You can see the various working pieces here:

  1. Objective 2 Curation Tools for Organizations

Now that Repomaker is working well, we have shifted to working on making it easy to run and deploy it. This includes turning it into a desktop app for OSX and GNU/Linux. It is now buildable as a regular Debian/Ubuntu package. We also created a "vendorized package" that installs everything needed to run Repomaker. It is available here, please try it out on a Ubuntu or Debian machine: https://grobox.de/tmp/repomaker_0.0.1a_all.deb

  • The text and screenshots for the new F-Droid tutorials is now complete. You can see the prototype in action in this video: https://youtu.be/u2nl8mxwYo0
  • We ran our final field UX test in Zimbabwe. We are working through the feedback and are currently working on the final report.
  1. Objective 3 Modern App Store with Built-in Circumvention
  1. Media handling

We fixed a number of bugs related to media support in the 1.0 alphas.

  1. Streamlining circumvention

We have the last piece of the transparent mirror support working in a prototyped form. And F-Droid app/media repo can include a list of official mirrors. The fdroidserver tools already make it easy to automatically publish to mirrors on Amazon S3, GitHub, Gitlab, and any webserver. With this new feature, F-Droid will try to download apps from the next mirror in the list whenever a download fails. This automates the "collatoral freedom" technique of distributing files via various CDNs that are two popular to be blocked.

You can follow the final integration work here:

https://gitlab.com/fdroid/fdroidclient/merge_requests/578

  1. Local and peer-to-peer malware tools

We are still testing and finalizing what was implemented in June and July. We want to be extra sure that there are not false positives so that users do not learn to ignore this feature. Follow integration progress here:

https://gitlab.com/fdroid/fdroidclient/merge_requests/558

  1. Objective 4 Partner Deployments

We demonstrated the F-Droid offline mode of operation to people with experience working in Cuba and other places where very limited internet access is common. This confirmed that our approach will indeed improve the user experience when users can only access an F-Droid app store at limited times, and in limited locations. There is a quick video of how it works here:

https://gitlab.com/fdroid/fdroidclient/uploads/6515b76e22220a5a4b9e3e45815db764/offline.mp4

Follow the integration of this feature here:

https://gitlab.com/fdroid/fdroidclient/merge_requests/455

  1. Objective 5 Usability Research on In-country Developers

Nothing new to report, this work is complete.

Bazaar: Bazaar2 Monthly Report - July 2017

Added by hans 5 months ago

  1. Bazaar2 Monthly Report - July 2017

July was mostly focused on wrapping up things from the final development sprint, including field testing and translation. We have decided that, with the completion of Bazaar2 project, the F-Droid suite of software is ready to be called 1.0. We are finalizing a cross-project 1.0 release, so we made 1.0 alpha releases of the Android client, are preparing to launch the fully localized 1.0 website, and released the beta version 0.8 of the server/repo tools. In preparation for this big release, we also did a lot of polishing and QA work on the localization across the whole F-Droid project.

In other bits of news:

  • The new website now uses the translated app store materials already in use by the Android client app (e.g. description, summary, etc)
  1. Weekly Meeting Logs

We have a weekly meeting on IRC mostly focused the developer facing sides of F-Droid. That happens every Thursday at 11.30 UTC on #fdroid-dev on FreeNode. The July 2017 logs can be found here:

  1. Objective 1 Simple multi-pronged distribution

We added new tools on the server side to make it easier to build apps that have complicated setups. This is in response to issues that we worked through with Ooni Probe, VLC, and Barcode Scanner. One notable new feature is the new sudo= field, which is a place to specify setup commands that need to be run as root. Since the official F-Droid build process happens in a virtual machine guest instance (VM), each app’s build process can run commands as root without harming security. After each build, the VM is reset to the original state.

[https://gitlab.com/fdroid/fdroidserver/merge_requests/297](https://gitlab.com/fdroid/fdroidserver/merge_requests/297)

  1. Objective 2 Curation Tools for Organizations

After completing the first round of user tests on Repomaker, we discovered the need for users to have a complete understanding of F-Droid and how it works. Each of the tutorials we’ve outlined for the Bazaar project will be available on the F-Droid website, and will work together to provide users with a complete understanding of what they can do with F-Droid. Tutorials include: how to add a repo, how to send and receive apps offline and how to create your own repo. The layout of the tutorials is designed to be easily viewed on desktop computers, tablets and mobile phones. This is important for our target audience. They are also designed to be easily updated by the F-Droid team when UI updates are made.

In addition to the progress on tutorials, a second round of user tests were conducted with trainers in Zimbabwe. 5 participants completed the study, hosted by our partners at Digital Society.

  1. Objective 3 Modern App Store with Built-in Circumvention
  1. Integrating crash and bug reporting

With the overhaul of the app details screen in the Android client, it is now a lot easier for users to find the developer’s issue tracker when they want to send bug reports. Each app has its own metadata field for the issue tracker URL. There is also a field to specify the developer’s website, in case there is general information for a set of apps from a given developer.

As for F-Droid catching any app’s crash dumps, that is only possible from a "system priv-app". The F-Droid Privileged Extension runs as a system priv-app, and is a natural place to incorporate the ability to catch crash dumps. We completed a prototype of this:

https://gitlab.com/pserwylo/exception-logger

Since Privileged Extension is already included in shipping devices, and is a small package of security sensitive code, we want to be very conservative about including new features in it. The actual integration work is minimal, so it makes sense to keep crash dump interceptor as a separate prototype until it gets well tested.

  1. Media handling

Media handling has been completed with the integration of the final piece in the Android client. This functionality is already available in the 1.0 alpha0 release.

[https://gitlab.com/fdroid/fdroidclient/merge_requests/541](https://gitlab.com/fdroid/fdroidclient/merge_requests/541)

  1. Streamlining circumvention

We implemented the automated selection of "collateral freedom" mirrors in the Android client app. When the current mirror stops working, F-Droid will try the next mirror that it knows about until it finds a working one. Each time F-Droid connects to a repo, it will get the updated list of available mirrors. This will be included in the 1.0 release.

[https://gitlab.com/fdroid/fdroidclient/issues/35](https://gitlab.com/fdroid/fdroidclient/issues/35)

  1. Local and peer-to-peer malware tools

Installed apps with known vulnerabilities will now be flagged in the "Updates" screen of the F-Droid Android client. This known vulnerability information comes from the metadata downloaded from F-Droid repos. This feature will highlight vulnerable apps, no matter where they were installed from.

https://gitlab.com/fdroid/fdroidclient/issues/1070

  1. Objective 4 Partner Deployments

Nothing to report for July.

  1. Objective 5 Usability Research on In-country Developers

Nothing new to report, this work is complete.

Bazaar: Bazaar2 Monthly Report - June 2017

Added by hans 6 months ago

June marks the end of the final big development sprint for the Bazaar2 project, and many parts of this whole project have been completed, with others just needing some final bits and pieces completed. For the remaining couple months of the project, a few of us will be working to close out all those remaining bits and pieces to deliver the last sections of this whole funding effort.

One big piece of news was that Boris Kraut aka krt retired from active work on F-Droid https://forum.f-droid.org/t/so-long-farewell-and-goodbye. He was one of the major contributors to F-Droid over the past few years, leading up the fdroiddata section where apps are added to f-droid.org. He will certainly be missed. He retired with grace, and indeed provided a shining example of how to retire from a free software project, since he drummed up a lot of new interest, as well as new contributors, with his announcement.

One key part of the Bazaar2 project was to make F-Droid a fully localizable app store ecosystem. We localized the Android app, the website, the developer tools, and the documentation. So now basically every string a user sees can be translated. Some of this work was just applying well known software, but we forged new ground on a number of aspects. The details are under "Objective 1: Make all text translatable" and “Objective 3: Website“.

  1. Organizations running their own F-Droid "repos"

One key piece of this project was to polish up the F-Droid server tools so that it was easy for anyone to run their own F-Droid repository. This turns F-Droid into a decentralized distribution ecosystem, where anyone can choose which distribution sources they use, and anyone can become a distribution source themselves. Whether other organizations set up their own F-Droid distribution "repos" is an important measure of this project. The first example is Copperhead, which uses F-Droid as its only app store, and runs a number of custom app repos for clients. F-Droid allows Copperhead to deliver a tightly controlled mobile system that anyone can run without relying on the big gatekeeper organizations like Google or Apple. Another organization, Security First, has setup their own repo for their apps, including Umbrella ([https://secfirst.org/fdroid/repo/](https://secfirst.org/fdroid/repo/)). There is a relatively new app repo known as IzzySoft (https://apt.izzysoft.de/fdroid) that is fulfilling an important role in the whole ecosytem. F-droid.org only includes apps that are 100% free software, built from source code. That excludes a lot of valuable software that includes proprietary libraries like Google GCM. IzzySoft includes lots of apps like these, serving as a stepping stone on the way to inclusion in f-droid.org.

We have also been working with Fairphone to get F-Droid integrated into their Fairphone Open Android system. They are working towards selling Fairphone Open devices directly on their website, so once that launched, then that will be the first hardware manufacturer shipping F-Droid that we know about.

And two last additional items we touched in June:

  1. Full localization waiting on final deployment

We can now show all the key parts of F-Droid localized, this will all be shipped in the next release of the various components.

  1. Weekly Meeting Logs

We have a weekly meeting on IRC mostly focused the developer facing sides of F-Droid. That happens every Thursday at 11.30 UTC on #fdroid-dev on FreeNode. The June 2017 logs can be found here:

  1. Following Work Related to this Funding

All related work on F-Droid is tagged using the "bazaar" label:

All related blog posts are tagged with the "bazaar" tag:

  1. Objective 1 Simple multi-pronged distribution
  1. Make all text translatable

In June, we had a major push to get all the strings throughout the project, from app strings to documentation, in a format that works well for automated translation. Those are all up on Weblate now, open for contributions. At least 95% of the strings used in the F-Droid software is now translatable and upon Weblate for easy contributing. We have been getting a steady stream of translation contributions in a variety of languages. We also hired some translators to finish the community translations and review them for Farsi, Simplified Chinese, and Spanish. We did not receive any contributions in Tibetan, but have hired two people to translate and review all strings in the F-Droid app, Repomaker, 10 app descriptions, and much of the website material. The source, translations, and activity for all the F-Droid projects can be seen on the Weblate project page:
https://hosted.weblate.org/projects/f-droid

An essential part of the work we do is integrating with other free software projects, and helping those projects improve. In order to provide a complete, smooth translation workflow, we working through a issues in four separate projects that each form an essential piece of the puzzle.

For tracking the localization work in F-Droid, see the localization tag in the gitlab tracker:

  1. Reproducible builds

We have be reproducing Android app builds for some months now on [https://verification.f-droid.org](https://verification.f-droid.org), it has reproducibly built 372 APKs from 319 different apps. The whole F-Droid ecosystem can now support matching APKs with an arbitrary number of signers. This is the last key blocker to allowing f-droid.org to also add the developer’s signature for any app that is built. Previously, the F-Droid tools only supported a single signer, and that signer was f-droid.org. This is also an important feature for cases where people are working with collections of APKs like Repomaker users or the Cuban app store example.

We have collected a large number of APKs that include the original developer’s signature, and are working to retroactively add the developer’s signed APK to f-droid.org whenever the build can be reproduced. Here are the signatures we are currently working with:

https://gitlab.com/fdroid/fdroiddata/merge_requests/2241

  1. Objective 2 Curation Tools for Organizations

In the beginning of June, our design lead did an user experience test where potential users of Repomaker tried out the software. From this test, we got lots of feedback to improve he workflow of Repomaker. Most of these improvements have already been implemented.

  • improved workflow for managing storage services
  • improved workflow for adding apps from remote repos
  • app details of remote apps
  • internationalization of JavaScript code
  • drag and drop to upload files
  • lots of other improvements after ux test
  • currently under review: endless scroll through apps
  1. Objective 3 Modern App Store with Built-in Circumvention
  1. Website

We have launched the new static site on https://f-droid.org, replacing the Wordpress site that has served us well for the past 6 or so years. This is the foundation for the fully localized website. We set a high standard for ourselves with this new localized website, in terms of the use cases we wanted to cover. On our staging server now is a version of the website that covers basically everything that we wanted to do:

  • fully localized without requiring Javascript or setting the language in the browser/system
  • automatic language selection based on browser preference
  • any supported language can be selected directly via a menu
  • static site of only files to greatly simplify the hosting and security maintenance
  • polish workflow with static site generation (Jekyll)
  • a static site is also more resistant to DoS attacks, especially when using a major CDN

The goal was to support both the most private setups as well as the most automatic. The site is designed to work well with both the bog standard Tor Browser or TAILS setup, as well as the standard Javascript-enabled browser with the language preference included in every web request. A high risk user can keep the default language, then only select their preferred language only when they require a translation for a given page, whether or not Javascript is enabled. Setting the language preference in the browser or the system can divulge a lot of information about a user, especially if it is a minority language. So we ensured that it was not a requirement for getting localized pages. We are happy to consult with other projects who have similar goals.

  1. UX Overhaul

We polished up two parts of the new Android user experience:

  1. Streamlining Circumvention

We sketched out how to implement the final missing piece of the work to automatically use "collateral freedom" mirrors. The F-Droid client will get the list of official mirrors from any repo that supports mirrors. It will then automatically retry failed downloads using the next available mirror. F-Droid repos can now automatically be hosted on Amazon S3, GitHub, Gitlab, and any webserver accessible via SSH. That webserver can then provide a Tor Onion Service. The Guardian Project F-Droid Repo is setup like this, here are the current mirrors (also visible at the top of the repo XML https://guardianproject.info/fdroid/repo/index.xml):

  1. Add media handling to app store experience

With the release of 1.0-alpha0, the F-Droid client can finally support "installing" media files. For common file types like music, video, etc. the files are downloaded into the standard Android folders for storing those media types (e.g. Music, Movies, etc). Then any Android app that handles those files will find and use them automatically. We had to forge new ground for OTA (Over-The-Air) update ZIP files, since there is no other app store that ships those kinds of files. In this case, F-Droid puts them into a standard, protected folder that is only accessible by the Android “recovery” system that installs such updates (e.g. TWRP).
https://gitlab.com/fdroid/fdroidclient/merge_requests/541

Bazaar: Bazaar2 Monthly Report - May 2017

Added by hans 7 months ago

May was another busy month for us. Now that we have released the new F-Droid Android client with an entirely new user experience, we shifted development focus to localization of the whole F-Droid suite of tools while fixing bugs as they arose. We also completed all of the usability research and user testing, and published the results.

There was a big focus on localization of all of the pieces of the F-Droid ecosystem. With that, we had an exciting realization: once our translators complete their work, then F-Droid will be the first app store fully available in Tibetan. Google Play and the big Chinese app stores do not support Tibetan on their websites, Android apps, etc. Even iTunes does not support Tibetan, even though iOS provides good support for Tibetan. This also opens the door to many other poorly represented languages, since translation is now the only thing needed to create a complete app store in any language Android supports. We already have active contributors for languages absent from the major app stores, like Arabic, Armenian, Belarusian, Burmese, Hebrew, and Shona.

To gauge interest in F-Droid, we sent a Copperhead/F-Droid device to be passed around a few internet freedom groups in Belarus and Ukraine to get feedback on whether they consider this a usable solution for them. We did a similar test about a year ago, and found that F-Droid was not useful there for high security users because it required Unknown Sources. Psiphon was enough to get around Google Play blocking, and complete internet outages are uncommon. Now with F-Droid built into Copperhead, we have a compelling offering that is not available with Google Play devices: a secure mobile phone that can only install from a small, trusted whitelist of available apps. This can then also be fully localized.

All of this localization work will be polished up and deployed starting in June. To see the development history and follow progress, check the “localization” label in the F-Droid gitlab trackers:
https://gitlab.com/groups/fdroid/issues?label_name%5B%5D=localization&scope=all&state=all
https://gitlab.com/groups/fdroid/merge_requests?label_name%5B%5D=localization&scope=all&state=all

In other news:

###Meetly Meeting Logs

We have a weekly meeting on IRC mostly focused the developer facing sides of F-Droid. That happens every Thursday at 11.30 UTC on #fdroid-dev on FreeNode. The May 2017 logs can be found here:

https://botbot.me/freenode/fdroid-dev/2017-05-04/?msg=85141062
https://botbot.me/freenode/fdroid-dev/2017-05-11/?msg=85471490
https://botbot.me/freenode/fdroid-dev/2017-05-18/?msg=85800895
https://botbot.me/freenode/fdroid-dev/2017-05-25/?msg=86123535

###New contributors

When we put together the Bazaar2 project, we thought that one important measure of the success of this project would be if more technically skilled people volunteer their time to improve F-Droid software. On top of the existing active contributors, a number of new contributors gave their time to F-Droid since January 2016, when this project started. Here are some notable new contributors:

We also managed bring some new people who were already contributors to other free software projects: Last but not least, the Bazaar2 project brought in a number of existing community contributors, allowing them to spend solid chunks of focused time working on software that they were originally drawn to by their own perception of the importance of the problems and the interesting opportunities for solving them:

###Following Work Related to this Funding

All related work on F-Droid is tagged using the “bazaar” label:
All issues: https://gitlab.com/groups/fdroid/issues?label_name%5B%5D=bazaar&scope=all&state=all

All merge requests: https://gitlab.com/groups/fdroid/merge_requests?label_name%5B%5D=bazaar&scope=all&state=all

All related blog posts are tagged with the “bazaar” tag:
https://guardianproject.info/tag/bazaar

##Objective 1 Simple multi-pronged distribution

In May, we finally completed the longest running merge request in F-Droid:
https://gitlab.com/fdroid/fdroidserver/merge_requests/176

This allows us to rebuild the entire buildserver from scratch as well as the latest version of every single app in f-droid.org. This all happens on a weekly basis, providing both an amazing continuous integration test on the F-Droid infrastructure as well as a platform doing reproducible builds on a mass scale. This is all running on Debian’s servers for reproducible builds:
https://jenkins.debian.net/view/reproducible/view/F-Droid

##Objective 2 Curation Tools for Organizations

A great amount of progress was made on the UX and UI of the Repomaker tool. By the end of the month, we had designs ready and implemented for the core functionality. The app was prepared for usability studies where we would test how trainers would use it to create collections of apps from other repos to share with a group of trainees. In the month of May, we also created a test plan to use internally for our tests and to share with our partners running field tests. This plan can be viewed here:
https://docs.google.com/document/d/1Nx4hP67vnffTzcLR_uqNe3J3UfvpOLUMY6k3PfLV3Q0

Below is a bullet point list of the major areas of design that were completed this month.

UX Design of Repomaker:
- workflow for creating a repo
- workflow for editing metadata
- workflow for adding apps from another repo
- workflow for creating and managing multiple repos
- workflow for accessing the user account and logging out

UI Design of Repomaker:
- empty state for the home view of all repos
- app details view
- edit mode for app details
- browse through other repos view
- repo index view
- repo info view
- repo share view
- repo homepage (public link) on desktop and mobile
- login/signup page
- drag and drop areas
- styling and font selections for the entire app

Videos showing the design progress:
May 11, 2017 https://www.youtube.com/watch?v=dLSiaEddjc8
May 4, 2017 https://www.youtube.com/watch?v=qMN9ZPbMyfE

We also published Repomaker’s source strings to Weblate for the first time, and already received Spanish and Turkish translations.

##Objective 3 Modern App Store with Built-in Circumvention

###Website

The biggest item was coming up with a workflow for allowing translations of the various pages on the new website. This includes (in order of priority) the content on the home page, the server documentation, and the news posts.

Due to some technical details about the websites implementation (using Jekyll + polyglot + po4a) the home page is internationalized using a different technique to that of the documentation + news posts. The result of this is that each of these three sections will have their own Weblate project for translators to contribute towards.

There are currently two pending merge requests, one for internationalizing the home page, and the other for the documentation + news posts. These are not earmarked for the launch milestone. However once the launch is done, then this can be tested, merged, and released.

The new website was finalized, the secure, automated deployment procedure is still being setup https://gitlab.com/fdroid/fdroid-website/merge_requests/72 https://gitlab.com/fdroid/fdroid-website/merge_requests/74

We finalized the architecture to support full localization, based on the jekyll polyglot plugin and the setup that Apache HTTPD’s official documentation uses. https://gitlab.com/fdroid/fdroid-website/issues/15#note_29880424 https://gitlab.com/fdroid/fdroid-website/merge_requests/70

###UX Overhaul

In addition to the website translations, there was also a handful of miscellaneous bug fixes for the client in response to the 0.103.1 release. This resulted in a 0.103.2 release which should be more stable, and there are also some more stability fixes which I have completed that will be merged in June to make a 0.103.3 which addresses more stability concerns.

We also completed the final user test of the Android client user experience overhaul as it is currently implemented. The results are available in this report:
https://docs.google.com/document/d/1WoyxBLnuYKt7GH2BKW2JnAL9rH-xg7QCvRrAwuRVBGI

###Streamlining Circumvention

The new “collateral freedom” mirroring on Gitlab, GitHub, and Amazon S3 was polished up so it is easy to use. This also now in used in Repomaker.:
https://gitlab.com/fdroid/fdroidserver/merge_requests/271
https://gitlab.com/fdroid/fdroidserver/merge_requests/272

##Objective 5 Usability Research on In-country Developers

The final results of this usability research effort are now available in final report written by Seamus Tuohy. Seamus has been part of our weekly meetings, and discussions over this research and findings have contributed directly to our development process. We hope that many other projects can learn from these published reports of our findings. The full report, “Technical Collaboration in a Closing World”, is now published on the Guardian Project website:
https://guardianproject.info/2017/05/15/research-report-on-developer-challenges

Also available as a Google Doc:
https://docs.google.com/document/d/1FS6fHyT5FFHMiDLMSccxTMkcsC85oXjhvvXJWY1Ox6A

The last piece of Objective 5 was usability research for developer tools. Seamus designed and ran user tests of the “fdroidserver” developer tool suite. This test confirms the basic usability of the tools and the documentation, while providing confirmation of the importance of localization. It will also serve as a guide for future work, especially on the documentation but also on the tool itself.

https://guardianproject.info/2017/06/01/fdroidserver-ux-testing-report/

Also available as a Google Doc:
https://docs.google.com/document/d/1uttj5knmFA_Z0SuOqoGBXHXSA5tQqDd7VUoHPX8vDUA

Bazaar: Bazaar2 Monthly Report - April 2017

Added by hans 8 months ago

April was a big month for us in terms of finishing up some big parts
that are directly visible to users, and easy to demonstrate. The
biggest is the final 0.103 release of the F-Droid app which includes
the complete overhaul of the user experience, which feels simple,
friendly and modern. This is one short step from a big 1.0 release,
once we nail down the last features and get some more testing
completed.

We also launched the first alpha of the new F-Droid Repomaker, a
simple web tool for creating and managing collections of apps and
media, and delivering them to users via F-Droid repositories (aka
“repos”). Try the alpha demo! http://repomaker.grobox.de/

On top of those two launches, there are many other small
accomplishments from this biggest and final development sprint for
Bazaar2.

Objective 1 Simple multi-pronged distribution

Make All Text Translatable

All texts within F-Droid and graphics associated with apps are now
translatable, including all the strings within the app itself, all app
names, summaries, descriptions, video links, recent changes, and
screenshots. With release of F-Droid client 0.103, it will use any
available language. For the F-Droid client app itself, many languages
are completely translated, and many more have reached the functional
level, thanks to the ongoing support from F-Droid community volunteers
and the Localization Lab:

  • 19 over 99%, including Belarusian, Brazilian, Persian, Russian,
    Spanish, Chinese, Turkish
  • 32 over 90%, including Arabic, French, Italian, Romanian, Shona, Ukrainian
  • 45 over 70%, including Burmese, Hungarian, Korean, Simplified Chinese,
    Thai, Vietnamese
  • see all and contribute here:
    https://hosted.weblate.org/projects/f-droid/f-droid/

We have not received any Tibetan translations yet. We will be hiring
translators to finish the Simplified Chinese and Tibetan translations.

For the per-app materials, we are now adding all the translated
materials for all the Guardian Project apps to the Guardian Project
F-Droid Repository, which users can enable with the flip of a switch
in F-Droid. We are also helping app developers to get their
descriptive materials integrated for automatic inclusion in
f-droid.org.

Reproducible Builds

For reproducible builds, we started out by doing mass rebuilds of all
apps in f-droid.org, as shown by https://verification.f-droid.org.
This let us fix the most common issues without getting stuck on a few
hard issues. Now that we have reproducibly built over 300 different
apps, we’re turning to focus on reproducibly building the most
security-sensitive apps. These tend to be the most difficult since
they frequently include “native” C code, which is much harder than
Java to build reproducibly.

Handling Media

While the core tools for adding media files to F-Droid repositories
were created months ago, we turned to focus on one specific use case
in order to polish up the media file support: the F-Droid Privileged
Extension “Over-The-Air (OTA) update”. This is a ZIP file that users
“flash” to their device to install it with elevated privileges. This
file is now built, signed, and released using the full F-Droid stack,
providing a trusted download method for users of any Android ROM to
flash to their device:
https://f-droid.org/repository/browse/?fdid=org.fdroid.fdroid.privileged.ota

That means the whole server-side deliver process is ready to handle
any file you can copy into a folder. The 1.0 release of the F-Droid
client app will fully handle installing common file types so that
media players, etc. will automatically find and play them. As part of
the Curation Tools section, RepoMaker already has some basic support
for handling media, which we are now working on completing and
polishing.

Developer Support

In collaboration with Guardian Project’s Developer Square effort, we
held a workshop on the internet called GLOW2017:
https://devsq.net/glow2017 . The videos are archived and available
for anyone to learn from.

Google Play Integration

When the Bazaar2 project was defined, there were not well known tools
for managing all of the localized files in Google Play. Now there are
two: Fastlane Supply and Triple-T Gradle Play Publisher. Both are
free open source software, so instead of reinventing the wheel, we
instead integrated with those existing tools. fdroidserver now
automatically detects the app store support materials in the app’s
source repo if it is already setup for Fastlane or Triple-T. So there
is now one place to put all of the app store materials (descriptions,
graphics, etc) to publish them to F-Droid and Google Play. Those
descriptions can be easily added to Weblate, Transifex, etc so that
the translations can be automatically synced when they are complete.

Objective 2 Curation Tools for Organizations

RepoMaker has reached a functional level with the core features
implemented. It is currently being developed around the two basic setup
modes: as a hosted web app. Apps can be manually added or automatically
fetched from other F-Droid app repos. RepoMaker can publish the repos
in all the same ways that fdroidserver can, e.g rsync GitHub, Amazon S3,
etc. There is a alpha demo of the multi-user mode for anyone to try:
http://repomaker.grobox.de

You can see demos of a number of key features in Torsten’s RepoMaker
playlist:
https://www.youtube.com/playlist?list=PLts8E5OKFffNMtw0HG3MaDiyfig-sfczT

We also began to build the foundations of the localization support.
This current implementation strategy will also allow for standalone
installations like a desktop app following the web app model like Riot,
Signal, etc.

Objective 3 Modern App Store with Built-in Circumvention

The new user experience is functionally complete and a full release,
v0.103, is now available via the normal release channels. We also
nailed down the full integrated experience using F-Droid Privileged
Extension, which allows for installs without enabling Unknown Sources
and automatic updates in background. It is now well tested and
working solidly on all Android versions. For the past month, we found
and fixed a number of issues specific to Android 7.x.

User Tests

We ran two parallel user tests in Lubbock, Texas and Vienna, Austria
of the new user experience for the F-Droid client app. Overall, we
are happy to say that they confirmed the general approach of the new
design, and users overwhelmingly found it simple to use. There were
two areas where users had difficulty: nearby app swapping and adding
new app repositories. This was not a surprise since, first and
foremost, those are totally new concepts for most mobile users, who
are used to getting everything from one source: Google Play.

The full report is available at:
https://docs.google.com/document/d/1WoyxBLnuYKt7GH2BKW2JnAL9rH-xg7QCvRrAwuRVBGI

Website

The new website is ready for launch, once we complete the secure,
automated deployment procedure. The new website is generated using
Jekyll and consists entirely of flat files with no code running on the
server side. On client-side, Javascript is only required for the
search function. This makes the website work well with Tor Browser,
and makes it easy for anyone to deploy their own app store using
simple cloud file hosting services like Alibaba Cloud, GitHub Pages,
Gitlab Pages, Amazon S3, etc. as well as simple appliance devices like
LibraryBox, FreedomBox, etc. We also began the process of making the
website fully translatable. The staging server is publicly available
here: https://fdroid.gitlab.io/fdroid-website/

Automated Circumvention

The fdroidserver tools for automated “collateral freedom” distribution
are in place. The current options for automatic publishing to mirrors
are: GitHub, Gitlab, Amazon S3, and SSH/rsync for webservers and Tor
Hidden Services. The F-Droid client app is already receiving the
metadata about those mirrors, but it does not yet automatically act on
it. Users can manually subscribe to individual mirrors now. The
Guardian Project app repo is currently setup for all of these types of
mirrors:

As for mirrors of f-droid.org, we launched a third mirror for the main
repo which is in the USA. This will better cover the Americas over
the two European mirrors.

Malware Tools

We added support for two sources of metadata about apps. Fdroidserver
can now automatically upload all new release to
https://androidobservatory.org and https://virustotal.com. These both
provide rich sources of metadata about apps and malware, viewable via
web pages or accessible via an API. They both are based on the SHA256
hash sum as a unique ID, so it is easy to link an APK on a device to
the data on those services. This data will be used to alert the user
to known malware in the new “Updates” tab of F-Droid client.

Objective 4 Partner Deployments

We have two prototype libraries for ensuring that apps have a
reliable, trusted update channel no matter where they were downloaded
from. There are lots of custom versions of this, from Firefox to
Signal. The libraries that we are creating are standardized, free
software libraries. They also integrate with the whole F-Droid
eco-system, using the same tools to manage the server-side as are used
for F-Droid “repos”. This provides the flexibility for app developers
to mix and match the features they need, like direct app updates via a
dedicated app repo, updates via https://f-droid.org, confirmed
reproducible builds of releases, “collatoral freedom” mirrors, etc.

Our first test implementations for these new libraries will be Zom for
the direct updates, and Ripple and Location Privacy for the F-Droid
update channel.

Objective 5 Usability Research on In-country Developers

The results of the survey have been compiled, and the public report is
nearing completion. We ran user tests of the fdroidserver tools in a
handful of locations. We were unable to run the tests in Eastern
Europe as we had hoped.

Bazaar: Bazaar2 Monthly Report - March 2017

Added by hans 10 months ago

Finally, after many months of doing behind the scenes plumbing, we now have a steady stream of very visible progress. The big news is that we launched our first client app alpha of the totally new user experience, after an intense development sprint. You can get it now in F-Droid by finding F-Droid in installed apps, and then selecting version 0.103-alpha from the list.

+ Implemented totally new designs for the Categories/Main/Updates screens
  + Better support for offline usage of F-Droid
  + Drastically improved workflow for bulk downloads + updates
  + New support for screenshots, feature graphics, and localized descriptions
  + https://post--new-ui-fdroid-website-pserwylo.surge.sh/2017/04/04/new-ux.html

We had a good meeting with Fairphone at their lovely Amsterdam office, and nailed down a plan to get F-Droid integrated in Fairphone Open OS, which can be installed on any Fairphone2. They are also working on shipping Fairphone OS devices directly. From Fairphone, we learned about https://uhuru-mobile.com/ which already includes F-Droid as its app store. Uhuru provides an open source “Mobile Device Manager” service which will integrate nicely with the F-Droid Repomaker service being developed from the “2 Curation Tools” effort.

There was also a lot of presentation activity in March. Torsten and Seamus attended the Internet Freedom Festival. Hans presented F-Droid at the Android Security Symposium (https://youtu.be/yBxIVM0-3Vk) and RightsCon, and attended Tor Dev Meeting and Iran Cyber Dialogue, where F-Droid was a topic of discussion. Seamus was also at Iran Cyber Dialogue and RightsCon.

At the Android Security Symposium (https://usmile.at/symposium/), there were lots of related discussions at the various private meals for the speakers, which included key security people from Google, AT&T, universities and private security research companies. There was agreement that the most effective single security measure is limiting access to what apps can be installed on the device. We agree, and are working to support this kind of setup, since it will be very useful for lots of high risk users. This is the same model used by Copperhead, Uhuru Mobile, Fairphone Open, and many DIY projects. To make this possible, the essential part is giving organizations control over the apps that they make available, and making this as easy as possible to manage.

Also, Nico Alt has joined us working on F-Droid as part of the Bazaar2 funding. He's a long time F-Droid contributor, working on the client, leading up the new forum, and the new website design.

Objective 1 Simple multi-pronged distribution

The new “binary transparency log” feature is now available. The idea is to publish an append-only log of all the binaries that an update system has published. Then anyone can check that the binary that they received on their device matches the official list based on hash. This feature has two parts:

1. Any F-Droid repository can make its own binary transparency log directly when `fdroid update` runs. This first example of this can be seen here: https://github.com/guardianproject/binary_transparency_log
2. Anyone can point the new `fdroid btlog`command at any F-Droid repository to make their own local log. This is designed to be run often so it will stay updated. Here is the first public version of a version we had running privately since 2014 that was pointed at https://f-droid.org: https://github.com/guardianproject/f-droid.org_binary_transparency_log

Other interesting tidbits:
  • Reproducible builds bug in the Android SDK bug reported by us was officially confirmed https://code.google.com/p/android/issues/detail?id=231886 Google is interested in reproducible builds these days, and seems to be fixing them.
  • The F-Droid server tools now support fully localized app metadata, including screenshots, feature graphics, and descriptions.
  • A full Android SDK is now included in Debian Stretch, so you can `apt install android-sdk` https://bits.debian.org/2017/03/build-android-apps-with-debian.html
  • We have preliminary free software Android emulator images that we aim to ship, since Google now only ships proprietary Google Play images. This makes it easy for people to develop using only the F-Droid stack: https://gitlab.com/fdroid/emulator-system-images
  • F-Droid server tools can now automatically upload releases to Android Observatory and VirusTotal. These services generate lots of useful indexes for discovering and tracking malware.

Objective 2 Curation Tools for Organizations

The first functional prototype of Repomaker (https://gitlab.com/fdroid/repomaker), the current name for the web tool building built to make it easy for anyone to build and manage F-Droid repositories. Here is a video of the prototype in action: https://youtu.be/GbpEX1LroRk There is also a video of the design prototype: https://youtu.be/yc4K9D7BCDU

We are also looking at the Flyve Mobile Device Management software since it provides some complementary and some overlapping functionality. It looks like the full source is available. It is also a web app, but written with PHP rather than Repomaker’s Python. The source is here: https://github.com/flyve-mdm and a free demo is available here: https://flyve-mdm.com/

Objective 3 Modern App Store with Built-in Circumvention

In March, the new user experience was mostly completed and is now available as an alpha release: 0.103-alpha. In addition, there were some additions to the UI which were implemented in response to the two user tests that we ran, one in Texas and the other in Vienna. F-Droid client now has much better support for the following, long awaited features:

  • Bulk Download: The previous stable release of F-Droid had rudimentary support for downloading multiple apps at once. However the feedback to the user was incomplete and it was prone to forgetting that a user had downloaded some apps (e.g. if they close F-Droid and come back later).
  • Now there is first class support for viewing the status of each download in one location, the "Updates" tab. This also includes all of the apps which
    can be updated, and will make it easier in the future to show other important information about each app (e.g. if security vulnerabilities are found, or if an app has to be removed from the repo).
  • Offline queue for download: One thing F-Droid can do that other stores cannot, is to let the user browse through apps while offline. Now, users are notified that they are using F-Droid without internet access. As they view apps, they are prompted to "Download later" which puts apps in a queue, to be shown in the "Updates" tab. This queue is automatically downloaded when they next come online. This feature is completed, but not yet merged into master.
  • The totally overhauled website is nearing launch. We have the full website built now using the Jekyll static site generator. We just need to nail down a secure and automated deploy process. This whole setup makes it much easier to run the F-Droid infrastructure since there will be almost no server-side code running. And it can be flexibly reused in custom app stores based on F-Droid.
  • We polished up the “F-Droid Privileged Extension”, which allows F-Droid to work without Unknown Sources, and do fully automated background updates. We worked with CopperheadOS to make sure that this system works well in the latest Android release, 7.1.1.
  • We submitted a complete patch to FairphoneOS to build and include the F-Droid Privileged Extension into their Fairphone Open builds as the core of the F-Droid integration: https://code.fairphone.com/gerrit/#/c/27/
  • We worked with security researchers who work on the CVE system and prototyped a way to support Android/Java libraries in the CVE system so that the automated scanners that we have implemented can use the CVE system as a source of data about known vulnerabilities. This data can then be used downloaded by the F-Droid client app to report known issues with any apps that are installed.

Bazaar: Bazaar2 Monthly Report - February 2017

Added by hans 10 months ago

Now that a lot of the work we have done over the past year is solidifying, we have started to do a lot more to promote it. To that end, there will be lots of activity at conferences around the world, as of February:

  • Peter represented F-Droid at FOSDEM in Brussels
  • Hans at Android Security Symposium in Vienna
  • Hans at RightsCon: “Internet Freedom App Store: we require alternatives to the two gatekeepers”
  • Hans at Iran Cyber Dialogue
  • Torsten at http://www.cubaconf.org in Havana
  • Peter at http://droidcon.vn in Ho Chi Minh City

There were also some interesting developments from people entirely unrelated to the F-Droid core developers and Bazaar2 development effort.

Objective 1 Simple multi-pronged distribution

We made progress on lots of little details over the past month, and some bigger, long running efforts. First and foremost, we know have an entire build infrastructure based on KVM that can run within a KVM guest (aka “nested KVM”). This setup is now running once a day on https://jenkins.debian.net. This will be the basis of our weekly rebuilds of the entire f-droid.org collection of apps to provide the feedback for working towards reproducible builds for as many apps a possible. Running the whole process from the very beginning each week gives us continuous integration testing for our whole build infrastructure.

  • we started working with libscout to detect library versions in apps. This will allow us to work with CVEs and other data sources for marking known vulnerabilities in libraries. This data is then included in app index metadata, which F-Droid can then use on the device to highlight vulnerable apps to prompt the user to update or uninstall.
  • we worked with a Cuban user group to fix the issues that arose from building an F-Droid app repository from 12,000 APK files.
  • we got our bug fixes integrated into the Debian packages needed to run the build intrastructure

Objective 2 Curation Tools for Organizations

We held a kick-off meeting in order to lay out the design issues and to set the stage for deciding the technical approach of the whole project. We decided to go with a web app over an Android app for a number of reasons, including that it was the most flexible approach. Carrie sketched the basic workflow to get the ball rolling. There is lots more information on the backstory of this work in Torsten’s blog post:
https://guardianproject.info/2017/02/22/build-your-own-app-store-android-media-distribution-for-everyone/

Objective 3 Modern App Store with Built-in Circumvention

The F-Droid Privileged Extension is now shipping with CopperheadOS and Replicant, so those devices no longer need to turn on “Unknown Sources” in order to use F-Droid. This also provides fully automatic background updates. Next steps are to get the Privileged Extension integrated into more devices and ROMs, and to make it easy for all the custom Android ROM developers to properly integrate F-Droid into their projects.

UX Overhaul

We have been working on wrapping up the designs for the improvements in the UX and UI that we are making after the first round of user tests. We will be doing another round of user tests in late March, this time with alpha releases of the real app, to confirm the design, and find and last glaring issues. In addition to the feedback from user tests, we have also received lots of great, unsolicited feedback from the F-Droid community via our issue tracker. While it was extra effort for us to have the design discussions on a public forum, it has paid off due to the quality of the discussions that we had there, including detailed reviews based on the Material Design Guidelines and ideas for handling some of the tricky design problems. This thread is a great of example:
https://gitlab.com/fdroid/fdroidclient/issues/709

The major design improvements include:

User Testing

I’ve outlined the areas that we’d like to gain feedback on in the next round of tests. The primary UX flows we want feedback on include: users’ ability to update apps, the offline experience, and the experience of searching within a category. We also are looking for feedback on users’ comprehension of the new menu icons, how much they trust F-Droid, and how likely they are to donate to developers.

Objective 4 Partner Deployments

We finalized the design of update libraries in conjunction with the Tibetan partner organization, and signed a contract for it to be implemented by Mark Murphy aka @commonsguy. These two libraries work together to provide alternate paths to app updates:

Objective 5 Usability Research on In-country Developers

The developer survey was completed and translated into Spanish, Chinese, Farsi, and Russian. It is now available at https://challenges.tech/ Seamus started the testing and promotion of the survey with the aim to kicking it off at Internet Freedom Festival in Valencia.

Bazaar: Bazaar2 Monthly Report - January 2017

Added by hans 11 months ago

This past month was dominated by organizing the upcoming large development sprint starting in February. This means hiring a number of people to do all the work. We had 20+ applications, lots of email, and 5 interviews. We hired two experienced developers, and 4 part time junior developers.

There were also a few notable achievements in the development work:
  • Completed an automated system for mass-verifying reproducible builds
  • Finalized possible technical approaches for curation tools
  • F-Droid website converted into a app store website toolkit
  • Designed multi-language survey about developer challenges
  • Designed user test of the developer tools and documentation

The first results from the user research into developers have been published:
https://guardianproject.info/2017/01/26/imagining-the-challenges-of-developers-in-repressive-environments/

Objective 1 Simple multi-pronged distribution

We now have https://verification.f-droid.org/ automatically building the latest apps and testing whether they are reproducible. We are up to 59 apps that can be built reproducibly using the F-Droid tools. To see which apps, search for “verified” on https://verification.f-droid.org/. Now that we have a mass rebuild process running automatically, the next step is to focus on some more important apps in order to fix the issues preventing them from being rebuilt reproducibly.

Objective 2 Curation Tools for Organizations

We hired Torsten Grote, who has worked with Briar Project among many other things, to lead up the development of the Curation Tools. We hammered out all of the technical possibilities and interviewed a number of people with key experience with the target use cases to figure out which is going to be the most useful approach. Since this project is addressing new uses cases for the F-Droid tools, the aim is to figure out which of the more popular use cases that we can address the easiest. This provides us the quickest path to figuring out whether this is a fruitful direction to pursue more after this initial project is complete. With that in mind, we nailed down these key points to guide us:

  • web v. mobile app
  • multi-user support v. ease of maintenance
  • Mobile is better aligned with our technical infrastructure but might not be nearly as useful to the target audience as a multi-user web app that’s easy to deploy

If any of you have ideas about this topic, and what to offer your feedback to help figure out the best direction, please do get in contact with us!

Ultimately, whether the curation tool is a web or mobile app, both will be deploying to web infrastructure like Amazon S3, GitHub, or even a standard web server. So for that, the work going into the f-droid.org website overhaul will provide building blocks for what the curation tools publish. For example, there is now an F-Droid plugin for Jekyll, which makes it easy to include all the data from an F-Droid app/media repository into a custom website. All of these bits got us thinking: in a sense, we are building a toolkit for anyone to build their own Paskoocheh, ASL19’s custom curated “app store” that has taken off recently in Iran.

Objective 3 Modern App Store with Built-in Circumvention

UX Overhaul

There new f-droid.org website is now usable in its prototype form, including listing all apps and a big overhaul of all the documentation. The old manual and wiki were merged into a new “Docs” section, and many pages there were edited and updated. We now have a single overview of the documentation needed for all the various parts of F-Droid.

We will be using this prototype version of the website https://eighthave.gitlab.io/fdroid-website/ for the upcoming developer survey and developer tools user test. The feedback from both of those will then guide us in finishing the overhaul of the website.

The new website is now based on a custom Jekyll plugin for working with F-Droid app/media indexes: https://gitlab.com/fdroid/jekyll-fdroid/ This plugin allows any Jekyll website to easily use F-Droid app index data, including available apps and media files, all available versions, all descriptive text and graphics, etc.

User Testing

We have been working through all of the feedback from the user tests, and updating the UX designs based on that.

Peter Serwylo was on a well deserved vacation all of January, after finished his Ph.D. Once he returns, he will be increasing his work time on this project to 3 days a week until the end of Spring. Since he’s the main client dev, implementation progress there was slow in January.

Objective 4 Partner Deployments

In China, where there is no single de facto Android app store, it is quite common to directly download apps to install them. The problem there is then there is no automatic update channel. A number of apps that care more about security include automatic updating directly in the app. But this is in conflict with the Google Play Terms of Service. From the feedback that we received from Tibetan partner, we are putting together two libraries to help with this problem. First, the F-Droid tools provide the essential architecture, then we just need to rebundle this to work as a standalone updater. This design is also based on feedback from people at Google to make sure that the library’s updating process complies with Google Play’s Terms of Service so that projects can embed it in their apps without worrying about whether their apps will be kicked out of Google Play for including self-update capabilities. A parallel library directs users towards installing the F-Droid client app to provide the update channel rather than self-updating. Using the F-Droid client app provides central update management as well as a more fine tuned update procedure that includes all of the working circumvention techniques (nearby swap, “collateral freedom” mirrors, Tor support, etc.).

Follow the implementation progress here:

https://gitlab.com/fdroid/fdroidclient/issues/852
https://gitlab.com/fdroid/fdroidclient/issues/714

Objective 5 Usability Research on In-country Developers

We began coding and analysis of interviews for the final report, continued work on the design of user tests of the F-Droid developer tools, and completed the design of the developer survey.

Research Report / (Interview Coding)

We began transcribing and coding the developer interviews conducted during this activity. Transcription is nearly complete, and coding has been completed for one third of the interviews. The interviews are being coded to identify similarities and differences between international developer:

• Goals: Why they develop software;
• Needs: What they need to meet those goals;
• Challenges: The things that get in their way of meeting those needs;
• Strategies: The tools and techniques they engage in to overcome those challenges; and
• Networks: The people they interact with who support, or thwart, the above.

Analysis of the interviews will be completed in the early half of February. Writing will begin upon the completion of analysis. Once survey data has been collected (middle of march) that data will be Incorporated into the final research output.

User Testing

We completed scoping the activities for UX testing during the last month. UX testing will focus on the F-Droid developer documentation, setup of an F-Droid binary application repository, and updating an application within an existing F-Droid repository. Fortuitously, there have been recent contributions to the F-Droid website that have provided an opportunity for a restructuring of the documentation. UX testing will be able to test this new documentation before it goes live. The UX testing documentation and technical setup will be completed in the early half of February and testing will be completed by the end of the month.

Surveys

While survey design was completed in December, unforeseen complications led to delays in translation. Translation is expected to begin in the first week of February. We have also begun collecting quotes from professional translation services in case the current provider is unable to begin the translation process.

Bazaar: Bazaar2 Monthly Report - December 2016

Added by hans about 1 year ago

There was some solid progress on the existing efforts, as well as some groundwork laid for the final big development sprint of this project funding. We nailed down the v0.102 stable release of the F-Droid client app, which includes a lot of core improvements. This stable release sets us up for a longer alpha cycle for the next round to support the major overhaul of the client app.

We also started the hiring process to find more contributors to take on more subprojects for the final sprint. This and other Guardian Project job descriptions here:
https://guardianproject.info/contact/join/

Objective 1 Simple multi-pronged distribution

The F-Droid package index metadata format was redesigned from scratch in order to support lots of essential new functionality: media and other non-app packages, screenshots, store graphics, and full localization of text and graphics. This is currently implemented, and is very alpha functional prototype.

One of the key issues of this whole project is how to build an app store ecosystem that is as difficult as possible to abuse, even for the people operating the app store or attackers who have gained full control of the app store’s binary repository. Reproducible builds allow anyone to reproduce the binaries served by f-droid.org, and binary transparency makes it possible to track the history of all binaries released. In support of this effort, we attended the Reproducible Builds Summit in Berlin, where we worked with most of the major GNU/Linux and BSD distros, the Google Bazel team, as well as a handful of other projects.

The first public instance of an F-Droid Verification Server, https://verification.f-droid.org/, is now up and running. This is wholly separate build infrastructure that automatically rebuilds all apps published to https://f-droid.org and then checks whether they match the official release. If they do not match, then it publishes the differences using https://diffoscope.org.

Good software update systems should release reproducible binaries, then have an unchangeable record of all releases made. This makes it possible to verify that an app that a device is using is the actual file that was by the update system, and is not an impersonator. At the Reproducible Builds Summit, we also we worked with a couple people who are focused on designing binary transparency systems to put together a prototype of a “Binary Transparency Log” for F-Droid. This is implemented as part of the fdroidserver app store kit, and it will eventually be deployed to f-droid.org, once it is proven stable.

Objective 2 Curation Tools for Organizations

No notable progress on this.

Objective 3 Modern App Store with Built-in Circumvention

The overhaul of the f-droid.org website has begun, led by NicoAlt, a long time volunteer contributor, and fxedel, a new contributor. The core of this work is converting almost the whole site to use Jekyll, a static website generator used by GitHub Pages and many other projects. This also generalized the website so that it can be easily reused for other people setting up their own app stores. This work will make it much easier to update the website’s user experience to match the new client app user experience.

UX Overhaul

There was a major push to get the entire base level of the new UX design implemented at a basically usable level. There is now a very raw but functional alpha of almost the whole new user experience.

User Testing

We reviewed the user testing results from the field tests, and put together a snapshot document with the primary takeaways from all tests conducted in 2016.
https://docs.google.com/document/d/1KlOdcErzrvA_XmxSekkUDYQ5uaFjYQhNJAnMSYM0fUw

Objective 4 Partner Deployments

As part of the new app/package index metadata format, the code for parsing the index metadata on the client side was modularized. This is the groundwork work for a library to allow apps to directly update themselves from the same index file that is used within the F-Droid client app itself. This expands the F-Droid toolset so that it can be used for both of the two major update approaches: an “package manager” which updates all installed apps; and apps that now how to update themselves. This approach also means that apps can seamless use both approaches without having different server-side setups.

Objective 5 Usability Research on In-country Developers

We started designing a user test based on some of the F-Droid server-side tools in order to test the whole process of figuring out issues that arise in developers’ workflows while finding, learning, and using tools for the app development process. This user test is slated to begin in late January.

1 2 3 ... 19 (1-10/188)

Also available in: Atom