News

Bazaar2 Monthly Report - January 2017

Added by hans 17 days ago

This past month was dominated by organizing the upcoming large development sprint starting in February. This means hiring a number of people to do all the work. We had 20+ applications, lots of email, and 5 interviews. We hired two experienced developers, and 4 part time junior developers.

There were also a few notable achievements in the development work:
  • Completed an automated system for mass-verifying reproducible builds
  • Finalized possible technical approaches for curation tools
  • F-Droid website converted into a app store website toolkit
  • Designed multi-language survey about developer challenges
  • Designed user test of the developer tools and documentation

The first results from the user research into developers have been published:
https://guardianproject.info/2017/01/26/imagining-the-challenges-of-developers-in-repressive-environments/

Objective 1 Simple multi-pronged distribution

We now have https://verification.f-droid.org/ automatically building the latest apps and testing whether they are reproducible. We are up to 59 apps that can be built reproducibly using the F-Droid tools. To see which apps, search for “verified” on https://verification.f-droid.org/. Now that we have a mass rebuild process running automatically, the next step is to focus on some more important apps in order to fix the issues preventing them from being rebuilt reproducibly.

Objective 2 Curation Tools for Organizations

We hired Torsten Grote, who has worked with Briar Project among many other things, to lead up the development of the Curation Tools. We hammered out all of the technical possibilities and interviewed a number of people with key experience with the target use cases to figure out which is going to be the most useful approach. Since this project is addressing new uses cases for the F-Droid tools, the aim is to figure out which of the more popular use cases that we can address the easiest. This provides us the quickest path to figuring out whether this is a fruitful direction to pursue more after this initial project is complete. With that in mind, we nailed down these key points to guide us:

  • web v. mobile app
  • multi-user support v. ease of maintenance
  • Mobile is better aligned with our technical infrastructure but might not be nearly as useful to the target audience as a multi-user web app that’s easy to deploy

If any of you have ideas about this topic, and what to offer your feedback to help figure out the best direction, please do get in contact with us!

Ultimately, whether the curation tool is a web or mobile app, both will be deploying to web infrastructure like Amazon S3, GitHub, or even a standard web server. So for that, the work going into the f-droid.org website overhaul will provide building blocks for what the curation tools publish. For example, there is now an F-Droid plugin for Jekyll, which makes it easy to include all the data from an F-Droid app/media repository into a custom website. All of these bits got us thinking: in a sense, we are building a toolkit for anyone to build their own Paskoocheh, ASL19’s custom curated “app store” that has taken off recently in Iran.

Objective 3 Modern App Store with Built-in Circumvention

UX Overhaul

There new f-droid.org website is now usable in its prototype form, including listing all apps and a big overhaul of all the documentation. The old manual and wiki were merged into a new “Docs” section, and many pages there were edited and updated. We now have a single overview of the documentation needed for all the various parts of F-Droid.

We will be using this prototype version of the website https://eighthave.gitlab.io/fdroid-website/ for the upcoming developer survey and developer tools user test. The feedback from both of those will then guide us in finishing the overhaul of the website.

The new website is now based on a custom Jekyll plugin for working with F-Droid app/media indexes: https://gitlab.com/fdroid/jekyll-fdroid/ This plugin allows any Jekyll website to easily use F-Droid app index data, including available apps and media files, all available versions, all descriptive text and graphics, etc.

User Testing

We have been working through all of the feedback from the user tests, and updating the UX designs based on that.

Peter Serwylo was on a well deserved vacation all of January, after finished his Ph.D. Once he returns, he will be increasing his work time on this project to 3 days a week until the end of Spring. Since he’s the main client dev, implementation progress there was slow in January.

Objective 4 Partner Deployments

In China, where there is no single de facto Android app store, it is quite common to directly download apps to install them. The problem there is then there is no automatic update channel. A number of apps that care more about security include automatic updating directly in the app. But this is in conflict with the Google Play Terms of Service. From the feedback that we received from Tibetan partner, we are putting together two libraries to help with this problem. First, the F-Droid tools provide the essential architecture, then we just need to rebundle this to work as a standalone updater. This design is also based on feedback from people at Google to make sure that the library’s updating process complies with Google Play’s Terms of Service so that projects can embed it in their apps without worrying about whether their apps will be kicked out of Google Play for including self-update capabilities. A parallel library directs users towards installing the F-Droid client app to provide the update channel rather than self-updating. Using the F-Droid client app provides central update management as well as a more fine tuned update procedure that includes all of the working circumvention techniques (nearby swap, “collateral freedom” mirrors, Tor support, etc.).

Follow the implementation progress here:

https://gitlab.com/fdroid/fdroidclient/issues/852
https://gitlab.com/fdroid/fdroidclient/issues/714

Objective 5 Usability Research on In-country Developers

We began coding and analysis of interviews for the final report, continued work on the design of user tests of the F-Droid developer tools, and completed the design of the developer survey.

Research Report / (Interview Coding)

We began transcribing and coding the developer interviews conducted during this activity. Transcription is nearly complete, and coding has been completed for one third of the interviews. The interviews are being coded to identify similarities and differences between international developer:

• Goals: Why they develop software;
• Needs: What they need to meet those goals;
• Challenges: The things that get in their way of meeting those needs;
• Strategies: The tools and techniques they engage in to overcome those challenges; and
• Networks: The people they interact with who support, or thwart, the above.

Analysis of the interviews will be completed in the early half of February. Writing will begin upon the completion of analysis. Once survey data has been collected (middle of march) that data will be Incorporated into the final research output.

User Testing

We completed scoping the activities for UX testing during the last month. UX testing will focus on the F-Droid developer documentation, setup of an F-Droid binary application repository, and updating an application within an existing F-Droid repository. Fortuitously, there have been recent contributions to the F-Droid website that have provided an opportunity for a restructuring of the documentation. UX testing will be able to test this new documentation before it goes live. The UX testing documentation and technical setup will be completed in the early half of February and testing will be completed by the end of the month.

Surveys

While survey design was completed in December, unforeseen complications led to delays in translation. Translation is expected to begin in the first week of February. We have also begun collecting quotes from professional translation services in case the current provider is unable to begin the translation process.

Bazaar2 Monthly Report - December 2016

Added by hans about 1 month ago

There was some solid progress on the existing efforts, as well as some groundwork laid for the final big development sprint of this project funding. We nailed down the v0.102 stable release of the F-Droid client app, which includes a lot of core improvements. This stable release sets us up for a longer alpha cycle for the next round to support the major overhaul of the client app.

We also started the hiring process to find more contributors to take on more subprojects for the final sprint. This and other Guardian Project job descriptions here:
https://guardianproject.info/contact/join/

Objective 1 Simple multi-pronged distribution

The F-Droid package index metadata format was redesigned from scratch in order to support lots of essential new functionality: media and other non-app packages, screenshots, store graphics, and full localization of text and graphics. This is currently implemented, and is very alpha functional prototype.

One of the key issues of this whole project is how to build an app store ecosystem that is as difficult as possible to abuse, even for the people operating the app store or attackers who have gained full control of the app store’s binary repository. Reproducible builds allow anyone to reproduce the binaries served by f-droid.org, and binary transparency makes it possible to track the history of all binaries released. In support of this effort, we attended the Reproducible Builds Summit in Berlin, where we worked with most of the major GNU/Linux and BSD distros, the Google Bazel team, as well as a handful of other projects.

The first public instance of an F-Droid Verification Server, https://verification.f-droid.org/, is now up and running. This is wholly separate build infrastructure that automatically rebuilds all apps published to https://f-droid.org and then checks whether they match the official release. If they do not match, then it publishes the differences using https://diffoscope.org.

Good software update systems should release reproducible binaries, then have an unchangeable record of all releases made. This makes it possible to verify that an app that a device is using is the actual file that was by the update system, and is not an impersonator. At the Reproducible Builds Summit, we also we worked with a couple people who are focused on designing binary transparency systems to put together a prototype of a “Binary Transparency Log” for F-Droid. This is implemented as part of the fdroidserver app store kit, and it will eventually be deployed to f-droid.org, once it is proven stable.

Objective 2 Curation Tools for Organizations

No notable progress on this.

Objective 3 Modern App Store with Built-in Circumvention

The overhaul of the f-droid.org website has begun, led by NicoAlt, a long time volunteer contributor, and fxedel, a new contributor. The core of this work is converting almost the whole site to use Jekyll, a static website generator used by GitHub Pages and many other projects. This also generalized the website so that it can be easily reused for other people setting up their own app stores. This work will make it much easier to update the website’s user experience to match the new client app user experience.

UX Overhaul

There was a major push to get the entire base level of the new UX design implemented at a basically usable level. There is now a very raw but functional alpha of almost the whole new user experience.

User Testing

We reviewed the user testing results from the field tests, and put together a snapshot document with the primary takeaways from all tests conducted in 2016.
https://docs.google.com/document/d/1KlOdcErzrvA_XmxSekkUDYQ5uaFjYQhNJAnMSYM0fUw

Objective 4 Partner Deployments

As part of the new app/package index metadata format, the code for parsing the index metadata on the client side was modularized. This is the groundwork work for a library to allow apps to directly update themselves from the same index file that is used within the F-Droid client app itself. This expands the F-Droid toolset so that it can be used for both of the two major update approaches: an “package manager” which updates all installed apps; and apps that now how to update themselves. This approach also means that apps can seamless use both approaches without having different server-side setups.

Objective 5 Usability Research on In-country Developers

We started designing a user test based on some of the F-Droid server-side tools in order to test the whole process of figuring out issues that arise in developers’ workflows while finding, learning, and using tools for the app development process. This user test is slated to begin in late January.

Bazaar2 Monthly Report - November 2016

Added by hans 2 months ago

In November, we started in earnest implementing the big overhaul of the user experience of the F-Droid client app. That also lead to the beginning of overhauling the server side to provide an updated app index format that supports localization, screenshots and other graphics, as well as synchronizing all the data formats from where apps are initially submitted to f-droid.org (aka fdroiddata) to where they are parsed and included into the index (aka fdroidserver), to finally, the index that the Android app receives and displays to the user (aka fdroidclient).
Objective 1 Simple multi-pronged distribution

Finished development work to support building and distributing “Over-The-Air” (OTA) update ZIP files as part of the whole F-Droid system. This is useful for distributing not only the F-Droid Privileged Extension, which lets F-Droid operate like Google Play, but also other apps that need to run with system privileges, like the MicroG Project’s Free Software replacements for the proprietary Google components of Android. This new build process is already live and working, we are just waiting on the final integration of the publishing procedure:

https://gitlab.com/fdroid/fdroidserver/merge_requests/193

Media files can be included into F-Droid repos now, the client does not yet install them. The client will fully support downloading media as part of the full UX Overhaul.

Objective 2 Curation Tools for Organizations
No notable progress on this.

Objective 3 Modern App Store with Built-in Circumvention
UX Overhaul

We posted the first alpha version of the new UX as a preview of the overall architecture. The design files were finalized and handed over to the developers. One last piece was added to the designs: a flow for installing an alpha version or older version of an app from the app details view.

The totally new index format to support localization and graphics was fully prototyped and is functional. It will be integrated as soon as the final kinks are worked out.
User Testing
We reviewed the results from the tests executed in Vienna
Prepared the testing plan for Zimbabwe
Made improvements to the nearby design in the prototype
Reviewed screen records from the field tests deployed in Zimbabwe
Objective 4 Partner Deployments
We discussed specific distribution approaches with two potential partners for environments with very limited internet access.

Objective 5 Usability Research on In-country Developers
1.1 Interviews
Mr. Tuohy conducted in-depth remote interviews with eight software developers and technologists from seven different regions where the internet is heavily monitored and filtered. This will make up a majority of the interviews that will be conducted. In total we have interviewed 11 developers/technologists from closed and closing spaces and anticipate one or two additional Interviews before the end of the interview period. While analysis of the interviews will occur over the next month there are some initial findings.

Culture has a deep impact on how developers perceive and respond to the challenges that they face.
In areas where the cost, speed, availability, or censorship of the Internet is a challenge local developers have strategies and technical systems in place for sharing software libraries and documentation among themselves.
Pseudonyms and operational security are the primary strategy used by developers who fear that they will be targeted for the software that they develop.
The lack of localized/translated guidance on software development and developer documentation for security/circumvention libraries are some of the greatest barriers to the development of security and circumvention software in repressive environments.
Local developer access to, and interactions with, members of the international security and circumvention technology communities was commonly referenced as highly valuable by many of the developers spoken to.

1.2 Surveys
Mr. Tuohy is in the process of building a developer survey based upon the initial findings from the interviews. This survey will be short (consisting of at most 25 questions) to increase the likelihood of developers spending their time to fill it out. This survey aims to reach a larger audience of local developers to test if the findings from the survey are broadly applicable. With support from localization lab this survey will be localized to ask about the impacts of censorship and surveillance on developers in a way that is culturally appropriate for a ”non-radical” developer audience and in their local language.

1.3 User Testing
One of the key findings from the interviews was how important it was for software documentation to be easy to navigate and read. Developers around the world often have learned to read technical English as a second language. This language barrier means that developers often can only read english, and do not actively engage in English language development communities. As such, documentation is often the only avenue for these developers to understand if the software meets their needs, and is worth investing time into. Sadly, documentation is often sub-par in the open-source security and circumvention software space.

In response to this we are developing one of the two components of the upcoming user testing to test the ease of navigation and understanding of the F-Droid documentation. The other component of the user testing will explore the process of setting up and using an F-Droid app repository to publish and update existing applications. This testing will be done with technologists who speak English as a second language.

Highlights
Conducted in-depth remote interviews with 8 software developers and technologists from seven different regions where the internet is heavily monitored and filtered.
The Localization Lab is working with the project to localize survey questions to be appropriate for a broad developer audience in the targeted regions.

Bazaar2 Monthly Report - October 2016

Added by hans 4 months ago

This past month, we ran a bunch of user tests to confirm that existing
parts were working, and to get feedback about the new UX overhaul of the
client app. Overall, we received solid feedback that things are
working, while the studies did point out areas where we have work to do.
At the OTF Summit, Seamus Tuohy kicked off the developer user research
portion of the project. We also had a number of good discussions on
various issues and challenges related to this project.

One realization that came out of the OTF Summit is that the differences
in the various context around the world mean that F-Droid needs to be
portrayed quite differently in each context. For example, in Zimbabwe,
the private local app/media swapping is the most valuable feature since
many parts of the country the internet is unreliable or expensive, but
otherwise people use Google Play and not much else. In China, the
internet is affordable and widely available and most people already use
multiple app stores, but it is often heavily filtered, with specific
sites and services totally blocked. So in Cuba, the local app swapping
is far and away the dominant feature while in China, the circumvention
is the key feature. When all of this is included in a single app, then
communicating what exactly this app is must be strongly tied to the
local context in order for people to effectively understand how it can
be useful to them.

Objective 1 Simple multi-pronged distribution

Media Support

The core “fdroidserver” tools now support adding any arbitrary file to a
repository. This was first done to support videos, e-books, audio,
etc., but it became rapidly clear that there wasn’t a need to limit what
kinds of files are supported. This opens things up for experimentation.
For example, perhaps it would be useful to also distribute desktop apps
via F-Droid.

One clear use case that has developed since this was implemented is for
distributing “Over-The-Air” (OTA) update files. This is the standard
format used to update the core Android OS. Then system updates and
additions can be safely distributed via F-Droid. Currently, there are
lots of lots of people who are downloading additions like “gapps”
(Google Apps) to add on to custom Android OS distributions like
CyanogenMod. These are usually just downloaded from random, insecure
places on the internet. With F-Droid’s new file support, these can now
easily be safely distributed via the F-Droid ecosystem. Follow the
progress of this via F-Droid’s own OTA update, the “F-Droid Privileged
Extension”:
https://gitlab.com/fdroid/privileged-extension/issues/9

Another potential use for OTA files in F-Droid is for securely
distributing optional system-level software packages comes from Mike
Perry’s “Mission Improbable” project for customizing the Copperhead
Android ROM distribution. Additions that Copperhead do not support like
https://microg.org/’s free replacements for Google Apps, or even Google
Apps itself, can be included in an F-Droid repository for easy
installation when the user wants. The Android method for managing these
files is based entirely around software updates, so it is not meant for
browsing and selectively applying OTA files.

Reproducible Builds

Finally, the completion of the fully reproducible build process is
within reach. This has been stymied by the difficulties of running a VM
in a VM. We are now quite close to getting fully automated, ground up
build server process that then in turn runs reproducible builds of
Android apps. We set up a new server to serve as the “verification
server” test platform on eclips.is. That will serve as a place to
polish up the verification server so that it is easy for anyone to
deploy to verify any app they are interested in. Follow that work here:
https://f-droid.org/wiki/page/Verification_Server

Objective 2 Curation Tools for Organizations

No notable progress on this.

Objective 3 Modern App Store with Built-in Circumvention

UX Overhaul

We ran a couple user tests using a mockup of the new F-Droid client app
UX designs. The tests were run in two southern African countries and
Vienna, Austria. Overall, the new designs were quite well accepted.
Testers navigated the app easily. There were no major issues with
completing the tasks that were given to the testers, including with
nearby app swapping. This points us to the need for getting the nearby
features very solidly implemented so the reality can match these user tests.

In the real world test of nearby app swapping in southern Africa, over
90% were able to successfully swap apps, with WiFi having a much higher
success rate over Bluetooth. The downside is that conceptually WiFi was
more difficult than Bluetooth, since all of the participants thought of
the word WiFi as interchangeable with the word Internet. Bluetooth was
generally understood as only local.

Additionally, we are working on a partnership with Svenja Schroeder of
University of Vienna’s Usable Security lab to run user studies that
highlight the usability issues of software that aims to protect privacy.
https://cs.univie.ac.at/cosy/home/

Here is the full report and raw materials from the Vienna test:

Final Report:
https://docs.google.com/document/d/1ZyrdUzkVdEjubhEsadLeSsAwqUF0ChWYOpr0QlIryrk

Task Success Rates/Survey Results:
https://docs.google.com/spreadsheets/d/1aDE7uCzO8FURGhjNn4gsjeeb7EmXJNc2WRjVb5V_4Mc

F-Droid Overhaul User Test Script:
https://docs.google.com/document/d/13CpKXBmvpuKnBfcajMFeef_840Z9Rnqkey3kd0E_vnA

User Test Printout materials:
https://docs.google.com/document/d/1NbxjWYXuYw7Wn9Dn-sZmNVdAX-DiTYWFGFtp7GwJREg

Implementation Begins

The implementation of the new UX overhaul designs has begun. The plan
is to get the basic user experience working as per the designs, before
moving onto more minute details such as exact
colours/fonts/paddings/etc. The basic UX is now in place for the main
featured apps screen, the categories overview screen, the list of apps
for a single category, which doubles as a general purpose search
interface, and the settings view (which I ported directly from the
current settings view in the old UI).

There are still many things missing which need to be added, most
prominently: * The "My Apps" screen where users can see updates to their installed apps * The "Nearby" screen, which will be a port of the current "Swap" interface * Integrating feedback from the app download process into the app list
screen (e.g. "This app is downloading", "This app can be updated").
Right now it either has an install button or it doesn't.

Some of these will wait until further feedback from usability studies
that we are working on. Some videos of the current implementation are
available here:

https://gitlab.com/fdroid/fdroidclient/issues/709

New Approaches for Security Scans

We discussed new security scanning approaches with academic security
researchers as part of the ACM CCS conference. In the academic world,
there is a chunk of work going on for doing automatic scans of software
for finding libraries and even specific versions. We plan to use this
information in combination with standardized vulnerability reports like
CVEs to notify users that the specific apps that they have installed or
are seeking to install have known security issues.

We planned out the implementation using some upcoming free software
libraries like LibScout and Alterdroid:
https://www.infsec.cs.uni-saarland.de/~derr/
http://www.seg.inf.uc3m.es/~guillermo-suarez-tangil/Alterdroid/

Objective 4 Partner Deployments

We discussed specific distribution approaches with two potential
partners for environments with very limited internet access.

Objective 5 Usability Research on In-country Developers

We kicked off at the OTF Summit with a series of interviews and a survey
to help establish the scope of the research. Over the next two months,
Seamus Tuohy will be conducting interviews with internet freedom
developers from a variety of closed and closing spaces on their
development processes and the challenges they face. This study will
produce guidance, user stories, and/or other information that can be
shared with organizations working on internet freedom issues. It aims to
help them better support developers in closed and closing spaces.

Here are the results of the survey:
https://drive.google.com/file/d/0B7TJ3OZ3bai_YmpqSjI4cDdKTFk

We are currently looking to interview individuals with insights into the
challenges of technologists and software developers in places where the
internet is heavily monitored and filtered and/or where developers could
be at-risk because of their work. If you, or someone you know, fits this
description and are willing to participate in a face-to-face, phone or
video conference interview please feel free to reach out to me.

Bazaar2 Monthly Report - September 2016

Added by hans 5 months ago

In September, we completed the redesign of the user experience of the Android client app as well as most of the underlying architectural changes needed. We also worked on some new features in the client as well as more underlying architectural changes on the server side. We started intensive user testing of the new client app design, with more user testing slated for October.

Also, I presented our new, work in the NetCipher library on making Orbot integration easier at Droidcon Vienna, an Android developer conference: https://droidcon.at/speakers/

Objective 1 Simple multi-pronged distribution
---------------------------------------------

We have been discussing with developers at Twitter about integrating the F-Droid tools into Twitter’s fastlane, an open source automation suite for mobile developers. Fastlane managing many aspects of deployment including translations, screenshots, release builds, etc. It does not currently provide good signing key management, hardened build processes, or reproducible builds. Since the F-Droid tools do provide those, integrating F-Droid with fastlane makes a lot of sense.

The drozer automatic, dynamic exploit scanning is and running on f-droid.org infrastructure. The final dedicated hardware is in place as part of f-droid.org, and the production setup is almost complete.

We made more progress on generalizing the buildserver, which is a automated sandbox for running the app builds. The buildserver now runs on VirtualBox and KVM, with Docker support sketched out. This provides a key piece of both the reproducible builds, as well as a relatively easy way to run secure release builds. Once this work is complete, we will then be able to run verification builds of all apps on f-droid.org on https://jenkins.debian.net in order to provide separate confirmation of the official releases on f-droid.org.

We still need advice on how best to structure and manage all of the various virtualization approaches, so we’d love to talk to anyone who is an expert on this stuff to give us advice.

Objective 2 Curation Tools for Organizations
--------------------------------------------

No notable progress on this.

Objective 3 Modern App Store with Built-in Circumvention
--------------------------------------------------------

UX Overhaul

We finalized the client design for the first round, and prepared prototypes for user testing. One thing we’ve learned is that there is some confusion about what F-Droid is among novice users. When preparing the prototype for testing, we considered a simple onboarding experience that will help overcome this issue. As part of that, we also considered the first use of Nearby, and segmented the main view into 2 different views to help people understand what the feature does.

These design updates can be viewed in the prototype. The feedback will help determine what we implement.

F-Droid Tutorials

We’ve created concepts for the tutorial experience and an initial prototype.
- Initial concept: http://pasteboard.co/9N9LkXKqE.png
- Prototype: https://invis.io/W88R80OVA

User Testing

We are doing user testing in two locations: Zimbabwe and Vienna. In Zimbabwe, the tests were a part of digital security trainings. In Vienna, we are aiming for a general audience for comparison.

Our partners in Zimbabwe did user testing at a trainer’s workshop with the design prototype. We hoped to test the comprehension of the new UI among this population. We are also preparing to do user testing with the same prototype in October in Vienna.

The preparation for these user tests has included:
- Determining a test method and plan https://docs.google.com/document/d/1YokzlLY6ABcw0NBDy0a0t1lmTAAI78GpRyLliqNMmT4/edit
- Creating a survey to gather contextual information from the participants https://okthanks.typeform.com/to/ecpfsv
- Creating and testing the prototype
https://invis.io/MZ8MJAYRX

Objective 4 Partner Deployments
-------------------------------

No notable progress on this.

Objective 5 Usability Research on In-country Developers
-------------------------------------------------------

We have hired Seamus Tuohy to work on the usability research on developers. He will be producing the final published report as well and leading up the research. That work will start at the OTF Summit, where we will be asking for discussions and interviews to help guide the direction of this research.

Bazaar2 Monthly Report - August 2016

Added by hans 5 months ago

The main focus of development efforts in August was on designing the new user experience for the Android client app. We have the design pretty much finalized, and the re-architecture of the software need to support the new user experience has been laid out to be fully implemented in September.

Based on our surveys of Android app stores around the world, as well as feedback from Digital Society of Zimbabwe, we have decided to emphasize some of the aspects of F-Droid that work well when the device is offline. The whole app collection can be browsed and searched without an internet connection. To improve the offline experience, we need to handle offline install requests gracefully.

Objective 1 Simple multi-pronged distribution

Adding dynamic malware scanning to the whole fdroidserver build process is functional and almost complete. It is working on the prototype setup, and we have new server infrastructure run by f-droid.org that will run the dynamic scanning as part of the regular build process for apps that are included in f-droid.org.

The fdroidserver tool suite is now available in Windows 10 Subsystem for Linux (aka Bash for Windows):
https://f-droid.org/wiki/page/Installing_the_Server/Repo_Tools#Windows_10_Subsystem_for_Linux

Objective 2 Curation Tools for Organizations

We started designing some user tests around trainers working with Digital Society of Zimbabwe. They are also helping to run user tests on other F-Droid tools.

Objective 3 Modern App Store with Built-in Circumvention

We finalized the new designs of the main screen and overall navigation through app listing, browsing by categories, searching, etc. We also finalized the design of the notifications related to installs, uninstalls, and background downloads. We decided on a core design pattern of a bottom navigation bar because it provides simple usability with one hand, is compatible with Google’s Material Design guidelines, and matches the navigation design pattern that is dominant in Chinese design.

We determined what recently updated and recently added apps would be displayed, and how to display them based on the artwork they provide (featured image, size of launcher icon), and tested concepts for a default background artwork to use for apps if no featured image is provided.

Layout designs are posted here:
https://gitlab.com/fdroid/fdroidclient/issues/709
Notifications Design
https://gitlab.com/fdroid/fdroidclient/issues/742

Field Testing

We worked with Digital Society of Zimbabwe to incorporate user/field testing as part of their regular trainings. We discussed the feedback gained from the first user test in a Zimbabwe trainings and brainstormed ideas for gathering and documenting learnings from the field more effectively.

Tutorials

We hired Hailey Still as a UX Intern to help with user testing and tutorial design. We kicked off work on click through tutorials for installing F-Droid and swapping apps with nearby devices with a discussion of the goals and challenges.

Re-architecting F-Droid client app

The internal database structure of F-Droid client was overhauled to fully support all of the possible states of apps, including multiple source repos, multiple builds, and multiple APK signing keys. The database structure will now allow repositories of varying "priorities" to provide metadata from the same apps. This work will also make it possible to transition apps away from the F-Droid signing key to developer’s own signing key. All together, this means that the F-Droid client app will be able to make better decisions about what to show the user, leading to more useful security alerts.

Objective 4 Partner Deployments

We designed a set of tools based on the F-Droid infrastructure that allows apps to have miniature, embedded “featured app” collection that also allow direct installation. One key example of this idea in action is the Tibetan keyboard for Android, which recommends other apps that also support Tibetan well. The library that we will build to support this will also work well for creating apps that can directly update themselves, with or without the F-Droid client app installed.

Objective 5 Usability Research on In-country Developers

We worked out how much of a physical presence we need in order to effectively gather information on developers who feel targeted. We started work on a plan for which countries would be most useful and most feasible to visit in order to conduct user research on developers.

Bazaar2 Monthly Report - July 2016

Added by hans 7 months ago

July was a busy month for new partnerships and people. The partnerships spread F-Droid to more users and use cases, while building a community that relies on F-Droid and is invested in its maintenance. The new people expand the work we are currently doing: now that we are nearly complete with the large architectural changes, we are starting the big overhaul of the user experience.

  • F-Droid was chosen as the app store for new partnership deal between Copperhead and SaltDNA, a startup to build a secure messaging platform.
    https://finance.yahoo.com/news/saltdna-copperhead-partner-end-end-140100909.html
  • We signed a contract with Blue Jay Wireless, a small telecom in the US, to develop two new core features.
  • Carrie Winfrey joins us again to lead up the user experience work on the F-Droid client app. She previously lead the UX for the app swapping work.
  • Brennan Novak to lead up the usability research and work around the user experience for Android developers.

Objective 1 Simple multi-pronged distribution

We now have the drozer setup automated and triggering based on the fdroidserver build process. Drozer actually runs the app in an emulator and probes it for vulnerabilities. Drozer can run pre and post build for F-Droid. Depending on how you'd like to proceed with reports (if an app fails the scan, should it be allowed to be built etc?) we can switch the workflow on the fly - that's the beauty of using Docker for this. Once we get it all integrated, we can start scanning all apps distributed by f-droid.org To start with, the Drozer reports will be shared privately, so we can manage when found exploits get divulged. Ultimately, we aim to have this information fully public.

Blue Jay Wireless has setup their own custom app store based on the F-Droid client app and developer tools. They have hired us to develop two chunks of functionality they need, which also help us with the Bazaar2 goals of developing tools for trainers and organizations to deploy apps, as well as to get app usage data in a privacy preserving way so that F-Droid can show how popular apps are without privacy concerns. The first is end user controllable “push” install/uninstall of apps, which can be used in trainings to easily setup people’s devices. The second is an opt-in “popularity contest” that provide counts of installs, uninstalls, and install failures without linking the data to the user. The provides user generated app ratings.

Objective 2 Curation Tools for Organizations

  • DigiSoc ran a training in rural Zimbabwe where they were user-testing F-Droid app swapping to get apps to trainees in places where the internet is constrained.
  • Now that Blue Jay Wireless is funding the development of push installs and user-generated popularity data, we have shifted the design goals of these tools around what those features can provide. For example: a trainer can setup a custom collection of apps and media, then enable the push installs. She copies the collection to a portable device, like a phone or a LibraryBox. The trainees connect and accept the push install opt-in. The trainer’s apps and media are automatically installed on the trainees’ phones. The trainer can track progress by seeing if the successful install count matches the number of trainees.

Objective 3 Modern App Store with Built-in Circumvention

Data Model Overhaul

This month the focus of development was on overhauling how all of the app store data is represented in the client app’s database. In addition to adding support to media, the new data model lets F-Droid represent lots of various edge cases in a much clearer and usable way. For example, it will now handle when an app has updates available that are signed by different keys. These changes to the database are nearing completion, many of them have been merged into production, and the last few should be merged in over the coming month.

UX Overhaul

We also have been focused on the UX overhaul of the main app store experience. The UI related meetings have taken place with Carrie, Hans, Mark, and sometimes others. In addition, other regular F-Droid contributors have provided valuable feedback on the issue tracker in response to these meetings. As such, the UI design from Carrie is now approaching something which is ready to implement. It is looking like we will be able to start working on implementing this UI in August. You can join in the conversations here: https://gitlab.com/fdroid/fdroidclient/issues?milestone_title=UX+Overhaul

Streamlined Install Process

The new install process has been incorporated in v0.101 alpha builds, and we have been receiving feedback and bug reports from testers. This install process covers both scenarios how F-Droid is installed: as a third-party app store installed like an app, or like a built-in app store that is included in a device or Android ROM by default (for example, you can buy a device from Copperhead now with F-Droid built-in https://copperhead.co/android/buy). In addition to fixing bugs, we added automated tests of the install process.

Objective 4 Partner Deployments

We had more conversations with Storymaker about their needs.

Objective 5 Usability Research on In-country Developers

We have hired Brennan Novak to lead up this research and to work on developer user experience in general for this project. Brennan has worked on Mailpile, Qubes, Transparency Toolkit and more as both a UX Designer and a developer, so we think he’s uniquely qualified to do this research.

Bazaar2 Monthly Report - June 2016

Added by hans 8 months ago

In June, there was two main pushes in the work:

1. Re-architecting the client app while starting the UX redesign discussions
2. Reworking the f-droid.org build tools to be flexible tools for general use

Objective 1 Simple multi-pronged distribution

Improved Malware Resistance

Copperhead started working on integrating dynamic malware scanning tools into the F-Droid build infrastructure, based on tools like drozer.

Updated Build Server

The build server setup tools got some much needed attention. They have served well for running the f-droid.org infrastructure over the years, now we’re working to make them into a general purpose tool so that people and organizations can easily run their own F-Droid infrastructure, including a “verification server” to make sure that the apps that they use match exactly what’s generated from the source code. Our biggest blocker here is getting things to run in a virtualized environment, since the build server itself is a virtual machine. VirtualBox on top of KVM seems almost impossible, so we are trying KVM on top of KVM.

Officially Debian

The Debian Android Tools team now has basic working builds using Android SDK components built entirely from source and included as official Debian packages

Objective 2 Curation Tools for Organizations

We have been exploring some use cases and app ideas for how best to handle the curation. For example, would a simple Android app be more useful than full featured desktop app? There are lots of ideas for making easy to use curation tools, but it is quite difficult then to provide the same level of security with, say, a dead simple web app for curation. We’re looking for feedback on how much the curation tools should prioritize security vs. ease of use. On one extreme, if the curation tools are really easy, and end up just being used as a malware-ridden piracy enabler, then we’ve clearly failed. On the other extreme, if the perfectly secure tool is so difficult to understand that it requires a training to understand, then its usefulness is severely curtailed. Based on current feedback, it seems that we should aim more for the secure side of things, but it is still an open question.

Objective 3 Modern App Store with Built-in Circumvention

We have started weekly brainstorming sessions for discussing how we can redesign the F-Droid client app’s user experience in a both efficient yet effective way. As compared to “corporate” budgets for app UX overhauls, we are operating on a much smaller level. We make up for that with creativity and community contributions. We have currently surveyed a number of app stores, including two of the larger Chinese stores, for ideas about what users are used to and how to represent things in an intuitive and familiar way:
https://gitlab.com/fdroid/fdroidclient/issues/705

It turns out that F-Droid current UX is closer to lots of Chinese app stores than it is to Google Play, especially with the use of tabs and app lists. We’re looking for all sorts of feedback and ideas about what an app store experience should look like. Join the conversations! The discussion items here:
https://gitlab.com/fdroid/fdroidclient/issues?milestone_title=UX+Overhaul

The new plumbing for this work is performing well: the latest stable release of the Android client, v0.100.1, includes lots of our recent big changes, is fully deployed and in wide use. We also have 0.101-alpha1 out with lots more changes, and we are already receiving a lot of feedback.

Objective 4 Partner Deployments

We have been discussing with Storymaker how to best structure their “content packs” to make it easy to distribute them in F-Droid, Google Play, CafeBazaar, and other app stores. We have implemented support for “APK Extension” OBB files, which are Google’s standard way of distributing large media collections aka content packs for apps, games, etc.

Objective 5 Usability Research on In-country Developers

We have almost hired someone to lead these research efforts. We aim to start a sprint on UX Research in August.

Bazaar2 Monthly Report - May 2016

Added by hans 8 months ago

For May, we focused on a major rearchitecting of the F-Droid Android app to allow for flexible, modern user interaction. This work also makes it easier for volunteer contributors to take on smaller chunks of work since the source code and app structure are a lot cleaner and more consistent. We also continued research on usability in places with low internet access.
Lastly, we kicked off another round of work for getting the Android SDK into Debian with three students working this summer funded by Google’s Summer of Code.

For an overview about how we are currently thinking about this work, see the latest blog post:
https://guardianproject.info/2016/06/02/building-the-most-private-app-store/

Objective 1 Simple multi-pronged distribution

We are always happy to see others build upon our work and so we are excited to see the Android Tamer project using our Debian packages of the Android SDK and related tools. Android Tamer is a pre-built system for malware analysis, penetration testing, and reverse engineering of Android apps, shipped in the form of a “Live CD” and virtual machine image. We are also kicking off a continuation of that effort as part of Debian’s participation in Google Summer of Code. Three students from India, Taiwan, and France are joining us as part of the Debian Android Tools team to get the whole Android SDK included in Debian via a reproducible build process.

Objective 2 Curation Tools for Organizations

We researched ownCloud, a free software cloud services platform, as a potential platform for curation tools. It provides lots of useful, web-based tools like file sync, music and media handling, authentication, and more. The big downside is that it has a history of relatively poor security practices, and it could be difficult to create a reasonably secure pipeline for working with Android apps within the way ownCloud works. However, it is still promising as a place for curating media collections and publishing them to F-Droid.

Objective 3 Modern App Store with Built-in Circumvention

This month we focused on a major re-architecting of the core of the Android app to provide a solid platform to build an extended and improved user experience, covering:

  • Media handling
  • Parallel, background operation
  • A less linear and more intuitive swap user experience
  • Seamless integration into Android ROMs like Copperhead, Replicant, etc.
  • Multi-tasking

The big ticket item already included in the 0.100 release of F-Droid is the ability to download multiple apps at once. Although we started and released this feature in alpha releases in April, many of the stability and architectural changes were implemented in May. Another big round of core changes is going into 0.101, preparing us for the big user experience overhaul (UX overhaul) starting in mid-June. While these core changes are improving the existing user experience, they will leave some things in an odd state until the UX overhaul is complete.

  • Notifications being able to be cancelled correctly, whether the app is in a queue waiting to be downloaded or actively being downloaded.
  • Proper management of “Tap To Install”. Previously (back in April) it would put the item back in the queue to wait for all other downloads to complete before directing the user to the screen where they can install. Now it takes you there straight away despite other downloads occurring.
  • Correctly showing progress for all types of downloads, whether for apps or repos, notifications or the “App Details” screen.

We also nailed down the architecture for a fully privileged F-Droid that has the same abilities as Google Play in terms of being able to securely and transparently install and update apps (i.e. operate without “Unknown Sources”). This functionality can be included by ROMs, flashed onto phones, or installed via root access. We worked with Copperhead and Fairphone to design the F-Droid integration with ROMs to be both secure and flexible.

We discovered a bug in the app installation process of the upcoming Android release, codenamed “N”, as well as a different issue related to the handling of file/content URIs. We reported the bugs to Google and both should be fixed in the upcoming Android “N preview 4”.

In order to improve the Tor integration in F-Droid, we are working on the NetCipher library to make it very easy for apps to integrate with Orbot, including automatically starting Tor when needed, and providing apps feedback on the status of Tor. NetCipher is then used in F-Droid, and is freely available to any Android project that wants to include simple Tor support. To that end, we are expanding the number of networking libraries that NetCipher integrates with, it now works with:

  • the built-in URLConnection API
  • Square OkHTTP
  • Google Volley
  • Apache HttpClient for Android
  • ch.boye HttpClient

Objective 4 Partner Deployments

No notable activity here, we are waiting to finish some of the core improvements before proceeding further with partners.

Objective 5 Usability Research on In-country Developers

We discussed strategies for reaching in-country developers with a potential UX research lead.

We are starting to work with researchers in Afghanistan on user research in areas where internet access is very limited, both for people looking to get apps and media, as well as people distributing them.

Bazaar2 Monthly Report - April 2016

Added by hans 8 months ago

Objective 1 Simple multi-pronged distribution

In April we introduced a security check to prevent the root installation of packages that reside in third-party repositories and have the same package name as the F-Droid Privileged Extension. In addition, we started working on an unattended installer based on the "device owner" feature in Android 6. Currently, this is just a proof of concept.

In addition to the package name, we now compare the "package signature" (it's more a certificate) with the package signature of the F-Droid app itself, see https://gitlab.com/fdroid/fdroidclient/merge_requests/256
Proof of Concept for Android 6 Device Owner unattended install, see https://gitlab.com/fdroid/fdroidclient/merge_requests/258

Our other activities included implementing background downloading of apps so that multiple apps can be downloaded, while users continue to browse F-Droid. Previously, users had to wait on the screen of a single app while it downloaded before navigating away. This is a huge improvement, and we will continue to make it even better over the coming weeks.
We also worked on the notification system. When apps are downloading in the background, Android will show a notification to indicate that this is happening. Now the notification is able to:

  • Show download progress correctly.
  • Indicate the name of the app being downloaded.
  • Disappear when it is supposed to (in some, but not all situations).
  • Navigate back to the app details page in F-Droid when touched.
  • Correctly show progress in app details when returning, e.g., from the lock screen

Towards the end of April we started looking into the general performance of F-Droid. Many people have pointed out that our recent change to ensure that low memory devices can use F-Droid has impacted performance in a very negative way. While we were aware of this trade-off, it seems that it is a little more problematic for users than we had hoped. As such,we are looking into changing the database schema in the hope of improving performance for updating repos, but also general F-Droid performance.

Objective 2 Curation Tools for Organizations

We informally presented our ideas for curation tools to some non-technical organization to start gathering feedback on how the tools should be structured and which kinds of use cases to prioritize.

Objective 3 Modern App Store with Built-in Circumvention

In April, we focused on re-architecting some of the core parts of the F-Droid client app to make it work more flexibly, to change to an event-based programming style, and to allow parallel background operation. This is mostly finished and included in release 0.100alpha5.
  • Streamlined app updating in F-Droid
  • Rearchitected core to enable parallel and background operations to support upcoming UX overhaul

For the NetCipher HTTP stack integration (https://github.com/guardianproject/NetCipher/pull/42), we added new code. We will be contributing another chunk of code shortly, as we slowly migrate some standalone implementation into the main NetCipher project.

Objective 4 Partner Deployments

We discussed distribution strategies with Storymaker and explored ideas to address country-specific challenges to getting Storymaker to people ready to produce their own media. We are waiting for feedback about the technical approach they are taking for distributing “content packs” to help us design the new media handling features in F-Droid.

Objective 5 Usability Research on In-country Developers

Interviewed potential community manager and discussed how to structure user experience research around the developer user experience.
In April we gave a talk about F-Droid and Bazaar work at LinuxWochen Austria, https://cfp.linuxwochen.at/de/LWW16/public/schedule

1 2 3 ... 5 (1-10/46)

Also available in: Atom