As Ars goes for Google I / O tradition, we recently got together with some people who are doing Android to learn about the operating system directly. In 201
This year, we have veteran interviewee Dave Burke, Vice President of Engineering for Android, at the Ars Android Interview Gauntlet. As head of the Android Team, Burke is an Android-aware encyclopedia and always manages to find insightful answers to my esoteric questions. And for the second year in a row, Iliyan Malchev, Principal Engineer on Android, is the head of Project Treble and all-round Linux integration guru.
But to improve on this latest deep dive, Ars also joined forces with Anwar Ghuloum, Android's Senior Director of Engineering and Project Mainline's lead. Ghuloum's recognition was especially welcomed given this year's I / O headliner: As the "Next Great Android Update Project," Mainline was without a doubt the biggest novelty at the conference. If you like – but first some background information on Mainline.
Project Mainline: A "Fundamental Change" in the Development of Android Operating Systems
For years, Google has been working continuously to break Android into more easily updateable parts. At the beginning of Android's life, the Google apps and core system apps were outsourced to the Android app store, so Google could provide new features to users at any time. Google Play Services then used many developer APIs and outsourced to the Android app store, so Google pumped API updates for developers whenever they wanted it. More recently Android 8.0 Project Treble took us away from the hardware support and made it easier to update.
In Android Q, the big new modularization effort is called "Project Mainline". In keeping with Google's beginnings of putting apps on the Play Store, Mainline is modularizing several core system components and moving them to the Play Store. However, Mainline goes deeper into the system than the surface-level apps – these are large parts of the system's functionality, such as the Media Framework and ART, Android RunTime.
Traditionally, the Play Store has apps distributed only in the form of APK, but for many of the components that are modularized in Project Mainline, they do not work when packaged as APK. Because the APK system was created for system-level and user-level apps, there are restrictions on things like permissions and when they can be enabled during startup. For the modularization of these core components, Google has come up with something stronger than an APK: the file type "APEX". Essentially, APEX files can have root privileges and can be launched very early in the startup process so that Google (or your OEM) can update many more components. APK files are packages for system-level and user-level apps, and APEX files are packages for core system components. This table shows the first batch of it in Android Q:
In the future, the Project Mainline modules will probably increasingly include the Android system. In this first version of Android Q, however, Google focused on three topics: "Consistency," "Security," and "Privacy." Prior to our I / O interview, Google provided us with the above table of Project Mainline components in Android Q, which outlines which components are being modularized and which recommendations apply to OEMs. And that brought us to the first question.
The following is a transcript, with some of the interviews being slightly edited for the sake of clarity. For a more comprehensive perspective, we have also included some current background comments in italics.
Ars: So I have this main project table, which specifies which components are recommended or not. How did you go about finding out what is compulsory and what is not?
Anwar Ghuloum, Project Mainline: Ideally, we want everything to be compulsory. The way we worked on these modules was to talk to all of our equipment manufacturers and say, "Hey, we do that, work with it." They have streamed a lot of code. They had a lot of future requests for things to start working on, and for those modules where we could actually meet all these requirements, we made them binding. For the modules that still have gaps, we have made them optional for this version, and they are mandatory for the next version. So this gives us time to get to parity because we do not want to undermine their device experience, but want to develop those modules to make sure their content is included.
Dave Burke, VP of Engineering: I think part of this work is up-streaming with our partners. When I say partner, I speak of equipment manufacturers. You add changes to the device you are creating, and we want them to all be transferred to the main code branch to ensure consistency. It only takes time.
Ghuloum: Yeah, I mean, we did a lot of upstreaming. It is wonderful. For some of these packages we have created more upstreams in the last year than in the last 10 years.
Burke: Yes, it's important.
I think part of it The background to this is that Project Mainline stands for Google and reclaims the final ownership of some core system codes from OEMs (also known as device manufacturers). If device manufacturers stop owning this code, Google wants to make sure that any customizations that OEMs have used to add are now supported in the standard AOSP (Android Open Source Project) code base that everyone uses.
You can imagine that Google scans the ecosystem for each module and does things like "Do you really need to customize how DNS resolver works?" When the answer was no, the Google version became mandatory. For modules where the answer was "Well, actually …", it's planned to transfer all of that to the Google version in AOSP and finally adopt the Google version. Ghouloum's statement that some packages have "done more upstreams than in the last 10 years in the last year" sounds like they are making a lot of progress. The more code is transferred in AOSP, the less code needs to be managed by OEMs. This leads to simpler and less complicated system updates.
Ghuloum: We've told our teams that the prerequisite for using a mainline module is that you publish it once a month. That you work actively with the partners, develop together, plan your roadmap and the like. The people in the team were quite taken with it and excited.
Ars: Oh, is this the plan – a monthly release for mainline modules?
Ghuloum: Well, this is our trained cadence and is determined by our security update schedule because some of the components are security relevant. The media component mainly includes codecs and extractors. One of the reasons for this module is that we addressed security holes last year and nearly 40 percent of the security holes in our security updates are due to these modules. So we say, "Hey, what if we could simply extend it to the entire ecosystem instead of burdening, taking over, testing and expelling the OEM?"
Android's media playback engine needs to download all sorts of weird file types from the Internet, and doing so in a secure manner has always been a security challenge. Android's media playback engine is called "Stagefright," a name you might know from the 2015 news cycle when a number of remote code execution vulnerabilities were discovered in the Stagefright engine. The official monthly security update schedule that Ghuloum is talking about was started in response to these vulnerabilities, and the hardening of the media framework continues today.
Today, Google creates the AOSP security updates every month and submits them to OEMs. However, this is not a perfect solution yet. Not every phone supports monthly security updates, not every phone sends these updates on a timely basis every month, and most phones only support them for two years. According to Ghuloum, Google not only takes over the testing, integration and rollout of these fixes, but also lets Project Mainline download these fixes (once all is available on Android Q) to a few flagship phones every month for the entire ecosystem.
Burke: The other thing is, we often hear from developers what we could do to simplify their lives on Android. One of the common problems is the fragmentation of slightly different behaviors in different parts of the operating system, even within the same manufacturer – the media framework they use. And so more consistency is also beneficial for the developers. It reduces errors and the work they have to do and increases the quality of apps, which is good for users.
Ghuloum: Yesterday I called this "bug consistency".
Burke: (laughing) mistake consistently! Yes that's true.
Ghuloum: There is this module named "ANGLE". It's basically implemented OpenGL on Vulcan. Currently, it is mandatory for OEMs, but developers can decide whether to use it or not. The idea is to rely on the kind of support Vulcan offers on all these devices. A consistent GL implementation is not necessarily bug-free because we never deliver error-free software that no one has ever done. For game developers, however, it is a problem that they are used to errors in drivers, but all of these different bugs and drivers are very painful. We can do that much more consistent.
Burke: The other way to think about it is: It's generally good hygiene. For example, look at the GPS rollover that took place on April 6. It brought down a few planes because they could not cope with rolling over the clock. There's always something that's going to happen in the software and you want to be able to update it, especially if it's really simple things like Conscript, our secure library, SSL library, and TLS. This is also updatable. And that's another area where you can fix the problem as certificates expire or your certification provider suddenly goes out of business. Ghuloum: Or the BoringSSL bugs.
Burke: Or the BoringSSL bugs, yes, exactly. They are kind of unsexy but basic components in the system.
Iliyan Malchev, Project Leader Treble: Years ago, there was an error in Bionic (the default C program library for Android) that was introduced by one of our partners, using the wrong character tables would have. As a result, the trading functions failed randomly, and part of the curve could in some way disrupt the games. This is incredibly hard to catch before you ship it.
Ghuloum: And the developers had to live with it for the next few years – unless it's ever patched. I've seen that with some of our own first-party apps [they]bugs in the entire ecosystem have to be bypassed. It's just a spaghetti code.
Ars: So, was there a test update that was made available to Beta Q users?
Ghuloum: Yes, actually starting from Beta 2 We've started to post updates. There are threads on reddit about it.
Ars: That's right, OK.
Ghuloum: Your devices have been restarted, and yes, we've pushed updates, tested updates, and restarted users' devices. We only do that in beta. During production, when Q is delivered, all reboots are just the user's organic reboots. We looked at the numbers, and it looks like we've reached a reasonable saturation level over the next few weeks for the people who are updating. We also have monthly security updates, which we will restart at least once a month. So you just take it. We do not want to put UX in the user's face. If an update is waiting, we are just waiting for the reboot.
Ars: OK. Do you think so will the final version look – a kind of quiet background that will not be very visible?