Structural Class Redefinition
Introduction
The Android Runtime (ART) in Android 11 introduces an extension called Structural Class Redefinition to the JVMTI API. This post will cover the capabilities of structural redefinition, as well as some of the considerations and trade-offs we encountered while implementing the feature and how we solved them. Structural class redefinition is a runtime feature that extends the class-redefinition functionality added in Android 8. Structural class redefinition allows tools, such as the Android Studio Apply Changes, to modify the class structure itself, allowing one to add new fields and methods to classes (the older ‘normal’ class redefinition limits the changes to the implementations of methods already present in the class).
This can be leveraged into great features such as extending apply changes to support adding new resources to apps. You can read about how the Android Studio ‘Apply Changes’ functionality works here and in a future blog post about how it was extended using structural class redefinition coming soon. More complex and powerful tools are being built into Android Studio to take advantage of these new features.
JVMTI is a standard API developer tools can use to interact and control the runtime at a very deep level. It is used to implement many developer tools you are likely familiar with, from the Android Studio Network and Memory profilers to the debugger to mocking frameworks like dexmaker-mockito-inline and MockK to the new Layout and Database inspectors. You can find out more about the Android JVMTI implementation and how to use it for your own tools from the android documentation.
Structural Redefinition
Structural class redefinition improves the support for class redefinition added in Android Oreo (8.0). In Oreo, only the definitions of existing methods could be changed. The object layout and the sets of fields and methods a class defines could not be changed in any way.
Structural class redefinition allows much more freedom in modifying classes, enabling one to add entirely new methods and fields to existing classes. There are no restrictions on what types of methods and fields may be added. Newly added fields are initialized to 0 or null, but agents can use other JVMTI functionality to initialize them, if desired. As with standard class redefinition, methods currently executing will continue to use their old definition, though subsequent calls to them will use the new definition. In order to ensure structural redefinition can have clear and consistent semantics the following changes are not allowed:
- Fields and methods may not be removed or have their attributes changed
- Classes may not change their name
- Class hierarchy (superclass and implemented interfaces) may not be changed
Combined with support from Android Studio structural class redefinition can be used to implement Apply Changes for most common edits. The rest of this post will cover part of how we went about implementing this feature and some of the considerations and tradeoffs we make when implementing new runtime features.
First, do no harm
The primary challenge of structural redefinition is to implement it without affecting apps running in release mode. For every developer running their code in debuggable mode and using tools such as Apply Changes or the debugger there are millions of users running these apps normally on their phones. Therefore, one of the paramount requirements for any new developer feature in ART is to not affect runtime performance when the app is not debuggable. This means that we are not able to make major changes to the core internals of the runtime. We cannot change things such as the basic layout, allocation, or garbage collection of objects, how classes are linked and loaded, or how dex-code is executed.
Objects, including java.lang.Class objects (which contain their class’s static-fields in ART), have their size and layout determined at the time they are loaded. This allows for very efficient execution since, for example in the ‘Parrot’ class shown in the image below, we always know that the `piningFor` field is contained at offset `0x8` of any Parrot. This means that ART is able to generate efficient code but makes it impossible to alter the layout of objects after they are created, since by adding new fields we change the layout of not just objects of that class but also all of its subtypes. To implement this feature what we do instead is transparently and atomically replace all the old classes and instances with their redefined counterparts.
In order to provide structural class redefinition in a way that does not degrade performance we need to reach deep into the internals of the runtime. Fundamentally, doing a structural redefinition of a class has 4 main steps.
- Create new java.lang.Class objects for all types being changed, with the new class definition.
- Recreate all objects of the old types with the newly redefined type.
- Replace/update all the old objects with their corresponding new objects.
- Ensure all compiled code and runtime state is correct with respect to the new type layout.
Chasing Performance
Like many programs ART is inherently multithreaded, both due to the (potentially) multi-threaded nature of the dex-code it runs and in order to avoid runtime pauses. At any point the runtime might be doing any number of things at once: running java language code, performing garbage-collection, loading classes, allocating objects, running finalizers or many other things.
This means that doing redefinition in a naive way has obvious races. For example what if, after we had already recreated all the old objects a new instance was created? We therefore need to be extremely careful about how we go about performing each of the steps, to ensure that nothing is ever able to see or create an inconsistent state. We need to ensure that every thread sees the transformation shown in the diagram above happens atomically, all at once.
The naive solution to this would be to stop everything dead as soon as we start redefining something. We then perform the redefinition in the way described above (create new classes and objects then replace the old ones). This has the benefit of giving us the atomicity we need without any real effort. All code is paused during any time when inconsistencies could be observed so nothing can be seen. Unfortunately, there are several problems with this approach.
For one it would be horrendously slow. There might be large numbers of objects to recreate and classes to reload (for example one might want to edit the java.util.ArrayList class which can typically have thousands of instances at any one time). A more pressing issue is that allocation is not possible with all threads stopped. This is to prevent deadlocks where, for example we wait for a paused GC thread to finish work before allocating something. This restriction is embedded deep into the design of ART and its GC. Simply changing it to remove this restriction is not feasible, especially for a feature that is only used in debugging contexts. Since a major part of structural redefinition is re-allocating all the redefined objects this is obviously not acceptable.
So what do we do now? We still need to ensure that, as far as all Java Language code is concerned the change happens instantly but we cannot stop the world as we do so. Here we can take advantage of a restriction of the Java Language, the heap and significant class-loading state is not directly observable by threads and that important GC management threads will never allocate or load classes.
This means that the only step we need to actually pause the rest of the runtime for is the replacement process. We can allocate all of the classes and new objects while other code is still running. Since none of those threads have any references to the new objects and the code being run is still the original no inconsistencies will be visible.
For anyone interested in seeing how this is all actually implemented we have linked to some of the relevant pieces of the implementation. These link to the Android Code Search interface where you can explore how Android and AOSP are created.
Since we are allowing app code to continue running, we need to be careful that the state of the world doesn’t change out from under us. To do this we have to carefully shut down various pieces of the runtime in sequence in order to ensure we can collect all the information we need and it is not invalidated while running. To accomplish this we need, at the instant of redefinition, to have a complete list of the java.lang.Class objects of the class being redefined¹ and all of its subclasses, a corresponding list of all the redefined Class objects, a complete list of all the instances of that class and a corresponding list of all the redefined objects.
Since loading new classes is rare (and we need the new Class objects in order to allocate the redefined instances) we can start by figuring out the list of classes being redefined and creating new Class objects for the redefined types. To ensure that this list remains valid and complete we need to completely stop class loading before we create this list². To do this we need to both stop new class loads from starting and wait for in-progress class definitions to finish. Once this is done we can safely collect and recreate all the redefined class objects.
Now that we have all the classes collected we need to find and recreate the instances that need to be replaced. Similar to what we did with classes, we need to pause allocations and wait for all threads to acknowledge it in order to ensure our list of objects doesn’t get stale³. Again like with classes we then simply collect all the old instances and create new versions of each of them.
Now that we have all the new objects all that is left to do is copy the field values from the old objects and actually perform the replacement. Since once we start giving threads or objects references to the replacements they will no longer be unobservable and threads while running can arbitrarily change any fields we need to finally stop all threads before we perform these last few steps. Now that all other threads have been stopped we can copy the field values from the old to the new objects.
Once this is done we can actually walk through the heap and replace all the old instances with the new redefined instances. All that is left now is doing some miscellaneous work to ensure that things such as reflection objects and all the various runtime resolution caches are updated or cleared as needed. We also make sure to keep track of enough data to allow all the code running when redefinition started to continue running.
Conclusion
With the capabilities of structural redefinition many brand new, more powerful debugging and development tools are possible. We have already talked about the improvements to apply-changes and many other teams in Android are looking to use this to make other great tools. This is just one part of the many improvements and new features we add in every Android release. You might also want to read our recent blog-post about how we improved launch time for apps in Android 11 using IO prefetching.
[1] Before doing this we perform some checks to make sure that all classes are even eligible for redefinition and the new definitions are valid but these validations are not terribly interesting.
[2] Technically, it is safe for unrelated classes to continue loading but due to the way class loading works there is no way to distinguish these cases early enough to be useful.
[3] Again details of how allocation interacts with cross-thread synchronization in ART prevents us from only pausing allocations that are instances of our redefined classes.
