Introduction
Mobile platform development is strictly associated with memory management. Although technological advancement allowed the memory capacity of mobile devices to reach the level of low-end desktop machines, the applications made by developers also grew proportionally. The main issue lies within the screen size of devices – a higher diagonal means higher graphics resolution to use and higher memory requirements. Developers that are familiar with the Android platform also know that the Garbage Collector will not completely protect the application from memory leaks. It is not hard to imagine the impact on the performance if this happens to a large, dynamic application. That’s why good memory analysis skills are essential for developers. This article will present some useful tools and practices to detect crucial memory leaks.
Tools
There are many tools that can help identify a memory leak. Here is a sample list with short descriptions of such tools:
tools
|
Name: |
Description: |
|
DDMS (Dalvik Debug Monitor Server) |
Debugging tool shipped with Android. Provides port-forwarding services, screen capture, thread information, heap dump and information, logcat, process and radio state information and many more features. Can be launched from the command line using “./ddms” command. Also it is integrated into Eclipse IDE (DDMS perspective). |
|
MAT (Memory Analyzer Tool) |
Fast Java heap analyzer which helps find memory leaks and reduce memory consumption. It is a very powerful tool that parses a heap dump and analyzes it. It is so rich-featured that it is hard to list all the features. There is a standalone version and an Eclipse plugin of MAT. For installation details please refer to this link. |
Nomenclature
Memory analysis is bound to have many dedicated terms. As these will be appearing very often in this article, here are their definitions:
Nomenclature
|
Term: |
Definition: |
|
Heap Size |
Memory allocated for the java heap. On Android platform it is limited for each Activity (depends on device). |
|
Heap Dump |
A Binary file that contains information about objects on your heap. |
|
Dominator Tree |
Graph used (in this case) to present relations between objects. Please see (wiki) for more details. |
|
Memory Leak |
A leak is when there is a reference to an unused object that is preventing that object from being garbage collected. One object can have a reference to a bunch of other objects and a single reference can prevent garbage collection of a large number of objects. The most important thing to remember is that having the GC does not prevent memory leaks! |
|
GC Roots |
GC Roots are objects that are assumed to be reachable. Typically, these include all objects referenced from the current call stacks and classes loaded by the system class loader. |
Memory Leak
A leak is when there is a reference to an unused object that is preventing that object from being garbage collected. One object can have a reference to bunch of other objects and single reference can prevent GC for large number of objects.
Fig. 1
Shallow Heap
Memory consumed by one object.
Example:
public final class myObject { // Header: 8 Bytes
private int valueA; // int: 4 Bytes
private int valueB; // int: 4 Bytes
private char valueC[]; // char: 4 Bytes
}
Total Shallow Size of myObject is 20 Bytes.
Retained Heap
Total size of Objects that could be freed by freeing one object.
Before GC : GC will run on Object A.
Fig. 2
After GC : Freeing Object A (300 bytes) will result in freeing objects B (50 bytes) and C (50 bytes). Also by freeing object B, object D (100 bytes) will be garbage collected. So by collecting object A - GC will be able to free 500 bytes (sum of shallow heaps) which is retained heap.
Fig. 3
Detecting Memory Leaks
There are several ways to spot a memory leak. This section will present a few ways of doing this.
Logcat Messages
The first way to spot a memory leak is to check the logcat output. When the garbage collector begins its work it puts message into the logcat output. It should look similar to this one:
D/dalvikm( 14302): GC_CONCURRENT freed 2349K, 65% free 3246K/9551K, external 4703K/5261K, paused 2ms+2ms
The first part of message indicates the type of GC (reason of GC). There are four different types:
Types
| Type |
Description |
|
GC_CONCURRENT |
Invoked when the heap gets too large to prevent overflow. |
|
GC_FOR_MALLOC |
Invoked when GC_CONCURENT was not run on time and the application had to allocate more memory. |
|
GC_EXTERNAL_ALLOC |
Used before Honeycomb for freeing external allocated memory. In Honeycomb and higher there is no external allocation. |
|
GC_EXPLICIT |
Invoked when System.gc is called. |
“freed 2349K,” – indicates how much memory was freed.
“65% free 3246K/9551K” – indicates the percentage of free memory left, the size of live objects and the total heap size.
“external 4703K/5261K” – indicates external memory allocation, how much external memory the app has allocated and the soft limit of allocation.
“paused 2ms+2ms” – indicates how much time it took the GC to finish collection.
With this information it is possible to tell after a few collections if the GC is successfully freeing the memory. If the allocated memory does not go down after some period of time (and keeps growing) it is clear that there is a memory leak. Here is an example of a rather insignificant leak:
Fig. 4
OutOfMemoryError Exception
When the available resources are exhausted an OutOfMemoryError exception is thrown. It may indicate that there is a memory leak. This method is not the best way to know if there is a leak as the exception may occur when the developer tries to allocate a large amount of memory (ex. bitmap) and the total heap size will exceed the platform limit. It surely indicates that the developer should rethink his memory management method.
Tracking Memory Allocations
After successfully diagnosing a memory issue it is time to find what the cause is. There are two tools that will help in profiling the application.
DDMS
DDMS is a powerful tool that can provide valuable information and a HPROF memory dump file used for MAT. To access DDMS from the Eclipse IDE select Window->Open Perspective->DDMS.
Fig. 5
The interface is pretty much self explanatory so there is no need to get into details describing it but there are two tabs that may be helpful. The first of them is the “Allocation Tracker” which shows detailed statistics of current allocations like the allocated class, allocation size and the location of allocated memory.
Fig. 6
The second one is called “Heap”. There are detailed information about how many objects of each class are allocated in memory and their actual size.
Fig. 7
Dumping the HPROF file is easy. Connect the device, run the application, use it for a while and click on the “Dump HPROF File” button located on the toolbar at the Devices tab. If there is a MAT plugin installed Eclipse will automatically open the file.
Memory Analysis Tool (MAT)
MAT is a very powerful memory analysis tool. It can be used as a standalone version or as an Eclipse plugin. The only difference between these two is how they deal with loading a HPROF file. The standalone version needs a converted HPROF file. This can be achieved using the hprof-conv program shipped with the android-sdk (localized in the tools folder). In case of using the plugin version there is no need to convert the file. To open MAT select Window -> Open Perspective -> Other -> Memory analysis.
Overview
After loading the HPROF file the user is presented with the Overview screen. It consists of the following elements:
-
Open Overview Pane – Open the main Overview screen.
-
Create a histogram from an arbitrary set of objects – Creates a histogram view (described later in the article) from objects.
-
Open Dominator Tree for entire heap – Opens the Dominator Tree view (described later in the article).
-
Open Object Query Language Studio – A tool that allows writing MAT queries.
-
Run Expert System Test –
-
Heap Dump Overview – Provides detailed information about the heap dump file.
-
Leak Suspects – Provides detailed information like size, loader and type of suspected leak objects.
-
Top Components – Provides information about the largest objects in memory. Also contains reports on possible memory wastes.
-
Open Query Browser – Consists of many useful queries which help analyzing the memory. This guide will describe the most useful ones.
-
List Objects – Shows objects with incoming or outgoing references (MAT will prompt for an object to be used for the query).
-
Show Objects by Class – Lists object sorted by the class.
-
Path to GC Roots – Shows the reference path to GC roots (with many filtering options).
-
Merge Shortest Paths to GC Roots – Finds the common paths from garbage collection roots to an object or a set of objects.
-
Immediate Dominators - Finds and aggregates on a class level all objects dominating a given set of objects.
-
Show Retained Set - Calculates the retained set of an arbitrary set of objects.
-
Find Object by Address – Allows finding of a specific object by providing its address.
-
Pie Chart – Shows the biggest objects in retained size.
-
Histogram – Shows the number of instances of each object class.
-
10. Dominator Tree – Lists object instances in a system organized by the amount of retained heap.
-
Inspector – Provides detailed information about the selected object.
Fig. 8
Histogram
One of the most useful tools is MAT. It can list the number of instances of any given class. When looking for a memory leak or any memory issues, it is good to first look at the most “endangered” classes and check how many instances there are. There is also a very useful regex field to allow searching for a specific class. The histogram view also allows calculating the minimum and precise retained heap for the selected objects.
-
Calculate Retained Size –
-
Calculate Minimum Retained Size – Calculates the minimum retained size and shows it in the table below.
-
Calculate Precise Retained Size – Calculates the precise retained size (this process may take a while) and shows it in the table below.
-
Regex pattern – Allows the user to search for the specified object class.
Fig. 9
Another part of the histogram is an item context menu. It allows to access queries (described in the overview section) which will be invoked for the selected object. It is a very important tool when it comes to diagnosing memory issues. If there is suspect for memory leakage, the developer can check the incoming references of the specific item to find out what is keeping it from being collected. Of course, MAT also allows filtering the results from such queries (for example by restricting objects to the specified class). After receiving the list of objects from the query the developers are presented with another useful tool – “Path to GC Roots”. It contains many filtering options like excluding weak references – which is one of the most common filters, since a weak reference does not prevent objects from being collected by the GC there is no need to list them. If the user is not happy with the predefined queries MAT allows him to customize or write one from scratch so there is an infinite number of possibilities. The details and practical usage of queries will be presented later on in this article.
Fig. 10
Fig. 11
Dominator Tree
The Dominator Tree is a second useful view in MAT. It lists instances of objects in a system organized by the amount of retained heap. It can be accessed from the overview or by selecting the “immediate dominators” option from the context menu of the specified object class. The former shows all objects from the memory dump, the latter finds and aggregates all the objects dominating a given set of objects on a class level. Here are some important properties of the Dominator Tree:
-
The objects belonging to the sub-tree of “x” represent the retained set of “x”.
-
If “x” is the immediate dominator of “y”, then the immediate dominator of “x” also dominates “y”.
-
The edges in the dominator tree do not directly correspond to object references from the object graph.
These three properties are really important for the good understanding of the Dominator Tree view. Using this view a skilled developer can quickly find references which are unwanted and the retained heap of each object.
Fig. 12
Queries
Queries are the basic tools designed to help get through the forest of objects. The memory analysis is a process of finding unwanted references in the stack of many objects – it is not an easy task. Filtering these objects and references makes it easier. There are two key skills that the developer should learn to successfully debug the memory. The first is the knowledge of their own application. There is no way to find a memory issue if the person looking for it does not know what the application’s object structure should look like. The second is skill with using filters and queries. If the developer knows the object structure and knows how to get the desirable view it becomes easy to find an abnormality. Here is a list of built-in queries in MAT:
Queries
|
Query: |
Options: |
Description: |
|
List objects |
With Outgoing References |
Shows objects that have an outgoing reference to the selected object. |
|
With Incoming References |
Shows objects that have an incoming reference to the selected object. [If there is a class that has many unwanted instances it is good practice to find incoming references that are keeping it from being GC] |
|
Show object by class |
With Outgoing References |
Lists objects that have an outgoing reference to the selected object and sorts them by class. |
|
With Incoming References |
Lists objects that have an incoming reference to the selected object and sorts them by class. |
|
Path to GC Roots |
With all references |
Shows the reference path from the object to GC roots including all references. |
|
Exclude weak references |
Shows the reference path from the object to GC roots excluding weak references [Those that will not keep objects from being swept by GC] |
|
Exclude soft references |
Shows the reference path from the object to GC roots excluding soft references. |
|
Exclude phantom references |
Shows the reference path from the object to GC roots excluding phantom references. |
|
Merge Shortest Paths to GC Roots. |
Same as “Path to GC Roots” |
Shows common reference paths from GC roots to the object. |
|
Java Basics |
References Statistics Class Loader Explorer |
Shows statistics for references and objects. List class loaders including their defined classes. |
|
Customized Retained Set |
Calculate the retained set of objects excluding references passed in the query argument. |
|
Open in Dominator Tree |
Creates a Dominator Tree for the selected object. |
|
Show as Histogram |
Shows a Histogram view of an arbitrary object. |
|
Thread Details |
Displays thread details and properties. |
|
Thread Overview and Stacks |
- |
|
Java Collections |
Array Fill Ratio |
Prints a frequency distribution of fill ratios of non-primitive arrays. |
|
Arrays Grouped by Size |
Shows a histogram of arrays grouped by size. |
|
Collection Fill Ratio |
Prints a frequency distribution of fill ratios of given collections. |
|
Collections Grouped by Size |
Shows a histogram of collections grouped by size. |
|
Extract Hash Set Values |
Lists elements of a selected Hash Set. |
|
Extract List Values |
Lists elements of selected LinkedList, ArrayList or Vector. |
|
Hash Entries |
Extracts key-value pairs of selected HashMaps and Hashtables. |
|
Map Collision Ratio |
Prints a frequency distribution of the collision ratios of map collections. |
|
Primitive Arrays With a Constant Value |
Lists primitive arrays with a constant value. |
|
Leak Identification |
Component Report Top Consumers |
Analyzes the component for possible memory wastes and other inefficiencies. Prints the biggest objects. |
It is not a full list of queries available in the item context menu as some of them were already described earlier. For further information about available queries please refer to the “Further Reading” section in this article.
Reports
The Memory Analysis Tool has a built-in report system which analyzes the memory dump automatically and generates reports for the user. The first type of report is a “Leak suspects” report. MAT analyzes the memory dump and checks if there are any big objects that may be kept alive by some references. Please remember that it doesn’t mean that the leak suspect will actually be a real leak. The second one is the “Top Components” report – it consists of information about possible memory wastes, largest objects and some statistics about references. It is a great place to look for memory optimization.
Leak Suspects Report
The Leak Suspects report consists of information about potential leak problems. It is important to remember that the listed objects do not have to be actual leaks. It is still a very informative report and a great place to start searching for the leak. The main components of the report are the Description and Shortest Paths to the Accumulation Point. The third component (Accumulated Objects by Class) is just a derivation from the second one (just sorted by classes).
Fig. 13
“Shortest Paths To the Accumulation Point” is a very useful component as the developer can instantly check what may keep the accumulation point from being garbage collected.
Fig. 14
Top Components Report
This is a very informative report, especially for those who are looking for a way to reduce memory usage. It consists of memory wastes suspects and some statistics about references. The whole report is self explanatory so there is no need to describe it in detail.
Fig. 15
Fig. 16
Detecting Memory Leaks using MAT
This section will present a practical use of the Memory Analysis Tool in case of a memory leak. For the purpose of this section some sample code with a prepared memory leak will be needed.
Sample Code with Memory Leak
This sample code was written to show the main causes of memory leaks and provide a material to work with MAT. There are two most common memory leak types. The first one is a static reference to a non-static inner class, which will keep a reference to the Activity and prevent it from being GC. Upon every screen orientation change the onCreate method will be invoked and a new MainActivity instance will be created. Due to old references, the old Activities will not be garbage collected. The second case is known as a “context leak”. This case is hard to spot as the main issue lies within the application’s context passed to a static class that keeps it as a field – which of course is a hard reference.
First Memory Leak Demo
Here is the first memory leak source code that will keep the MainActivity from being freed by GC. Of course object and class names were chosen so it is easier to demonstrate the leak. In a real life situation it will not be that easy.
public class MainActivity extends Activity {
// Static field of non-static inner class - very bad idea!
static MyLeakedClass leakInstance = null;
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
// Static field initialization
if (leakInstance == null)
leakInstance = new MyLeakedClass();
ImageView mView = new ImageView(this);
imView.setBackgroundResource(R.drawable.leak_background);
setContentView(mView);
}
/*
* Inner class that is not static.
*/
class MyLeakedClass {
int someInt;
}
}
First take a look at the LogCat output from this code. Connect the device, run logcat, launch the application and rotate the device a few times. Here is the LogCat output from the sample as shown in figure 17:
Fig. 17
The difference after the first rotation is about 3mb. After another rotation the heap size grows again by 3mb. It is the first warning that something is not right. Also LogCat does not show any sign of freeing the allocated memory. Now it is time to check out what is going on inside the application’s memory. Run the DDMS tool, get the HPROF file and open it inside the Memory Analysis Tool.
The Main Screen does contain some leads to the leak – it should look similar to this one shown in figure 18:
Fig. 18
The Pie Chart shows that a significant amount of the whole memory is reserved for resources – that is normal behavior of any application that has a GUI, but inside this sample code there is only one resource so the issue may lie there. There are also two FrameLayout instances (3mb each) that need investigating. There are a few paths that the developer can follow to detect memory leaks.
Histogram Based Search
Let’s take a look at the Histogram view. There is a significant amount of memory allocated by the bytes and Bitmap class. Take a look at the incoming references of a Bitmap class object. The list should look like this as shown in Figure 19:
Fig. 19
There are three large bitmaps in memory. The sample application should have one or two memory leaks at worst. The amount of memory fits the LogCat output. Let’s check the path to GC roots (excluding all types of weak references) for each of these instances. The first bitmap object seems to be fine it has a reference to itself so it is not held by any external reference and is waiting for the garbage collector to take care of it.
Fig. 20
The second has a little more references but everything seems ordinary. It seems that this object is an active (visible) bitmap – that is why it has references to the Activity, Context, WindowManager etc.
Fig. 21
The third bitmap is a hit. It has only one hard reference path to GC Roots and it is held by the “leakInstance” object. That shows that the “leakInstance” object is preventing collection of the bitmap.
Fig. 22
Also there is MainActivity on the path – which is not surprising looking at the fact that with every screen rotation (which creates a new Activity) there is a leak. Let’s take a look what is happening there. First use the Regex filter to find MainActivity objects inside the Histogram view.
Fig. 23
The first warning is that there are three instances of MainActivity. This of course does not have to be a leak, as sometimes Activities are kept alive longer than other objects but let’s take a look if anything is preventing them from getting GC. To do this, list all the objects with incoming references. As the histogram has shown there are three different instances. It is time to check the path to GC Roots.
Fig. 24
Again the first MainActivity object has a reference to the context, and ActivityThread so it looks like it is currently the active Activity.
Fig. 25
The second object again has a reference to itself so it is waiting to be garbage collected. Everything is fine up to this point.
Fig. 26
Now for the third one – that’s a hit! There is a hard reference to the leakInstance object which prevents this Activity from being swept by the GC.
Fig. 27
Dominator Tree Based Search
There are many ways a developer can find a memory leak. This article will only demonstrate a few of them. The second one will be based on a Dominator Tree view. Open a Dominator Tree view of HPROF dump. Sort the contents of the list by the Retained Heap size. As predicted the resources on top of the list are the ones with the largest retained heap but there are also three instances of the FrameLayout class (3mb each) and a Bitmap class instance (1mb). The FrameLayout objects look suspicious so let’s take a closer look at them. As the Dominator Tree already shows real objects so it is possible to show the path to GC Roots without first listing the references.
Fig. 28
The first item is actually a hit! The only path to GC Roots is through the “leakInstance” object so there is a leak.
Fig. 29
The second and third objects are the current Frame Layout view and one that is waiting to get collected.
Fig. 30
Let’s take a look at the bitmap object as it may also lead to a leak. Select android.graphics.Bitmap and show its path to GC Roots excluding all weak references.
Fig. 31
There are three objects of bitmap type so for each of them find a path to GC Roots.
The situation repeats itself – two of the three instances are clear (they have references to the system classes) but the third instance points to the “leakInstance” so it is kept alive by this reference.
Fig. 32
There are probably hundreds of ways of getting to the actual leak. It is up to the developer to choose which path he should take and how to analyze the memory.
Second Memory Leak Demo
The second memory leak scenario includes the application context. It is passed as an argument to another singleton class and kept there in a field. The whole process will keep MainActivity from being collected by the garbage collector. The same result could be achieved by keeping the context as a static field so such practices should be avoided. To avoid repetition only one way of finding leaks will be described.
Here is the code:
public class MainActivity2 extends Activity {
SingletonClass mSingletonClass = null;
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
mSingletonClass = SingletonClass.getInstance(this);
}
}
class SingletonClass {
private Context mContext = null;
private static SingletonClass mInstance;
private SingletonClass(Context context) {
mContext = context;
}
public static SingletonClass getInstance(Context context) {
if (mInstance == null) {
mInstance = new SingletonClass(context);
}
return mInstance;
}
}
After running the application, dumping the HPROF file and running it in MAT an Overview screen similar to this one will be presented.
Fig. 33
The overview screen does not tell anything important so the developer has to continue his search. In this snippet there are no bitmaps or many resources but the Histogram view shows that there are many instances of MainActivity – checking them may provide some further information.
Fig. 34
The device was rotated 3 times and the Histogram shows that there are four instances of MainActivity. It is time to check if there is any object that keeps them from garbage collection. To do this, list the objects with incoming references. Just by extending the view of the first instance there is a hint that this object is the current Activity (it contains references to the ActivityThread).
Fig. 35
Continuing the search and listing the paths to GC Roots there are two instances that contain references to themselves and one instance that points to mInstance inside SingletonClass and a reference to the current Activity (from mSingletonClass). That is a leak.
Fig. 36
It is clearly visible that garbage collection is prevented by the context. There is also another issue – after creating another instance of the Activity the context passed to SingletonClass remains unchanged. This is a critical issue. The context reference points to an Activity that is no longer needed but is kept alive because of this. Such issues may be critical for the application, can create unwanted behavior and are hard to spot in bigger projects.
Additional Reading
Memory Analysis is an extensive topic and cannot be presented with details in a short article. There are many books, articles and presentations dedicated to memory management. Here is a short list that the reader may use to further sink into this topic.
