MyNotes: Jacoco Round 2

 

The testing landscape has changed since my first jacoco post.  A bug was introduced in v22 of the tools.  If you are interested you can find the details at https://code.google.com/p/android/issues/detail?id=170607.  The gist of the bug is the coverage report file is empty which interferes with JaCoCo working. Basically, it shows you 0% coverage across the board.

The good news is that there is a workaround!  You have to create a custom test runner and then everything is golden.   Since some the last post is invalid now, I am including the full steps to getting JaCoCo to work for the android tests again.

1. Creating the Workaround Test Runner

• Copy the following class into the androidTest folder:

   1:  

   2: /**

   3:  * An annoying work around for

   4:  * <a href="https://code.google.com/p/android/issues/detail?id=170607">issue 170607</a>

   5:  */

   6: public class JacocoTestRunner extends AndroidJUnitRunner {

   7:  

   8:         static {

   9:             System.setProperty("jacoco-agent.destfile",

  10:                                "/data/data/"+BuildConfig.APPLICATION_ID+"/coverage.ec");

  11:         }

  12:  

  13:         @Override

  14:         public void finish(int resultCode, Bundle results) {

  15:             try {

  16:                 Class rt = Class.forName("org.jacoco.agent.rt.RT");

  17:                 Method getAgent = rt.getMethod("getAgent");

  18:                 Method dump = getAgent.getReturnType().getMethod("dump", boolean.class);

  19:                 Object agent = getAgent.invoke(null);

  20:                 dump.invoke(agent, false);

  21:             }

  22:             catch (Throwable e) {

  23:                 e.printStackTrace();

  24:             }

  25:             super.finish(resultCode, results);

  26:         }

  27: }

2. Update Gradle

• Make the following updates to the modules gradle file (the real file can be at build_coverage.gradle):

   1: ...

   2: apply plugin: "jacoco"

   3:  

   4:  

   5: android {

   6:     ...    

   7:     jacoco {

   8:         version "0.7.1.201405082137"

   9:     }

  10:  

  11:     defaultConfig {

  12:         ...

  13:         //workaround for issue 170607.

  14:         testInstrumentationRunner "com.fsk.mynotes.JacocoTestRunner"

  15:         //testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"

  16:     }

  17:  

  18:     buildTypes {

  19:         debug {

  20:             testCoverageEnabled = true

  21:         }

  22:     }

  23:  

  24: //

  25: //    dexOptions {

  26: //        incremental true

  27: //    }

  28: ...

  29: }

  30:  

  31: task jacocoTestReport(type: JacocoReport) {

  32:  

  33:     def coverageSourceDirs = [

  34:             'src/main/java'

  35:     ]

  36:  

  37:     group = "Reporting"

  38:     description = "Generates Jacoco coverage reports"

  39:     reports {

  40:         xml {

  41:             enabled = true

  42:             destination "${buildDir}/reports/jacoco/jacoco.xml"

  43:         }

  44:         csv.enabled false

  45:         html {

  46:             enabled true

  47:             destination "${buildDir}/jacocoHtml"

  48:         }

  49:     }

  50:  

  51:     classDirectories = fileTree(

  52:             dir: 'build/intermediates/classes/debug',

  53:             excludes: ['**/R.class',

  54:                        '**/R$*.class',

  55:                        '**/BuildConfig.*',

  56:                        '**/Manifest*.*',

  57:                        '**/*Activity*.*',

  58:                        '**/*Fragment*.*'

  59:             ]

  60:     )

  61:  

  62:     sourceDirectories = files(coverageSourceDirs)

  63:     additionalSourceDirs = files(coverageSourceDirs)

  64:     executionData = files('build/outputs/code-coverage/connected/coverage.ec')

  65:  

  66:     doFirst {

  67:         new File("$buildDir/intermediates/classes/").eachFileRecurse { file ->

  68:             if (file.name.contains('$$')) {

  69:                 file.renameTo(file.path.replace('$$', '$'))

  70:             }

  71:         }

  72:     }

  73: }

  74:  

  75: dependencies {

  76:     ...

  77:  

  78:     //Unit test dependencies

  79:     testCompile 'junit:junit:4.12'

  80:     testCompile 'org.mockito:mockito-core:1.10.19'

  81:     testCompile 'org.powermock:powermock-module-junit4:1.6.2'

  82:     testCompile 'org.powermock:powermock-api-mockito:1.6.2'

  83:  

  84:     androidTestCompile 'org.hamcrest:hamcrest-library:1.1'

  85:     androidTestCompile 'com.android.support.test:runner:0.3'

  86:     androidTestCompile 'com.android.support.test:rules:0.3'

  87:     androidTestCompile 'com.android.support.test.espresso:espresso-core:2.2'

  88:     androidTestCompile "com.google.dexmaker:dexmaker:1.2"

  89:     androidTestCompile "com.google.dexmaker:dexmaker-mockito:1.2"

  90: }

• Sync the project to the gradle file. This should happen automatically, but if it doesn’t go to Tools=>Android=>Sync Project with Gradle Files.

3. Running the tests for Coverage

• Start an emulator or a device that can run the tests.


• Open the Android Studio Terminal and enter the following commands:

   1: gradlew :MyNotes:createDebugCoverageReport

   2: gradlew :MyNotes:jacocoTestReport

(substitute the MyNotes with your module name)

• Assuming that everything succeeded, open the module’s build\jacocoHtml folder.


• Right-Click on the index.html file and open it in a browser.

Caveats

• The androidTests only contain the tests that I either can’t get working as local tests or that will take more work than it is worth.  As a result, I have two test categories, Android Tests and Local Tests, that give me coverage for different areas of the code.  Unfortunately, this means that I will likely never get a single tool to give me 100% coverage.


• Sometimes the terminal commands will fail or will not produce a coverage report.  When that happens just clean the build and repeat the terminal commands.


• I created a special gradle file, build_coverage.gradle, to contain the changes for the coverage.  I didn’t want it in my main gradle file because I do not want to deliver the changes in the release version.  When I want to run coverage, I copy the special gradle file into the main gradle contents and then revert the change when I am done.

Commits

The majority of the  changes can be found on github at https://github.com/fsk-software/mynotes/commit/ed209572c5e1302ac03288a84b4e2003cb5894e7.

MyNotes: Decisions, Decisions, Decisions

 

I have finally reached feature parity (gratuitous buzzword usage) with the last released version of MyNotes. That leaves me at a big decision point.  I can either keep working on new features:

• Table UI designs
• Share Features
• Voice control
• Photo Notes
• Android M updates

or I can begin prepping to release.

I still have to do a lot of unit test updates (the gift that keeps on giving…) before I make my choice.  So that gives me some time, but I will have to decide soon.  It would be nice to have it back on the market, but I am really tempted to get the tablet design working first.

I haven’t pushed the latest round of changes yet.  I want to get those pesky unit tests whipped back into shape first.  I should be able to do that within a couple of days(hopefully/maybe?).

Which leads to my next decision : what should my next blog entry be about: Observer\Obsversable pattern, Compound Custom Views, or ViewAnimators(ViewFlipper/ImageSwitcher)… 

We will all know that when I finish the unit tests and find time to write it.

I think I also need to find a new Plugin for formatting code on the blog.  I am really not happy with the current one I am using,  Maybe I will figure that out by my next post also…

MyNotes: Unit Test Round 2

Recently, Android updated the unit test framework to include Junit4 and mockito.  The testing framework now comes in two flavors: Local Tests and Instrumentation Tests.  The local tests are true units that do not run on an emulator or device.   As a result, everything has to mocked.   The instrumentation tests are pretty much the same as the earlier tests: They require an emulator or device and run the full android system.  

Android provides a handy write up on the Android Developers site: Building Local Unit Tests

I followed their instructions fairly closely with the big deviation that I brought in PowerMockito as well.

I followed the same pattern in both the MyNotes and CommonLibrary projects.  For simplicity, I am only going to detail the changes to one of the projects.

Step 1: Update Gradle

Here are the changes I had to make in order for the new unit test framework to work:

android {
    ...

    defaultConfig {
        ...
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
    }
	...
}


dependencies {
	...
    //Unit test dependencies
    testCompile 'junit:junit:4.12'
    testCompile 'org.mockito:mockito-core:1.10.19'
    testCompile 'org.powermock:powermock-module-junit4:1.6.2'
    testCompile 'org.powermock:powermock-api-mockito:1.6.2'

    androidTestCompile 'org.hamcrest:hamcrest-library:1.1'
    androidTestCompile 'com.android.support.test:runner:0.3'
    androidTestCompile 'com.android.support.test:rules:0.3'
    androidTestCompile 'com.android.support.test.espresso:espresso-core:2.2'
    androidTestCompile "com.google.dexmaker:dexmaker:1.2"
    androidTestCompile "com.google.dexmaker:dexmaker-mockito:1.2"
}

Step 2: Create The Local Unit Test Packages

So the instrumentation tests (those that require an emulator) are in the androidTest folder that is a sister to the main folder.  The local unit tests get their own folder, test, at the same level as the main folder also:

The “test” folder must contain a folder named “java”.  Under the java folder is the basic package layout for the classes that will be tested.

Step 3: Creating a Local Unit Test

@RunWith(PowerMockRunner.class)
@PrepareForTest({LocalBroadcastManager.class, ThreadUtils.class, MyNotesApplication.class, Intent.class,
        NoteFilterBroadcast.class})
public class NoteFilterCacheTest {

    @Mock
    private Context mMockContext;

    @Mock
    private SharedPreferences mMockSharedPreferences;

    @Mock
    private SharedPreferences.Editor mMockSharedEditorPreferences;

    @Mock
    private LocalBroadcastManager mMockLocalBroadcastManager;

    @Mock
    private Intent mMockIntent;

    @Before
    public void prepareForTest() throws Exception {
        when(mMockContext.getSharedPreferences(NoteFilterCache.CACHE_NAME, Context.MODE_PRIVATE))
                .thenReturn(mMockSharedPreferences);

        PowerMockito.mockStatic(LocalBroadcastManager.class);
        when(LocalBroadcastManager.getInstance(mMockContext)).thenReturn(mMockLocalBroadcastManager);

        when(mMockSharedPreferences.edit()).thenReturn(mMockSharedEditorPreferences);
        when(mMockSharedEditorPreferences.clear()).thenReturn(mMockSharedEditorPreferences);
        doNothing().when(mMockSharedEditorPreferences).apply();
        when(mMockSharedEditorPreferences.putBoolean(anyString(), anyBoolean())).thenReturn(mMockSharedEditorPreferences);

        PowerMockito.whenNew(Intent.class).withAnyArguments().thenReturn(mMockIntent);
    }

    /**
     * Test constructor
     */
    @Test
    public void testConstructorFailure() {
        try {
            new NoteFilterCache(null);
            assert false;
        }
        catch (NullPointerException e) {
        }
    }


    /**
     * Tests method {@link NoteFilterCache#isColorEnabled(NoteColor)} and
     * method {@link NoteFilterCache#getEnabledColors()}
     */
    @Test
    public void testRetrievingTheDefaultValuesForEachFilter() {
        NoteFilterCache cache = new NoteFilterCache(mMockContext);

        when(mMockSharedPreferences.getBoolean(anyString(), eq(true))).thenReturn(true);
        Set < NoteColor > actualEnabledColors = cache.getEnabledColors();
        for (NoteColor color : NoteColor.values()) {
            assertThat(cache.isColorEnabled(color), is(true));
            assertThat(actualEnabledColors.contains(color), is(true));
        }
    }


    /**
     * Tests method {@link NoteFilterCache#isColorEnabled(NoteColor)}
     */
    @Test
    public void testCheckingEnabledColorForNullValue() {
        try {
            new NoteFilterCache(mMockContext).isColorEnabled(null);
            assert false;
        }
        catch (NullPointerException e) {
        }
    }


    /**
     * Tests method {@link NoteFilterCache#enableColor(NoteColor,
     * boolean)}.
     */
    @Test
    public void testDisablingEachColor() {
        final NoteFilterCache cache = new NoteFilterCache(mMockContext);

        when(mMockIntent.putExtra(eq(NoteFilterBroadcast.Extras.COLOR), anyInt())).thenReturn(
                mMockIntent);
        when(mMockIntent.putExtra(NoteFilterBroadcast.Extras.ENABLED, false)).thenReturn(
                mMockIntent);

        for (final NoteColor color : NoteColor.values()) {
            cache.enableColor(color, false);
        }

        verify(mMockSharedEditorPreferences,  times(NoteColor.values().length)).putBoolean(
                anyString(), eq(false));
        verify(mMockLocalBroadcastManager, times(NoteColor.values().length)).sendBroadcast((Intent) anyObject());
    }


    /**
     * Tests method {@link NoteFilterCache#enableColor(NoteColor,
     * boolean)}.
     */
    public void testEnablingEachColor() {
        final NoteFilterCache cache = new NoteFilterCache(mMockContext);

        for (final NoteColor color : NoteColor.values()) {
            cache.enableColor(color, true);
            verify(mMockSharedEditorPreferences.putBoolean(color.name(), true));
        }

        verify(mMockLocalBroadcastManager, times(NoteColor.values().length)).sendBroadcast((Intent) anyObject());
    }
}

This is your basic Junit 4 test.  The important thing to remember is that you have to mock all Android classes that are called from the unit under test.  For most part, this will go smoothly, but there are a few classes that just do not mock well (Point, Rect, etc).  This is because the classes use final public attributes instead of accessor methods which make them incredibly hard to mock.  Also be warned, that you will have to mock everything from the framework, this includes things some very basic and commonly used classes like TextUtils.  Android does provide a way to use default mock values, but I typically don’t use it. If you are interested, just add the following statement to your gradle file:

android {
  // ...
  testOptions { 
    unitTests.returnDefaultValues = true
  }
}

The local unit tests work really well for most of the business logic in the app.  It does not work well for UI components, the Database, Loaders, Broadcast Receivers, etc.  Basically, anything that is highly dependent on an actual Android framework. 

Step 4: How to run

• Open the Build Variants tool in the Android Studio (located in the lower left corner:

• From the Test Artifacts menu, select Unit Tests
  
• Navigate to a unit test and do a right-click:

• Select the Run or Debug option of your choice.
  
• After the tests run, the results are displayed in the junit results:

Instrumentation Tests

These are very similar to the old instrumentation tests.  The biggest change is that they are now JUnit4 (Yeah!) and also support Mockito.  It will support PowerMockito as well, but you can’t mock any statics (unfortunately).   You also have to add a Test Runner to the top of each test:

/**
 * Test the {@link Note class}.
 */
@RunWith(AndroidJUnit4.class)
public class NoteContentValuesTest {

Running these is very similar to the local unit test, except you choose the “Android Instrumentation Tests”  in the Build Variant menu:

My Final Note

This is more of an FYI.  The Test Artifact selected in the Build Variant are the only ones that will compile and run.  So if you are working on Local Unit Tests, make sure to select the correct build variant first so that you can see the syntax issues while you writing the tests.  The same goes for the Instrumentation Tests.

It is also important to compile each variant before considering the task complete since compilation issue won’t show up for the unselected variant.

Commits

As always, the commits to support this entry can be found on GitHub at https://github.com/fsk-software/mynotes/commit/ef1dd5389c7783deb05cbacf43c5e5f5628398f6.

MyNotes: Coverage Reports

A lot has changed since I initially created this post.  Please see the updated version at http://fsk-consulting.blogspot.com/2015/08/mynotes-jacoco-round-2.html.

MyNotes: Updating the Database

I am going to keep my database schema the same.

I don't really like the names of the database, table, or columns.  However, I don't want to change the schema since the app has already been released and the schema will still work for the expected functionality.  It sucks, but that is one of the trade-offs that have to be made sometimes.

The same cannot be said for the code that defines and manages the database.  That is going to get a major overhaul.

Step 1: Updating the Common Library

My first order of business is create some classes in the CommonLibrary to help with future database needs.  The classes are :

1. CommonTerms - An interface that defines common terms and phrases used to define and create database schemas. Basically, just a constants class.
2. DatabaseModel - An interface that defines the common methods for defining and setting up a database.  This will let me use the DatabaseHelper for my database management.
3. DatabaseHelper - A helper class to manage the databases for an app.

These classes are located under the com.fsk.common.database folder.  Take a peek at them when you have a chance to understand what they are.

Step 2: Adding the CommonLibrary Unit Tests

The first step here is to update the manifest to let me build an apk.  The library manifest suffers the same issue the MyNotes manifest has with respect to the duplicate file problem.  That was discussed briefly in the Setting Up the Unit Tests entry.

Basically, I just added the following entry to the Android block in the CommonLibary gradle file:

    //Required to prevent duplicate files during the build due to the imported libraries.

    packagingOptions {

        exclude 'LICENSE.txt'

        exclude 'META-INF/DEPENDENCIES.txt'

        exclude 'META-INF/LICENSE.txt'

        exclude 'META-INF/NOTICE.txt'

        exclude 'META-INF/NOTICE'

        exclude 'META-INF/LICENSE'

        exclude 'META-INF/DEPENDENCIES'

        exclude 'META-INF/notice.txt'

        exclude 'META-INF/license.txt'

        exclude 'META-INF/dependencies.txt'

        exclude 'META-INF/LGPL2.1'

    }

I also added a block to the same gradle file to support java 7:

    compileOptions {

        sourceCompatibility JavaVersion.VERSION_1_7

        targetCompatibility JavaVersion.VERSION_1_7

    }

Next, I created the first unit test for the DatabaseHelper file.  I don't need unit tests for the other two files so we can move onto the MyNotes module changes.

Step 3: Updating the MyNotes Module

These changes all resolve around removing or deprecating the old database code and replacing it with the new database setup.

In the original code, the database setup/management was messy and hard to follow because it was split between three main files: DatabaseKeys, NotesTableManager, and DatabaseManager.  Unfortunately, some of the Database management leaked into other classes, which I am going to deal with in the next entry.

For the new design, I moved the entire database schema into the MyNotesDatabaseModel class (com.fsk.mynotes.data.database).  This class implements the CommonLibrary's Database Interface so that I can just use the DatabaseHelper (or an extension of it) to manage my database setup and access.

The original code also has some issues with the timing and survival of the database.  Unfortunately, I created the database in the Activities which meant that sometimes the database went down and there was a lag between the activity starting and the database being ready.

This time around I want to centralize my database initialization into one area that will guarantee it stays alive for the entire application life cycle.  I also want to be sure it is alive before the first activity creates. Luckily, I can do this by creating a custom Application class : MyNotesApplication.

I have to modify the manifest to tell it to use the MyNotesApplication class instead of the system default one. this is done by adding the following line to the manifest application element :

<application

  android:name=".MyNotesApplication"

  android:label="@string/app_name"

  android:icon="@drawable/my_icon">

...

</application>

In MyNotesApplication, I extended the OnCreate method to create the database and initialize the DatabaseHelper with it.

My last step is to do a little cleanup and preparation for removing the NoteTableManager.  The NoteTableManager is going to be removed soon, but it is too big of task for this entry.  For now, I am just going to deprecate and modify it internally to use the DatabaseHelper.

Commits

That is it for this round of mods.  The commit can be found at https://github.com/fsk-software/mynotes/commit/9e368c82f2b5fc7291e2a48127e35ea1da26c7b5

*Technically, the code will run at this point, but it looks bad and is only semi-functional.  The next few sets of changes will probably render it inoperable for a while.

MyNotes: Setting up the Unit Test Framework

The next step is to set up the unit test framework for the project.  

There are a lot options for this but I prefer a very simple setup using only junit, mockito, and hamcrest.  Robolectric is also a great option and will dramatically speed up the tests.  However, I have had issues with it when mocking statics so I tend to stay away from it unless I think the slower speed of  running on an emulator will cause problems.

I start my setup by creating a sister folder to my main folder.  The new folder is named androidTest and will contain all of my unit tests.  Under  the androidTest folder,  I duplicate the same folder structure as my main folder.

Next we have to modify the gradle.build file in the MyNotes modules to add dependencies for the unit tests.

This can be tricky to get working correctly, but the setup that works for me is to add the following lines to the dependencies block:


    //Unit test dependencies
    androidTestCompile group: 'org.hamcrest', name: 'hamcrest-core', version: '1.3'
    androidTestCompile group: 'junit', name: 'junit-dep', version: '4.11'
    androidTestCompile("org.mockito:mockito-core:1.9.+") {
        exclude group: 'org.hamcrest'
    }

This will only include the hamcrest, junit and mockito libraries only when running the unit tests.
The junit and hamcrest inclusion will cause a duplicate file build error due to both libraries including a LICENSE.txt file.  This error (and many more like it) can be fixed by including the following block in the android section of the same gradle file:

packagingOptions {
        exclude 'LICENSE.txt'
        exclude 'META-INF/DEPENDENCIES.txt'
        exclude 'META-INF/LICENSE.txt'
        exclude 'META-INF/NOTICE.txt'
        exclude 'META-INF/NOTICE'
        exclude 'META-INF/LICENSE'
        exclude 'META-INF/DEPENDENCIES'
        exclude 'META-INF/notice.txt'
        exclude 'META-INF/license.txt'
        exclude 'META-INF/dependencies.txt'
        exclude 'META-INF/LGPL2.1'
    }

 Now we just sync the gradle file and it should be ready to start writing unit tests.

The first test is really simple and mainly to test the unit test setup.  I named it NoteTest.java and it is located under androidTest/java/com/fsk/mynotes/data.

The class inherits off AndroidTestCase and only includes one test for now.  All of the test methods must start with "test" or have the @Test annotation :

package com.fsk.mynotes.data;

import android.test.AndroidTestCase;

/**
 * Test the {@link com.fsk.mynotes.data.Note class}.
 */
public class NoteTest extends AndroidTestCase {

    public void testNote() throws Exception {
        Note note = new Note(-1, null, 0, null);
        assertEquals(0, note.getColor());
        assertEquals(-1, note.getRow());
        assertEquals(null, note.getText());
    }
}

And of course the next step is to run the test.  Here are the steps to create the unit test run configuration in AndroidStudio:

1. Run->Edit Configurations...
2. Highlight Android Tests in the left menu
3. Press the green + button
4. Give the configuration a name.  In my case I called it "Test All".
5. Select the module to test (MyNotes)
6. Select the tests to run.  For this configuration, I want all tests to run.

My ending configuration looks like this:

You can also take a shortcut for individual test files:

1. Select the test in the projects window
2. Right click
3. Select either "Run xxxx" or "Debug xxxx"
4. Select the unit test configuration.  It is the one is the one with the little android icon.

Then just the run the configuration of your choice and watch the test results in the Studio:

Tada, it works!

The commit for the entry is https://github.com/fsk-software/mynotes/commit/f025e582c5dddc06fe9b746d194f52c7ad04d7aa

BTW: Git doesn't like committing empty directories, so I added a ".gitkeep" file into the empty directories for the commit.  The file will be removed later once I actually add content.