Showing posts with label Database. Show all posts
Showing posts with label Database. Show all posts

Wednesday, April 22, 2015

MyNotes: Loaders

So I now have data and I have a rudimentary UI. Now I how did I get them to work together?  The answer: Loaders.
Loaders are fun.  They take care of the data synchronizations issue for you.  If the data updates in the database, the loader gets notified.  This makes things really nice in an app that is going to be manipulating the database via user input.
In some cases the notifications are automatically.  I didn’t get that lucky.  I already built my data layer and was happy with it before I decided to use Loaders so I get to write my own.  Meet the FilteredNoteLoader.

FilteredNoteLoader

/**
 * The loader to retrieve the filtered note list and listen for any changes to it.
 */
public class FilteredNoteLoader extends AsyncTaskLoader<List<Note>> {

    /**
     * The filtered notes retrieved from persistent storage.
     */
    private List<Note> mNotes;


    /**
     * The notes manager that retrieves data from persistent storage.
     */
    private final NotesManager mNotesManager;


    /**
     * The filter manager that retrieves data from persistent storage.
     */
    private final NoteFilterCache mNoteFilterCache;


    /**
     * A local broadcast manager to listen for broadcasts upon any changes to the notes or note
     * filter.
     */
    private final LocalBroadcastManager mBroadcastManager;


    /**
     * A flag that indicates that the loader is monitoring changes to the notes or note filter.
     */
    private boolean mObserverRegistered;


    /**
     * The broadcast receiver to handle any changes in the note or note filter.
     */
    private final BroadcastReceiver mOnChangeBroadcastReceiver = new BroadcastReceiver() {
        @Override
        public void onReceive(final Context context, final Intent intent) {
            onContentChanged();
        }
    };


    /**
     * Constructor.
     *
     * @param context
     *         The context to associate with this loader.
     */
    public FilteredNoteLoader(@NonNull Context context) {
        super(context);

        mNotesManager = new NotesManager(DatabaseHelper.getDatabase());
        mBroadcastManager = LocalBroadcastManager.getInstance(context);
        mNoteFilterCache = new NoteFilterCache(context);
    }


    @Override
    public List<Note> loadInBackground() {
        Set<NoteColor> enabledFilters = mNoteFilterCache.getEnabledColors();
        List<Note> data = mNotesManager.getNotesWithColors(new ArrayList<>(enabledFilters));

        return data;
    }


    @Override
    public void deliverResult(List<Note> data) {
        if (isReset()) {
            return;
        }

        mNotes = data;

        if (isStarted()) {
            super.deliverResult(data);
        }
    }


    @Override
    protected void onStartLoading() {
        if (mNotes != null) {
            deliverResult(mNotes);
        }

        registerReceivers();

        if (takeContentChanged() || mNotes == null) {
            forceLoad();
        }
    }


    @Override
    protected void onStopLoading() {
        cancelLoad();
    }


    @Override
    protected void onReset() {
        onStopLoading();

        if (mNotes != null) {
            mNotes = null;
        }

        unregisterReceivers();
    }


    /**
     * Register the observer to monitor changes to the note or note filter.
     */
    private void registerReceivers() {
        if (!mObserverRegistered) {
            mBroadcastManager.registerReceiver(mOnChangeBroadcastReceiver,
                                               NoteFilterBroadcast.createIntentFilter());

            mBroadcastManager.registerReceiver(mOnChangeBroadcastReceiver,
                                               NoteTableChangeBroadcast.createIntentFilter());

            mObserverRegistered = true;
        }
    }


    /**
     * Stop monitoring changes to the note and note filter.
     */
    private void unregisterReceivers() {
        mObserverRegistered = false;

        mBroadcastManager.unregisterReceiver(mOnChangeBroadcastReceiver);
    }
}

This class reads from two persistent sources:  The database is accessed via NotesManager and the selected color filters via NoteFilterCache (a façade to the PreferenceManager for saving the filter note colors). The loader queries for the filtered notes and then stores them for later use. It will automatically notify any listeners of the updated data. 

The one big glitch is that the Loader requires a data observer to monitor the data sources.  I have to provide that observer. My solution is too use BroadcastReceivers.  When any note commits a change to the database it also sends a broadcast about the change.  The loader listens for the broadcast and marks the content as changed.  It does the same thing for the note filter cache.

Here is the magic code from Note to make this work:

    /**
     * @throws com.fsk.common.threads.ThreadException
     *         when call from the UI thread.
     */
    @Override
    public void save(final SQLiteDatabase db) {

        ThreadCheck.checkOffUIThread();
        long row = db.insertWithOnConflict(Tables.NOTES, null, createContentValues(),
                                           SQLiteDatabase.CONFLICT_REPLACE);

        if (row != NOT_STORED) {
            mId = row;
            MyNotesApplication.sendLocalBroadcast(NoteTableChangeBroadcast.createIntent());
        }
    }


    /**
     * @throws com.fsk.common.threads.ThreadException
     *         when call from the UI thread.
     */
    @Override
    public void delete(final SQLiteDatabase db) {

        ThreadCheck.checkOffUIThread();
        if (getId() != NOT_STORED) {
            int deletedRows = db.delete(Tables.NOTES, Columns.NOTE_ID + " = ?",
                                        new String[] { Long.toString(mId) });
            if (deletedRows > 0) {
                setId(NOT_STORED);
                MyNotesApplication.sendLocalBroadcast(NoteTableChangeBroadcast.createIntent());
            }
        }
    }


For the record, if I had used a ContentProvider for my data access layer I wouldn’t need this setup.  I would just use a CursorLoader instead.

Hooking it Up


I now need to hook up the Loader to my Activity.  The Activity provides a LoaderManager.  Just register the loader into the LoaderManager and then implement the LoaderManager.LoaderCallbacks interface.

public class MainActivity extends Activity implements LoaderManager.LoaderCallbacks<List<Note>> {

    /**
     * The identifier of the main loader.  This loads the color filtered note list.
     */
    static final int MAIN_LOADER_ID = 0;

    ...

    @Override
    protected void onCreate(final Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        ...
        getLoaderManager().initLoader(MAIN_LOADER_ID, null, this);
    }


    @Override
    public Loader<List<Note>> onCreateLoader(final int id, final Bundle args) {
        return new FilteredNoteLoader(this);
    }


    @Override
    public void onLoadFinished(final Loader<List<Note>> loader, final List<Note> data) {
        mCardAdapter.setNotes(data);
    }


    @Override
    public void onLoaderReset(final Loader<List<Note>> loader) {
        //nothing
    }
}

Commits


The commits associated with these changes can be found at github here.

Wednesday, February 18, 2015

MyNotes: Building the Data Access Layer

Now the groundwork is laid, I can create my actual data access layer.  This basically consists of two classes : Note, and NotesManager.

Note

The Note class is a glorified POJO.  It is a very simple data object with 3 attributes:
  •  a long identifier that is defaulted to -1
  •  a non-null String text object to contain the note text.  This is defaulted to an empty String.
  •  a non-null NoteColor that is the color for the note. This defaulted to the yellow color.
It also has the associate getter and setters for each attribute.

The glorified piece comes from the two interfaces it implements: DatabaseStorable (discussed in the previous post) and Parcelable.  This makes the note aware of the Database so that it can update/delete its backing data.  The Parcelable interface means that the data can be marshalled into a Parcel.

What is Parceling?


Serialization in Android is very slow.  As a result, they created another way to marshal data in an efficient way.  It is very simple to implement, but remember that order matters.  To marshal the Note I only have two implement the writeToParcel method:

@Override
    public void writeToParcel(final Parcel dest, final int flags) {
        dest.writeLong(mId);
        dest.writeString(mText);
        dest.writeInt(mColor.ordinal());
    }

Then I implement a special constructor that takes in a Parcel object:

    public Note(Parcel source) {
        mId = source.readLong();
        mText = source.readString();
        mColor = NoteColor.getColor(source.readInt());
    }

Like I said earlier, order matters.  The order that the data is written into the Parcel is the order that it must be read back out.  No deviations.  Parcel objects can contain other Parcelable objects easily. There are complications when it comes to complex types like Maps.  The maps have to be broken down into component pieces in order to be stored.  It is then reassembled in the constructor.

I also a created some boilerplate code that is common to Parcelable objects.  It gives users of the Parcelable data more options for using the object.  That boilerplate is :

public static final Parcelable.Creator<Note> CREATOR = new Parcelable.Creator<Note>() {
        @Override
        public Note createFromParcel(final Parcel source) {
            return new Note(source);
        }


        @Override
        public Note[] newArray(final int size) {
            return new Note[0];
        }
    };



So Why Parcel?

There are a lot reasons, but the biggest is too allow the objects state to be stored in a Bundle.  This means that a copy of the object can be passed between Activities, Fragments, Services, etc via the Intent Bundle.  It also lets the object be stored in the save instance bundle in Activities and Fragments.  This will make things a lot easier when I get to the UI levels.

The Caveat

If the parcel is going to be passed in an Intent, you must watch the size carefully. It cannot exceed 1MB.  If it goes over, your app can hang and then crash.  This is due to a annoying size constraint in the OS.

Luckily, the size constraint problem does not exist if the parcel is going into a save state bundle to preserve the state of an Activity or Fragment.

I didn't mention in the database entry, but the same size issue can exist there as well.  It will let you store blobs over 1MB, but good luck on getting it back out because the Cursor class does not like to exceed that limit.

NotesManager

This is a pretty straightforward class. It is the way I will create Note objects from the database.  It provides methods to get all of the notes, or get the notes with certain colors, or get a single note.  That is pretty much it.

Commits

I did some of the development for these changes late at night after a busy week so it is broken into two commits:

Wednesday, February 11, 2015

MyNotes: Creating the Data Access Layer in the Common Library.

To start work on the data access layer I need to setup some common classes to make it easier for this app and future apps.

Manage the UI Thread

Since my access layer will be available to any area of the app, I have to worry about my threads more than I would if I was using a ContentProvider.

Prior to Lollipop, Android apps are single threaded by default.   This is often called the UI thread. However, Lollipop added a RenderThread that handles animations to make the smoother.  By default, your app will still perform all work (except animations) on the UI thread unless told to so otherwise. 

In either case, accessing the database on the UI thread is very bad.  You should never do any potentially long-lived operation on the UI thread because it will degrade your UI performance and might lead to an Android Not Responding error.

In order to prevent this, I created a class, ThreadCheck, that validates which thread is running during a method.  I will add a check to each method that access the database to verify that it is not running on the UI thread.  If it is on the UI thread, it will throw a special exception, ThreadException. This will drive out any threading issues quickly and early on.  

To check if a method is called on the UI thread is very easy.  You just have to call Looper.getMainLooper().getThread().  The MainLooper is always associated with the main thread, which happens to always be the UI thread.

Common Ways to Update Items in the Database

Now I need a common approach to updating items in the database.  This is done with an interface, DatabaseStorable, and a Utility class, DatabaseUtilities.  The DatabaseStorable contains methods to store or remove a single item from the database.  Any item that is backed by the database can implement it to update its individual content.

That interface is then used by the DatabaseUtilities to do saves/deletes on a single item or to do them in bulk in a single transaction.

Commits

I did some of the development for these changes late at night after a busy week so it is broken into two commits:

Wednesday, February 4, 2015

MyNotes: How to get the data?

Now that I have a new and improved database layer.  Now I have to figure out my primary way to access it.  My choice comes down to two options:

  • ContentProviders
  • Not ContentProviders
If you aren't familiar with ContentProviders, you can read up on them here.  They are a nice way to expose the database for manipulation, but the main use is to expose Data for manipulation by other apps.  

Since I only have a single table, the ContentProvider would map really nicely.  On the other hand, ContentProviders can be a pain to setup and get working correctly.  They can also be hard for new developers to understand.

After weighing my options, my choice came down to the decision to not expose my data beyond my app.  So I went with the non-ContentProvider option.  This doesn't preclude me from adding one in later if I change my mind about exposing the data and it simplifies some of my setup.

With that decision, I have to setup my data access layer...



Wednesday, January 28, 2015

MyNotes: Updating the Database

I am going to keep my database schema the same.


I don't really like the names of the database, table, or columns.  However, I don't want to change the schema since the app has already been released and the schema will still work for the expected functionality.  It sucks, but that is one of the trade-offs that have to be made sometimes.

The same cannot be said for the code that defines and manages the database.  That is going to get a major overhaul.

Step 1: Updating the Common Library

My first order of business is create some classes in the CommonLibrary to help with future database needs.  The classes are :

  1. CommonTerms - An interface that defines common terms and phrases used to define and create database schemas. Basically, just a constants class.
  2. DatabaseModel - An interface that defines the common methods for defining and setting up a database.  This will let me use the DatabaseHelper for my database management.
  3. DatabaseHelper - A helper class to manage the databases for an app.
These classes are located under the com.fsk.common.database folder.  Take a peek at them when you have a chance to understand what they are.

Step 2: Adding the CommonLibrary Unit Tests

The first step here is to update the manifest to let me build an apk.  The library manifest suffers the same issue the MyNotes manifest has with respect to the duplicate file problem.  That was discussed briefly in the Setting Up the Unit Tests entry.

Basically, I just added the following entry to the Android block in the CommonLibary gradle file:

    //Required to prevent duplicate files during the build due to the imported libraries.
    packagingOptions {
        exclude 'LICENSE.txt'
        exclude 'META-INF/DEPENDENCIES.txt'
        exclude 'META-INF/LICENSE.txt'
        exclude 'META-INF/NOTICE.txt'
        exclude 'META-INF/NOTICE'
        exclude 'META-INF/LICENSE'
        exclude 'META-INF/DEPENDENCIES'
        exclude 'META-INF/notice.txt'
        exclude 'META-INF/license.txt'
        exclude 'META-INF/dependencies.txt'
        exclude 'META-INF/LGPL2.1'
    }

I also added a block to the same gradle file to support java 7:

    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_7
        targetCompatibility JavaVersion.VERSION_1_7
    }

Next, I created the first unit test for the DatabaseHelper file.  I don't need unit tests for the other two files so we can move onto the MyNotes module changes.

Step 3: Updating the MyNotes Module

These changes all resolve around removing or deprecating the old database code and replacing it with the new database setup.

In the original code, the database setup/management was messy and hard to follow because it was split between three main files: DatabaseKeys, NotesTableManager, and DatabaseManager.  Unfortunately, some of the Database management leaked into other classes, which I am going to deal with in the next entry.

For the new design, I moved the entire database schema into the MyNotesDatabaseModel class (com.fsk.mynotes.data.database).  This class implements the CommonLibrary's Database Interface so that I can just use the DatabaseHelper (or an extension of it) to manage my database setup and access.

The original code also has some issues with the timing and survival of the database.  Unfortunately, I created the database in the Activities which meant that sometimes the database went down and there was a lag between the activity starting and the database being ready.

This time around I want to centralize my database initialization into one area that will guarantee it stays alive for the entire application life cycle.  I also want to be sure it is alive before the first activity creates. Luckily, I can do this by creating a custom Application class : MyNotesApplication.

I have to modify the manifest to tell it to use the MyNotesApplication class instead of the system default one. this is done by adding the following line to the manifest application element :

<application
  android:name=".MyNotesApplication"
  android:label="@string/app_name"
  android:icon="@drawable/my_icon">
...
</application>

In MyNotesApplication, I extended the OnCreate method to create the database and initialize the DatabaseHelper with it.

My last step is to do a little cleanup and preparation for removing the NoteTableManager.  The NoteTableManager is going to be removed soon, but it is too big of task for this entry.  For now, I am just going to deprecate and modify it internally to use the DatabaseHelper.

Commits


That is it for this round of mods.  The commit can be found at https://github.com/fsk-software/mynotes/commit/9e368c82f2b5fc7291e2a48127e35ea1da26c7b5


*Technically, the code will run at this point, but it looks bad and is only semi-functional.  The next few sets of changes will probably render it inoperable for a while.