What happens when everything is asynchronous...

So, I don't know if some people just had a conversation one day on how to deal with the problems inherent in asynchronous systems, but if they did, I think Reactive (Rx) is what likely came about as a result of that conversation. For anyone who's had to write any amount of Javascript, you've likely had to deal with the problems of "Spaghetti code." You make calls to JQuery, JQuery returns a result, and then you need to update the page with that new result. Sounds easy enough, but when almost your entire page is meant to be AJAX-enabled, you very quickly have issues of trying to figure out what the current state of the page is, updating it, etc. The "spaghetti" moniker is from the image of code reaching across the global state, and it being impossible to untangle who modifies what and when. Technologies like Angular-js, Knockout-js, and Backbone.js attempt to remedy these issues through numerous techniques.

The Reactive Extensions are a set of tools built for the .NET platform that allow you to define your interactions as a series of transformations applied to data. This way, each transformation is capable of responding to errors, and can "react" to the data of the transformation before it. The entire structure is incredibly similar to functional programming (which I'm totally okay with). And if you've worked with [Monads](http://en.wikipedia.org/wiki/Monad(functionalprogramming) before, you'll feel right at home. If you have no idea what Monads are, read this blog post. It's by far and above the best explanation I've ever heard.

So, of course Reactive is not just a .NET thing, because otherwise I wouldn't be writing this post. Netflix is at work implementing the same API on Java via RxJava, and I think it may just be the best thing to happen to the JVM behind it's creation. Lambdas are a close third.

A time for experimenting...

I recently finished the first feature-level functionality of MinimalBible. As of a couple weeks ago, you can now download and remove Bibles and other books from the device. Given that I had to learn a lot to make it to this point (Dependency injection, Android Studio/Gradle, etc.) I think that's definitely an accomplishment. But, before moving on the next stage, I wanted to take a moment and play around with the existing code base that I had. Two projects that I thought were really cool that I wanted to give a shot were RxJava (and retrolambda), and Grooid.

Long story short, I don't think Grooid is ready for the big time. It looks cool, but I wasn't able to get it set up. While it could definitely shorten development time (the language itself looks fantastic) in the mean time there are plenty of libraries available for Android that can similarly help.

Rx on the other hand, has been a blast to use. It will definitely force you to think differently (unless you come from a functional-language world - Ruby/Python count too), but it has the potential to make life much easier.

Going Reactive

The idea behind Rx is that you have data (Observables), transformations on that data, and then Subscribers doing things with that data. A real-world example for MinimalBible is that I have a list of Installer objects (that are able to download/install Bibles) and I want to do things with it. An example might be something like this (for sake of brevity, I'm using lambda notation):

List<Book> finalList = new ArrayList<Book>();
   
Observable.from(getAllInstallers()) // Get the installers
    .map((Installer i) -> i.getBooks()) // Get the books from each installer
    .reduce((new ArrayList<Book>(), accumulator, bookList) -> accumulator.addAll(bookList)) // Compile the overall list
    .subscribe((List bookList) -> displayList(bookList)) // Display all books in one shot

That's pretty cool, right? I have a group of installers to start with, and I end up with all the books I can install. But, those installers get their information from the Internet, and Android doesn't let me do networking on the main thread. Let's fix that.

List<Book> finalList = new ArrayList<Book>();
   
Observable.from(getAllInstallers())
    .map((Installer i) -> i.getBooks())
    .subscribeOn(Schedulers.io()) // Do the operations on a different thread
    .reduce((new ArrayList<Book>(), accumulator, bookList) -> accumulator.addAll(bookList))
    .subscribe((List bookList) -> finalList.addAll(bookList))

It took me exactly one line of code to handle doing everything asynchronously. No AsyncTask listeners, callbacks, nothing. Just pure clean [asynchronicity](http://en.wikipedia.org/wiki/Synchronicity(ThePolice_album)).

Now, trying to run this code can be pretty expensive, but the results don't change that often. What Rx allows me to do is cache the results so that anyone who needs them later is automatically provided with them.

List<Book> finalList = new ArrayList<Book>();
   
Observable.from(getAllInstallers())
    .map((Installer i) -> i.getBooks())
    .reduce((new ArrayList<Book>(), accumulator, bookList) -> accumulator.addAll(bookList))
    .cache() // Subsequent calls will complete immediately
    .subscribeOn(Schedulers.io())
    .subscribe((List bookList) -> finalList.addAll(bookList))

Now, the list of books will only be compiled once and anyone can use it. In a very clean, coherent manner, I've accomplished an incredible amount. That being said, if I want to use this list on the UI side, I somehow need to get on to the main thread. Not too complicated:

List<Book> finalList = new ArrayList<Book>();
   
Observable.from(getAllInstallers())
    .map((Installer i) -> i.getBooks())
    .reduce((new ArrayList<Book>(), accumulator, bookList) -> accumulator.addAll(bookList))
    .cache()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedules.mainThread()) // Receive the objects on the UI thread
    .subscribe((List bookList) -> finalList.addAll(bookList))

So I started with a list of installers, I now have a list of books. In between the list is cached, and Android is happy about networking done off the main thread, UI manipulation on the main thread. All in 8 lines of code.

There are two important downsides to this though:

Inner-class shenanigans

The first (obvious) downside to using RxJava is that you are trying to make a functional-style library work in a imperative/procedural language. This means that you will have a large number of inner classes that are created in order to accomplish your goals. I kind of cheated a bit to get the lines of code down to 8 above - in actual code, it would be closer to 20 counting all of the Anonymous classes made. That being said, 20 lines of code is not a lot for the amount of functionality it generates.

So I don't think inner classes are a big issue for two reasons. First, and this is not a great reason, is that you can enable lambdas on Java prior to Java 8 via the retrolambda library. Unfortunately, this requires that everyone who works on your project uses Java 8.

The second reason is that Android Studio will "fold" the code in the editor for you. So even though you write out those classes once, you can hide them to keep everything from being ugly. Quick, simple, easy, and it's built into the system.

Reactive Everywhere

The second problem with using RxJava, or reactive in general, is that there really isn't a good way to do things half-heartedly. Technically you can, but it's not simple. In my experience, once you start with reactive programming, you want everything to be reactive. Simply put, it is nigh on impossible to port a significant code base to using a reactive style, and so I would not recommend this for existing projects.

That being said, it's doable - you can use BlockingObservables to make your API play nice, and then port your high-level code to taking advantage of the Observables. From what I did though, it was a lot easier (and a lot more fun) to just convert everything all at once.

Conclusion

So, working with RxJava has been a fantastic experience, and I absolutely love it. Shout out to the guys at Netflix for making this possible, it's certainly made development a lot easier for me. Looking forward to continuing to use Rx, and I'll be writing more soon!

What we'll think was a terrible idea in 5 years

So, I've been working with some legacy code as of recently, and becoming particularly frustrated with it (think Spring 2.5 legacy - late 2007). Trying to wade through multiple files-worth of XML and private constructors were the two things that really stuck out as "I'm not sure why this was ever considered a good idea."

XML configuration: So, the problem is that you have classes with multiple dependencies, and if you just look at the java code, you have no idea why anything should work. You never instantiate the dependencies, you never do anything to touch the fields, they just work. Dependency injection is a great idea, don't get me wrong, but it's confusing to have to read through (and write!) XML to "wire up" the application. Most specifically, writing the XML is simply painful. Plus, if you have people who don't understand the Spring way of doing things (like I did), it's borderline impossible to figure out what's going on. Plus, how often does the configuration actually change? I can almost guarantee that it doesn't change more often than you compile the project (and if it does, we need to have another talk). And when it does change, do you always remember to edit the XML?

Private Constructors: I don't have as much to say about these, but I do think they need to be pointed out as a "this just isn't a good idea." I think the idea of private constructors was to force you into using the dependency injection framework, rather than having hard-coded references everywhere. This way, you force people to abide by the framework in place.

The initial problem is that you destroy the possibility of unit testing, or at least make it incredibly complicated. Gone is the ability for Test classes to instantiate the objects they're trying to test. Instead, you need to use some reflection hacks to bypass the intentions of the person who wrote the class in the first place. It just doesn't make sense to write something one way, and then test it a completely different way.

But enough of the legacy problems. Here's what I see being problematic five years from now:

Annotations

Annotations are fantastic, don't get me wrong. Currently, there are a significant number of annotations in use for writing MinimalBible. The problem is, I think we're using too many. Here's a sampling of some Android libraries using Annotations:

• Dagger - An annotation-based dependency injection library for Android. Designed to also work on Java, and be small and light. Actively used and developed by Square and Google.
• Butterknife - An annotation-based view injection library for Android.
• RoboGuice - An annotation-based dependency injection library designed to imitate Google Guice
• Dart - Annotation-based injection of "Extras" - both for [Activity extras](http://developer.android.com/reference/android/content/Intent.html#getExtras() and [Fragment arguments](http://developer.android.com/reference/android/app/Fragment.html#getArguments()
• Michelangelo - Annotation-based layout inflation injection. Even comes with support for Butterknife, and can call that simultaneously.
• Flow & Mortar - Annotation-based libraries for arranging the "screens" of your application, injecting them with Dagger (note: Mortar actually depends on Dagger), and then managing the back-stack
• And finally: Android Annotations - A library to do annotation-based injection of everything. Includes POJO, layout, REST, Activity, Fragment, SharedPreference, Event, and more injections. The list is available here.

I guess what I'm trying to say is: If you don't want to code it, chances are that someone has created an annotation-based library for it. This isn't entirely fair, but you have everything from an annotation framework (Android Annotations) down to simple dependency injection (Dagger) all using Annotations.

Now, the implications of all this: 1. It's that much less code that you have to write yourself. That's awesome. 2. It can enforce good practice where things are lacking (I have Android Annotations Fragment builder in mind) 3. Most of the work is done at compile-time. This means that you don't have run-time reflection running, which is also cool. Dagger even validates the ObjectGraph at compile time, meaning it will catch errors before you run anything.

These are all good things. The problem though, is this: using annotations forces you to understand what the framework does and doesn't accomplish for you.

The worst offender of this is Android Annotations. When you use one of its annotations on a class, you must remember to not use the original class, but use the generated class in the rest of your code. So even though you may have something like this:

@EActivity(R.layout.my_layout)
class MyActivity extends Activity {
    // ...
}

You use it like this:

// ...
    Intent i = new Intent(MyActivity_.class);
    // ...

The next problem is knowing when the activity injects its resources (at which point you have an @AfterInject annotation), so on and so forth. The penalty for using annotations to write code for you is that you're locked in to understanding the code another programmer wrote.

Libraries like Dagger and Butterknife are better about this - you only have to call objectGraph.inject(this) or Butterknife.inject(this), but I've frequently run into strange issues because I forgot to call those.

Fragmentation

The other problem I see with Android is that there are a bunch of different ways to accomplish the same thing. You can go with the vanilla SDK, or you can mix in some basic libraries like Dagger, or you can add some more in-depth tooling with Flow/Mortar, etc. If you want to consume REST data, there's Retrofit, though some prefer Volley. The (fantastic) Android Weekly Newsletter has a section each week on new libraries available to do cool new things.

All of this (in my experience) leads to a bit of framework paralysis, when you try to mix in different libraries to see how they work. There's a lot of fancy cool new technology; there's also a lot of projects with a completely unproven track record waiting to distract you. Which is not to say that they're useless - but I did spend significant time trying to get things like Robolectric to work before scrapping it.

What I'd appreciate, and what will simply take time, is to see which projects are really in it for the long haul and do actually produce significant developer productivity (without sacrificing time to bring other developers up to speed).

Where do we go from here?

So the question of the hour then is: so what?

A couple years down the line, I think we'll end up looking at some of the annotation-heavy code that we've written and have no idea what's going on or how we're supposed to use it. Yes, the code is more descriptive and much smaller in size. However, it's hard to remember how 7 different annotation libraries expect you to use them, or why you're not getting the features working correctly (forgot to call a .inject() method, or use MyClass_ instead of MyClass).

I think the way forward might be less reliance on annotations to handle the code we don't want to write, and relying more on well-designed API's. So, I think things like Dagger and Butterknife will stay - the penalty for using them is a couple lines of extra code, for a huge benefit. Things like Android Annotations and RoboGuice I'm guessing will largely disappear simply because it's hard to remember how they're supposed to be used, or nobody will be able to contribute to your codebase because they don't understand the framework.

I also think there's a huge potential in projects like RxJava and Groovy on Android. With RxJava you get a very well put-together API allowing you to define behavior and manage the complexity of a largely asynchronous application. It will definitely require you to structure an application differently, and think differently (there's a learning curve to a functional style), but the flexibility is incredible. Also, I haven't used Groovy at all yet (I'm planning to try and demo it a bit and see about writing MinimalBible with it), but so far it seems a lot like Python. I've had very good experience with mixed-paradigm (partly functional, partly object-oriented) languages (mostly Python) in the past, and it allowed me to write clean code quickly. I'm all down for that.

The tradeoff in using things like these is that they are bleeding-edge new. RxJava specifically is going to be a pain due to inner-class shenanigans. I don't think there was any other way that it could've been done, but that doesn't mean that 8 levels of nesting is any more readable. Unless you use lambdas, Rx code in Android is going to be ugly. And Groovy was announced less than a week ago. There's a huge risk of the project dying off quickly and leaving you with a deprecated app. Given Apple pioneering the Swift language, I think Groovy will be a strong competitor, but that all remains to be seen.

So this post definitely looks far into the future, but that's where I think the scene will be shifting. I'm going to give RxJava (likely with retrolambda support) and Grooid (Groovy Android? I think they need a better name) a try, so I'll write more posts on those coming up.

Unless you wanted to know a lot

So, I ran into some interesting issues with ListView caching in trying to get everything implemented. I'm going to document them here, because I imagine I'm not the only one to struggle with these, and it could save people a lot of time and frustration later. Hope this is helpful!

Why does HolderView work anyway?

One of the first questions I had about HolderView was why it actually worked. I mean, if classes are not static any more, they're being garbage collected when they go out of scope. And if they're being garbage collected, I'll have to keep on re-creating them, and I'm back to having the issues the ViewHolder and HolderView were designed to solve in the first place.

Well, to understand why HolderView works you need to know a bit about how the ListView operates in the backend. Long story short, when you scroll through elements, the ListView recycles things from the top of the screen down to the bottom. Think of it like a climbing wall treadmill. This way, you don't have to store every single element of the list, and you don't have to create them either.

So, the getView() method is where the actual view inflation of a ListView happens. The ListView supplies us with one of the Views it's already had created for us, so we can get right to work. Now, the ListView will display whatever we return from getView(). So there's nothing against us returning a completely different view, unrelated to the one the ListView gave us originally. And that's why this works.

Because the HolderView returns it's own custom View object, rather than using the View that the ListView gave us, we can guarantee attributes like the bind() method will actually exist when cast to/from View/ItemView. This way, when the views are recycled by the ListView, it's actually giving us the custom HolderView views, rather than generic views. All we have to do is re-bind() the HolderView, and we're good to go. Fancy, no?

Switching to ViewHolder from HolderView

Now, let's say that for whatever reason, you want to switch to a ViewHolder pattern from a HolderView. Not too challenging, but make sure you're thorough about it. When I tried to do this I wasn't so thorough, and so instead of caching only references to the ViewHolder, I just cached the HolderView instead. It led to the same bugs I was seeing prior to the switch (of course) and so I was a bit frustrated. I also started getting some NullPointerException errors, which was really strange. Understanding these errors is incredibly technical, so stick with me.

Part of the reason for using HolderView at all is memory management. The standard ViewHolder pattern uses static classes, meaning they just stay in memory until the actual application is killed. The standard HolderView is not a static class, which means that when it goes out of scope, it is destroyed.

Well, in the implementation linked to above, I stored the HolderView object into the cache in a kind of hybrid approach. I kept the memory management of HolderView while using the getTag() style of ViewHolder caching. And to be honest, it wasn't until trying to write this post (4 days after the fact) that I understand why it was broken. Since the HolderView gets destroyed when it goes out of scope (for example, when you scroll past the view), the tag of the View will be null when you scroll anywhere. Thus, I had to modify the code so that I check both the view and tag, rather than just the view:

BookListAdapter.java

// Note that convertView==null will short-circuit the getTag() check
if (convertView == null || convertView.getTag() == null)

View recycling leaves views partially initialized

Now, this last piece is a very nasty issue I had seen ever since adding the initial ProgressWheel (took 3 days to get it fixed). What was happening was that when I showed the ProgressWheel, individual rows in the ListView would have the wheel displayed when I had never clicked them!

Well, there wasn't ever really an a-HA! moment in trying to fix this, but by the time I had understood enough of how the ListView actually works to debug some of the issues above, this one started to make sense.

What happens is that the views get re-cycled when you scroll down, but the bind method never explicitly set the state, or inflated the layout. Thus, the view that was given already had the ProgressWheel showing even if the item hadn't actually clicked on it. After the fix above, everything started working again.

Conclusion

I hope that answers any questions you may have about the ListView. I think the component is challenging to use, and if you don't really understand the optimizations it does, things get very strange and confusing very quickly. But now you know enough of how everything is laid out in memory to be efficient yourself! Go forward and write great code!

Further semantics and practices

Well, I personally think that ListViews are some of the most complicated Views in the Android ecosystem. So, I'm going to make two blog posts talking about them! The first is the ViewHolder vs. HolderView pattern, the second will be some nasty issues on ViewHolder caching I ran into. So, without further ado:

What is ViewHolder?

One of the biggest problems in displaying a ListView in Android is just how inefficient it is. Specifically, calling findViewById() is really expensive, so we try and do everything we can to avoid it. That's why the ViewHolder pattern was invented. The basic principle is:

• The ListView gives us a View object when it requests a view from the adapter with getView()

• If that View is null, we need to inflate a new one. Then, cache it in the view with setTag()

• If that View is not null, we can use the View's getTag() method to retrieve a cached version of it (since it's been inflated prior)

So, what we actually cache in that sequence above is the ViewHolder object. Basically, it just stores a static reference to the inflated fields so we don't have to reinflate everything. Then, the actual adapter is responsible for updating the inflated View using the references in the ViewHolder.

If you have any other questions on ViewHolder, check out this page because the author did an amazing job of explaining everything.

What is HolderView?

The HolderView pattern has the same basic principle. We want to make as few calls to findViewById() as possible. There's a big difference though: The HolderView is responsible for its own presentation.

• The ViewHolder just stores a reference to the View elements that are being inflated (by the adapter). The HolderView actually inflates itself.
• The Adapter driving the ListView is responsible for the presentation of data in each element. The HolderView handles it's own presentation through a bind() or some similar method.
• The ViewHolder is a static class, and so can't be garbage collected. Each ViewHolder takes up incredibly little space, but if you're OCD, you still have unused objects you can't remove from memory. The HolderView is allowed to go out of scope and be deconstructed.

There are a few downsides too:

• The ViewHolder pattern is nearly ubiquitous on the Internet. Honestly, if I wasn't forced to use ViewHolder by Android Annotations, I would never have known it exists (I can't justify it, but I'm guessing a similar pattern is used on iOS development, feel free to let me know).
• The HolderView likely means you'll have another class file. You don't necessarily have to do this, but it's usually better that way.
• The HolderView is used in place of the view that the ListView is actually trying to inflate. That is, the ViewHolder just caches references to elements inside the View. The HolderView actually replaces the View (which means typecasting).

By any means, the HolderView pattern kind of acts like an MVC layer for your views. You can check out my original implementation over here.

So the biggest benefit of using a HolderView pattern is that you can move the presentation logic of a View to its own class, and thus do some pretty cool stuff (for example, Dagger won't let you inject non-static inner classes).

So those are the two patterns! Coming up next: more than you ever wanted to know about ListView caching.

Because everyone needs more dependency injection

Clarification: If you work in enterprise Java, you probably don't. But most Android projects I've seen seem like they could benefit.

So, I've covered Dagger previously in trying to evaluate a DI framework for MinimalBible. I did settle on using Dagger + ButterKnife for my injections, as Dagger provides compile-time validation, and I'm not forced into someone else's lifecycle (like Android Annotations).

Getting Dagger set up then was a bit of an interesting experience. Maybe I just wasn't looking hard enough (or scanning the docs too quickly), but I was totally unaware of how to get the code running. When I initially added my @Inject handlers, I kept getting the feared NPE. This was the result of me fundamentally misunderstanding how Dagger as a whole works, so let me see if I can't break it down.

Compile Time (Modules)

Like I said above, part of the benefit of using Dagger is compile-time validation. Well, in order to do that, you have to declare the entire structure of your dependency injection. Don't worry, it's not actually that complicated. To do so, we create @Module annotated classes.

BasicModule.java

@Module (
    injects = MyApp.class;
)
public class MyModule{}

We've now got our Module! This module is responsible for handling the dependencies of MyApp, and that's all it cares about.

MyApp.java

public class MyApp {
    @Inject MyObject object;
   
    public static void main(String[] args) {
        System.out.println(object);
    }
}

And now the basic app is set up. We'll get to the Android implementation later. Let's define MyObject, and then we'll be ready to run!

MyObject.java

public class MyObject {
    @Override
    public String toString() {
        return "I'm alive!";
    }
}

So if you try and run MyApp right now, you'll get a NPE. But wasn't Dagger supposed to do the dependency injection for us? And the fact that we were even able to run this at all means that the validation must have run correctly...

Actually, that second point is correct. The validation has run at this point, and you're good to go. Because MyObjection has a no-arg constructor, Dagger is able to do everything it needs to set up dependency injection. The problem is, we need to actually set up the object graph and inject ourselves.

Generating the object graph is pretty simple - we just need to add a couple lines to MyApp so that Dagger builds in memory all the dependencies, and then can inject them.

MyApp.java

// We only need to modify `main()`
    public static void main(String[] args) {
        ObjectGraph objGraph = ObjectGraph.create(BasicModule.class);
        objGraph.inject(this);
        System.out.println(object);
    }

Huzzah! It's now working. Now here is where I started messing myself up. Let's say I wanted to use injection on another class. I'll modify the @Module to inject the next class.

BasicModule.java

@Module (
    injects = {MyApp.class, AnotherClass.class};
)

And the next class:

AnotherClass.java

public class Another Class() {
    @Inject MyObject object;
   
    public String toString() {
        return object;
    }
}

And update MyApp to use the new class:

MyApp.java

// Again, only need to modify `main()`
    public static void main(String[] args) {
        ObjectGraph objGraph = ObjectGraph.create(BasicModule.class);
        objGraph.inject(this);
        System.out.println(new AnotherClass());
    }

So now MyApp delegates the object.toString() call to AnotherClass. Well, trying to run this code will give you another NullPointerException. And this is where I misunderstood how Dagger actually works: When using Dagger, each class is responsible for injecting itself. This gives me flexibility over when in the lifecycle the injection happens, but I was originally under the impression that Dagger just auto-magically did injection for me.

Now, if each object handles its own injection, you'll remember that we had to construct an ObjectGraph for actually injecting. The ObjectGraph is relatively expensive to compute, so we don't want to keep rebuilding it every time we need to do injection (it doesn't change anyway, unless you're using [plus()](http://square.github.io/dagger/javadoc/dagger/ObjectGraph.html#plus(java.lang.Object...), in which case you likely don't need to be reading this article).

This is where the Android-specific stuff comes in. Long story short, the Application itself will stay around for as long as your application is running. Seems like a great place to store the ObjectGraph, as there are a lot of people who will need it.

I'm not going to go into the specifics, as it should be pretty obvious how to implement that given what I've outlined above (the way I did it was create the graph during onCreate(), and hold it as a reference inside the actual Application singleton).

Hope that gives you guys a better understanding of how to bootstrap your Dagger projects, it's definitely been awesome so far getting to use it!