IntelliJ Idea loses my test resources!

Are your tests failing because the resource files cannot be found? All of the sudden I hit this problem where a few of my tests started to fail for no reason. They were unable to locate the test resources I was using and at first this looked like an ancient issue. The old issue I had been familiar with before involved a clumsy work-around where you would hack your Gradle build to copy the test resources folder into the test classes folder. This still works but is not necessary. While the bug still exists the current work around is to click the Gradle sync button to resynchronize your project with the Gradle build. (I discovered the fix here.) This seems to work for me for now.

Test First, ask questions later…

I’ve been doing this test first thing for a while and now I’m in a position where I’m teaching others how to properly design code with this practice. I’m using it in my current project and I’ve also been hanging out in the code newbie community answering questions and confusing others. I usually respond to arbitrary questions with something hinting to the need to write test code. Here’s some typical exchanges I have throughout my week:

Them: “I can’t find the source to this null pointer exception…”

Me: “Can you isolate the issue with a failing test?”

Them: “My REST call is returning 403”

Me: “You know you can use Selenium in these scenarios to…”

Them: “I think this method is going to be slow.”

Me: “Do you have a failing performance test that indicates we should optimize here?”

Them: “You have a few dollars? I forgot to pack a lunch…”

Me: “I usually have a checklist I refer to prior to leaving the house for things like my employee badge, my lunch, etc. I highlight each item with a green bar prior to jumping in the car…”

My Kids: “Daddy, I don’t feel well. Can I stay home today?”

Me: “I thought I told you about going outside without proper test coverage! You probably got a regression of your symptoms from last week!”

Needless to say it’s a very important topic and always forefront on my mind. I lead with a test case. One person in particular drew an interest and asked a basic question about unit testing. Not paying attention to who it was I sorta went all out in my explanation without really understanding the level of expertise. Usually I get the typical response of, “You’re a bit extreme, bro! Calm down!” She was not intimidated at all. Instead, she was rather appreciative, which I’m thankful for because I really went full throttle in my response. I don’t really have an off switch when I get on that topic so I ended up writing roughly 2 pages of text before I came up for air. The question was something along the lines of, “how do I test my method that calculates my layout dimensions based on screen size?” My brain jumped immediately to “Test First!” without asking any pause to ask questions about the project, deadline, or backstory. In short, here’s what I attacked with. (Yes it was an unsolicited attack on an innocent bystander but I have no filter when it comes to these things):

See there are 3 levels of testing as I like to see it.
* Functional testing
* Integration testing
* Unit testing

Each level sits on top of the levels beneath it and has certain characteristics.

[8:36 AM]
Functional testing is the type testing where you test the app in its entirety end to end for broad functionality.

This involves testing the app in its deployed environment (device or emulator) and may or may not be automated by tools.

Integration testing is where you test two or more components of your app in isolation. In these types of tests you are exercising the integration points of particular components. I.e. does objectA integrate and play nicely with objectB?

A component may be represented by something as coarse grained as a library or module, or something as fine grained as an object or a method.

Again, you are primarily testing to make sure the pieces work together.

Unit testing the the final level of testing. It is here where you test specific objects and/or methods in isolation.

Why am I telling you all of this?

(I’m so glad you asked!)

The thing is that each level should focus primarily on a specific type of testing.

With functional tests you test broad functionality. These should be very coarse grain tests and not too specific. These tests are a way of asking, “does my app function as a whole?”

They run on device and as a result they are slow and cumbersome. These are the tests that normally make use of Espresso, UIAutomator, etc.

If you try to test too much or test too specific of a scenario you burn most of your effort using the wrong test for the job and the test will always be brittle and unreliable. (edited)

Integration testing is more fine grained. These tests may or may not run on the device. Once again, they exercise the integration points between components. These tests answer the question, “does my component integrate or play nicely with this other component?”

With these tests you can be slightly more specific but you should still focus on integration. they should be coarse grained in general but can be slightly more fine grained than the functional tests.

That brings me to Unit tests. These are where your fine grained test scenarios live. whenever you are trying to find/fix a very specific bug or exercise a very specific scenario you can rely on these to give you what you need. This is where you would focus primarily to get 100% coverage on the behavior of everything.

I use a vacuum cleaner analogy (which may or may not make sense to you)…

You know how the vacuum cleaner has the big apparatus that sucks the dirt from the floor but this big thing can’t get into all those hard to reach places like in the corner by the door and waaay under the couch towards the back? Well the vacuum cleaner companies usually supply you with attachments like the edger piece, which can get in the tight corners, and the extension piece that lets you vacuum high or in long narrow places.

In the same way your functional tests are like the big apparatus on the vacuum. It’s used for general testing, and broad strokes but there are places that it will not reach.

You should make use of the lower levels of testing for these more specific scenarios. (edited)

Whew, that was a long answer to your specific problem…

To your issue you wrote:
“To test my calculation I thought I’d set a value for display height and width and see if the method returns the expected LayoutParams… I tried different approaches and gave up in the end because all that mocking of Android dependencies” (edited)

You approach indicates that you are attempt to test something very specific at a higher level than necessary.

To test your calculation you should have a unit test which, when given a dimension returns an expected size.

This test can be written in pure Java without android dependencies, which means it can run outside of the container under the JVM using JUnit.

I don’t know if you’ve ever used regular JUnit in your Android project but it something to look into.

What you can do is create a Java module in your project and place your Junit test code there along with the classes that these tests exercise.

I wasn’t finished there. I came back (after coffee) with an example:

Here’s an example. Say you have an app with a single button which, when pressed, makes a network call to a server to increment a counter.

The functional test would load the entire app on the device, deploy the server code to the cloud and launch it, start the app on the device and send a button press.

It would then verify the counter reflected on a text label on the view has been incremented.

(I forgot to mention that there is a textView in the example that shows the value of the counter.)

There are actually quite a few pieces in this seemingly basic example that you should consider

The initial value of the counter needs to be read from the server.

this involves an object that can make a network call to the server, an actual server, an actual network… etc.

A functional test would load two or more of these pieces and test them together.

you could write a test that says:
Given a CounterRequestor
when I ask for the count
then I should get some number

which in Java/JUnit equates to:
CounterRequestor requestor;

@test public void shouldReceiveANumber() {
Integer initialCount = requestor.getInitialCount();
assertEquals(initialCount, 0);

This test requires an INTERFACE not a concrete class because you are DESIGNING (the 2nd “D” in TDD) the intention of the app.

The interface would be implemented by some concrete object, say InternetServerCounterRequestor which you could instantiate inside your test in setup…

(also I would write the test before defining the Interface so I could get a feel for how I would use it. I usually use Intellij/AndroidStudio’s Alt+Enter combo to generate the interface after I’ve written a test that shows how I’d use it.) (edited)

then to run this integration test you would need only the CounterRequestor defined, the InternetServerCounterRequestor implementation and an actual running server.

That’s testing the InternetServerCounterRequestor the internet connectivity in your house or office, and the server in isolation without involving your app.

You could then write more tests the specify how you would interact with the count service running over the internet.

The important reason why you would use an interface here is because you want to be able to test pieces in isolation. So now let’s say you want to test the Activity without the internet and without the server deployed…

You could write a test that uses Roboelectric or something to load just the activity and mock the CounterRequestor.

in such a test you could test:
Given a MainActivity with a textField
When it is created
Then it should ask the CounterRequestor for the initial count

you could also test:
Given a MainActivity with a textField
And a mocked CounterRequestor
When it asks the CounterRequestor for an initial count
And the CounterRequestor returns the number 77
Then the textField should be equal to 77

There are a few other moving pieces here as well…

you have the basic button you could write tests for

Given a MainActivity with a button
And a mocked CounterRequestor
When I press the button
Then my mocked CounterRequestor should be asked to send a counter update message

these are all integration tests that test pieces of your solution in isolation

You could probably see a useful pattern here with this example…

The fact that we used an interface allows us to easily vary what/how the counter is implemented…

you could implement a PeerToPeer counter that uses bluetooth instead of HTTP to maintain a counter across devices.

because nowhere in your test code do we specify HTTP, URLs or anything related the change is localized to the implementation of the CounterRequestor

so then this brings me to unit testing…

going through this example (which I am honestly making up as I go along here) you will find that making the network request to the server to get a count might involve XML or JSON parsing…

because usually a web service would return something like:

instead of just

so then you might find yourself needing to do something like parse the response which gets into the micro level of “I have this JSON object with a few keys. I want to design an object that can read the initialCount key and give me its value,

you could do this with a unit test.

There is more where that came from but the above explanation should paint a rough picture of how I see the world. The person I was helping is a developer named Alina from Sweden, I believe. She maintains an interesting blog and has an even more interesting backstory coming from marketing into coding.

P2P Git

Have you ever had your version control server go down just when you were ready to commit some important changes for your team? Did you feel stuck, blocked, or hopeless? This happened to me only yesterday but luckily we use Git for version control. Git is a decentralized version control system which means that you don’t actually need a central host to continue working. While lots of people already know that you can commit, merge, and branch locally without a network connection you may be surprised to know that you can also push and pull changes while the central Github repository is offline using a Peer 2 Peer connection.

Last night I wanted to push changes from my iMac in the office so I could pull them with my Macbook and work from home. Our internal Github repository went offline and I went to Google to find a little trick I’d used in the past. Setting an alias to run a local git server is as simple as:
git config --global alias.serve "daemon --verbose --export-all --base-path=.git --reuseaddr --strict-paths .git/"

This creates a git command, serve, that you can run from the command line to launch a local Git server over your current repository.

git serve

I ran this command on my iMac and got output saying something like, “Ready to rumble“. My local git server was ready to accept connections from other computers. On my Macbook I was then able to pull the commits I had just made on the iMac using it’s IP address with the git pull command:

git pull git:// feature/newWidget

The “feature/newWidget” was the name of the branch I was pulling. Some output text on the Mac confirmed a remote connection happened and the new commits magically showed up on my Macbook Pro. This same idea could apply if I needed to share my latest branch with a coworker before heading home. I could commit to a feature branch on my iMac and then say, “pull the changes from my iMac at”. My coworker could run a similar git pull and see all of the commits appear in their local repository. The alias I used above is read only, however you could also set it up so that coworkers have write access and can push directly to your local git repo as they complete work.

So the next time your internal version control host goes offline don’t sweat it! Run a local git server and share away! I haven’t tried it but I believe this might work with a git-svn setup as well.

How To Suck At Programming #912 – Shave a yak

You started out trying to create a simple Android app when you realized you needed a third party component. You went off to download the component and realized it needed a fix. Since it’s open source you decide to fix it yourself cause, hey, you write code right? To perform the fix you need work on Linux while your laptop has a illuminated fruit on its lid. After you install Virtualbox so that you can install Linux so that you can fix the 3rd party component so you can complete your project you realize you need to download the actual Linux appliance which sends you to Google where you find a link to a torrent. Your search for the best torrent client for OSX is successful and you manage to download the VM which comes in the form of a “.7z” file that your laptop can’t open. You then search Google for “best 7zip client OS X”. Does this sound like you? Hi, I’m Cliff. You’re here because you shave yaks for a living. I’m here because I need to introduce you to my pet Fuzzy, an animal I recently acquired with these two large horn things on the sides of his head. You thought you had a career as a software engineer but things took a tragic turn where people line up for miles at your door with these heavy hairy four-footed mammals each of which is overdue for a haircut.

What is Yak shaving?
Before you go on yet another Google search for the term allow me to elaborate. Yak shaving is the meta-work you encounter in your day job. It’s the work you need to do in order to begin the work you intend to do. It can often times nest itself in other meta-work. In other words, it can be the work you need to do in order to begin the meta-work you need to do in order to start the work you intend to do. This nesting can become several layers deep which is, I guess, why we call it yak shaving. If you’ve ever seen one of these animals you can probably imagine how an initial shave would only uncover the next layer of hair.

When do you stop shaving?
I’d written about yak shaving in the past but the question begs, “when does one stop shaving these animals?” It’s a valid question and rather interesting dilemma because you can easily find yourself unable to tear away from these animals after pulling out the clippers. It’s tough because you get attached… and who wants to leave poor Fuzzy half shaven? His backside is all exposed but he has all this hair up front! Professionally you may find yourself several layers deep in a problem before you realize you’re shaving a yak. At that point you feel you may have crossed the point of no return. This is the point where it costs more to start over with a different approach than it does to complete the task. You want to recognize the situation before it gets that far and put away the clippers! Stop shaving the moment you get a yak!

But how do you recognize a yak?
Sometimes you start with an innocent problem and a naive over-simplistic solution. The solution seems to fit the problem well and everything feels right. Take my example above. Use an open source component to plug a hole in your Android app. Ask yourself up front what happens if the component you’re using also has a hole. Be honest about what it takes to iterate (or change source, build, and test) the component. Sometimes these simple questions can reveal serious assumptions and flaws in your approach.

How do you stop shaving?
Honesty can help avoid the problem but transparency will pull the plug once you’ve started. That means you should constantly communicate your status and goals with your manager and team on every occasion. Managers have schedules and deadlines and don’t get along with these hairy beasts. They will often be the first to unplug your clippers. Divorce yourself from the problem. Early assumptions lead to attachment if only to validate the assumptions. You assume your idea will not introduce much overhead but as the overhead starts to increase you work harder just to validate your earlier (incorrect) assumptions. Being transparent about all the work you are involved in takes strength because you have to resist the urge to trivialize everything. However it allows you to see how far off track you’ve gone while inviting others to dig you out. Instead of saying, “alls I haff duh do is run a VM and do my my compilation there” be honest and say, “I need to download, install, and configure virtual box just to start the project since we have a Linux dependency”. This highlights how the simple act of running a VM has its own pre-requisites, downloading, installing, and configuring. It also gives you a moment to consider how each prerequisite can have its own prerequisites. Some of your colleagues may actually have simpler solutions or fixes for these pre-requisites. When things feel slow invite friends/coworkers.

Yak Hero
Some of us have become so good at yak shaving that we have achieved hero status. In other words we can plow through a series of meta-tasks and nested meta-tasks and make it appear as a single everyday task. We do this by cutting corners where possible. Over time you tend to get a feel for which corners you can cut without it being overly obvious. Taking shortcuts like avoiding a separate project for the component you’re modifying are common. I’ve worked on a number of projects where 3rd party source was included in directly in the project. Another common shortcut is to skip a formal test cycle when modifying source code. These two approaches alone can reduce the time spent yak shaving by 80%. With enough practice you can easily become a yak hero.

In the end it helps to remember there is a cost associated with every choice you make during development. Do not under estimate these costs and do not try to play hero. A yak hero hurts his team tremendously as you merely delay the actual and unavoidable cost of implementation long enough for a team member to assume it. The hero looks powerful and important while the rest of the team that cleans up the hero-slack appears slow and doggish. If you want to suck at programming, play the role of yak hero and shave away! Focus on shaving only the face and clipping a bit around the ears. Your clippers may be dull but, hey, who all will know!

Your Android Project doesn’t work in my IDE!

You just downloaded an online example project which will show you how to create a cool app for Android. You open it and you are presented by a popup dialog telling you the project is using an old version of the Gradle plugin. This wise popup suggests you update to project to take advantage of some new Android hotness like Instant Run. You’re all, “sure I’ll update!” After all, the popup is clearly wiser than you and who are you to argue with this manifestation of modal window management placed dead center in the screen? You update then you are greeted with build errors! How can this be? All you did was open the project and follow the suggestion to update! You haven’t modified any additional source files! Hi, I’m Cliff. You’re here because you think Android development sucks potatoes… and not just any kind of potatoes. You believe it sucks those big thick Idaho monsters… through a McDonald’s milkshake straw… on a hot summer day… when you’re extremely thirsty… and you are in dire need of hydration… and all you have is this Android suckiness which is severely impeded by these big ol’ potatoes! …That sort of sulkiness.

Maybe you’e a newcomer to the platform, or maybe you’re an author who recently released a written tutorial or video how-to and your audience is complaining about how all of your examples are broken. Whatever the case, the situation is not optimal. Unfortunately this problem happens quite often but today I’m going to attempt to explain how to solve the problem and why it exists in the first place.

Android Studio is in rapid development and updates feel like they come each week. With such a rapid cycle things sometimes get out of sync leaving you with version mismatch problems. Updating an Android Studio project is sometimes a 2-4 step process and you have to satisfy a different set of errors at each step. Taking the update option from the dialog will faithfully update the Gradle plugin as promised but it leaves the gradle wrapper at whatever version was originally configured along with another error! In the latest version of Android Studio you get a hyperlink thing next to the error that you can use to automatically fix the problem, but most people new to Android will stop at this point and feel lost. (Two errors in a row? I must be making the problem worse!) If you’re a seasoned dev you may overthink the solution and go drilling through all of your Gradle build files flipping all different kinds of versions possibly introducing further errors. (I’ve done this on several occasions.) If you’re an intermediate level dev then you’ll probably Google the error and hit a Stack Overflow post where the top answer tells you something completely helpless like “use the default Gradle wrapper”. Any one of these common fixes will leave you frustrated.

The solution is actually quite simple. Just pay attention to what the error says, “Error:Gradle version 2.10 is required. Current version is 2.4.” It’s implying you need to update the Gradle version used by the Gradle wrapper to a specific version, 2.10. In your set the distributionUrl to:
Do NOT get this confused with the version of the Gradle plugin which may be set to something like 2.1.0. Using this
will give further errors and increase your frustration level, sending you off to Google what the latest version of Gradle actually is and which version works best with the Android Gradle plugin.

In the end always pay careful attention to your error messages and learn to distinguish between them. Be prepared to walk through 2-4 different types of errors when updating an Android project built in older versions of Android Studio. Each error should convey a different message and a unique proposed solution. I’m sharing this tip because I’ve been bitten several times by my own projects which were developed under Android Studio 0.9x migrated to 1.x and no longer work out of the box in 2.x. I’ve also been stung when downloading example projects created by others which were developed only months ago and no longer work out of the box. Android Studio gets updates at an incredible rate and projects will feel broken but with a little patience and understanding you can work through the initial errors.

Windows git thinks my file is dirty

Do you have dirty files in git? Are you frustrated with how git thinks you’ve soiled a file that you haven’t actually touched? Are you looking for a way to tell git to stop bugging you? Hi I’m Cliff. You’re here because you have dirty files (but you have clean underwear). I’m here because I finally managed to work around a long standing issue where git thought my files were modified under Windows. The problem originates from machines with two different operating systems working on the same project. If you commit files from a *nix system and set the executable bit on then when you clone the project or pull the commit on Windows, git will treat these files as being altered. This is because the executable bit does not exist on Windows so the file appears different on the filesystem. To prevent git from treating these changes as different you should run this command:

git config core.fileMode false

That sets the global config for git to ignore file mode changes (things like read/write/execute). You can set it globally or use it for one off commands like:

git -c core.filemode=false diff

Pi in the Sky

It’s that moment you waited several months for, when you finally get your hands on a new piece of tech that you’d been eyeballing all along. You’re all worked up, the thing that use to adorn the shelves in the store is finally in your grubby little home, resting in the palm of your dirty little hands… ready to do whatever your little heat desires! Hi, I’m Cliff. You’re here because you just acquired a Raspberry Pi and you’re lost in the excitement. I’m here wondering why I used the word “little” so many times just now. Whatever the case, we’re here together and I’ve also recently acquired a Pi along with the bundle of excitement. For those of you that don’t spend days and nights deciphering logic, let me explain what this thing is. A Raspberry Pi is a tiny little palm-sized computer that can be used to make Robotics-like stuff. Coupled with the Arduinos I’ve already acquired it can be programmed and wired up to make a physical virtual assistant, a bow tie fastener, an automatic salt shaker that monitors your sodium intake, and eye glasses adjuster, or any crazy invention you can think of. For me it represents the bridge between the logical emulated software world I live in with the physical.

I don’t yet know what I plan to do with it but the sky is the limit. I’ll probably connect it to the turn signal switch in my car so I can finally negotiate lane changes without checking my blind spots. I’ll finally have an excuse for the very next car accident I get into. “Officer the other driver had ample time to avoid me as my blinker was definitely on well in advance of my lane change because I’ve automated the process…”

My next adventures will be back in the land of Linux as I explore how to boot this thing. Finding a bistro is always lots of fun! (Do they have a Pi dedicated section on DistroWatch these days?) Oh no! I just remembered my Mac has a problem mounting disks these days so I don’t even know how I’ll load the dang thing! Have you or anyone you’re related to faced the “I can’t mount any DMG” bug that seems to be spreading with the latest OS X? Leave a comment!