diff options
| author | Scott Main <smain@google.com> | 2012-06-21 17:14:39 -0700 |
|---|---|---|
| committer | Scott Main <smain@google.com> | 2012-06-21 21:27:30 -0700 |
| commit | 50e990c64fa23ce94efa76b9e72df7f8ec3cee6a (patch) | |
| tree | 52605cd25e01763596477956963fabcd087054b0 /docs/html/tools/testing | |
| parent | a2860267cad115659018d636bf9203a644c680a7 (diff) | |
| download | frameworks_base-50e990c64fa23ce94efa76b9e72df7f8ec3cee6a.zip frameworks_base-50e990c64fa23ce94efa76b9e72df7f8ec3cee6a.tar.gz frameworks_base-50e990c64fa23ce94efa76b9e72df7f8ec3cee6a.tar.bz2 | |
Massive clobber of all HTML files in developer docs for new site design
Change-Id: Idc55a0b368c1d2c1e7d4999601b739dd57f08eb3
Diffstat (limited to 'docs/html/tools/testing')
| -rw-r--r-- | docs/html/tools/testing/activity_test.jd | 1334 | ||||
| -rw-r--r-- | docs/html/tools/testing/activity_testing.jd | 375 | ||||
| -rw-r--r-- | docs/html/tools/testing/contentprovider_testing.jd | 217 | ||||
| -rw-r--r-- | docs/html/tools/testing/index.jd | 40 | ||||
| -rw-r--r-- | docs/html/tools/testing/service_testing.jd | 176 | ||||
| -rwxr-xr-x | docs/html/tools/testing/testing_android.jd | 640 | ||||
| -rw-r--r-- | docs/html/tools/testing/testing_eclipse.jd | 535 | ||||
| -rw-r--r-- | docs/html/tools/testing/testing_otheride.jd | 690 | ||||
| -rw-r--r-- | docs/html/tools/testing/what_to_test.jd | 86 |
9 files changed, 4093 insertions, 0 deletions
diff --git a/docs/html/tools/testing/activity_test.jd b/docs/html/tools/testing/activity_test.jd new file mode 100644 index 0000000..8288249 --- /dev/null +++ b/docs/html/tools/testing/activity_test.jd @@ -0,0 +1,1334 @@ +page.title=Activity Testing Tutorial +parent.title=Testing +parent.link=index.html +@jd:body + <div id="qv-wrapper"> + <div id="qv"> + <h2>In this document</h2> + <ol> + <li> + <a href="#Prerequisites">Prerequisites</a> + </li> + <li> + <a href="#DownloadCode">Installing the Tutorial Sample Code</a> + </li> + <li> + <a href="#SetupEmulator">Setting Up the Emulator</a> + </li> + <li> + <a href="#SetupProjects">Setting Up the Projects</a> + </li> + <li> + <a href="#CreateTestCaseClass">Creating the Test Case Class</a> + <ol> + <li> + <a href="#AddTestCaseClass">Adding the test case class file</a> + </li> + <li> + <a href="#AddConstructor">Adding the test case constructor</a> + </li> + <li> + <a href="#AddSetupMethod">Adding the setup method</a> + </li> + <li> + <a href="#AddPreConditionsTest">Adding an initial conditions test</a> + </li> + <li> + <a href="#AddUITest">Adding a UI test</a> + </li> + <li> + <a href="#StateManagementTests">Adding state management tests</a> + </li> + </ol> + </li> + <li> + <a href="#RunTests">Running the Tests and Seeing the Results</a> + </li> + <li> + <a href="#TestFailure">Forcing Some Tests to Fail</a> + </li> + <li> + <a href="#NextSteps">Next Steps</a> + </li> +</ol> +<h2 id="#Appendix">Appendix</h2> +<ol> + <li> + <a href="#InstallCompletedTestApp">Installing the Completed Test Application File</a> + </li> + <li> + <a href="#EditorCommandLine">For Users Not Developing In Eclipse</a> + </li> +</ol> +<h2>See Also</h2> +<ol> + <li> + <a href="{@docRoot}tools/testing/testing_android.html">Testing Fundamentals</a> + </li> + <li> + {@link android.test.ActivityInstrumentationTestCase2} + </li> + <li> + {@link junit.framework.Assert} + </li> + <li> + {@link android.test.InstrumentationTestRunner} + </li> +</ol> +</div> +</div> +<p> + Android includes powerful tools for testing applications. The tools extend JUnit with additional features, provide convenience classes for mock Android system objects, and use + instrumentation to give you control over your main application while you are testing it. The entire Android testing environment is discussed in the document + <a href="{@docRoot}tools/testing/testing_android.html">Testing Fundamentals</a>. +</p> +<p> + This tutorial demonstrates the Android testing tools by presenting a simple Android application and then leading you step-by-step through the creation of a test application for it. + The test application demonstrates these key points: +</p> + <ul> + <li> + An Android test is itself an Android application that is linked to the application under test by entries in its <code>AndroidManifest.xml</code> file. + </li> + <li> + Instead of Android components, an Android test application contains one or more test cases. Each of these is a separate class definition. + </li> + <li> + Android test case classes extend the JUnit {@link junit.framework.TestCase} class. + </li> + <li> + Android test case classes for activities extend JUnit and also connect you to the application under test with instrumentation. You can send keystroke or touch events directly to the UI. + </li> + <li> + You choose an Android test case class based on the type of component (application, activity, content provider, or service) you are testing. + </li> + <li> + Additional test tools in Eclipse/ADT provide integrated support for creating test applications, running them, and viewing the results. + </li> + </ul> +<p> + The test application contains methods that perform the following tests: +</p> + <ul> + <li> + Initial conditions test. Tests that the application under test initializes correctly. This is also a unit test of the application's + {@link android.app.Activity#onCreate(android.os.Bundle) onCreate()} method. Testing initial conditions also provides a confidence measure for subsequent tests. + </li> + <li> + UI test. Tests that the main UI operation works correctly. This test demonstrates the instrumentation features available in activity testing. + It shows that you can automate UI tests by sending key events from the test application to the main application. + </li> + <li> + State management tests. Test the application's code for saving state. This test demonstrates the instrumentation features of the test runner, which + are available for testing any component. + </li> + </ul> +<h2 id="Prerequisites">Prerequisites</h2> +<p> + The instructions and code in this tutorial depend on the following: +</p> + <ul> + <li> + Basic knowledge of Android programming. If you haven't yet written an Android application, + do the class + <a href="{@docRoot}training/basics/firstapp/index.html">Building Your First App</a>. + If you want to learn more about Spinner, the application under test, then you + might want to review the "Spinner" sample app. + </li> + <li> + Some familiarity with the Android testing framework and concepts. If you haven't explored + Android testing yet, start by reading the + <a href="{@docRoot}tools/testing/testing_android.html">Testing Fundamentals</a> + guide. + </li> + <li> + Eclipse with ADT. This tutorial describes how to set up and run a test application using + Eclipse with ADT. If you haven't yet installed Eclipse and the ADT plugin, + follow the steps in <a href="{@docRoot}sdk/installing/index.html">Installing the SDK</a> + to install them before continuing. If you are not developing in Eclipse, you will + find instructions for setting up and running the test application in the + <a href="#EditorCommandLine">appendix</a> of this document. + </li> + <li> + Android 1.5 platform (API Level 3) or higher. You must have the Android 1.5 platform + (API Level 3) or higher installed in your SDK, because this tutorial uses APIs that + were introduced in that version. + <p> + If you are not sure which platforms are installed in your SDK, + open the Android SDK and AVD Manager and check in the + <strong>Installed Packages</strong> panel. + If aren't sure how to download a platform into your SDK, + read <a href="{@docRoot}sdk/exploring.html">Exploring the SDK</a>. + </p> + </li> + </ul> +<h2 id="DownloadCode">Installing the Tutorial Sample Code</h2> +<p> + During this tutorial, you will be working with sample code that is provided as part + of the downloadable Samples component of the SDK. Specifically, you will be working + with a pair of related sample applications — an application under test and a test + application: +</p> + <ul> + <li> + Spinner is the application under test. This tutorial focuses on the + common situation of writing tests for an application that already exists, so the main + application is provided to you. + </li> + <li> + SpinnerTest is the test application. In the tutorial, you create this application + step-by-step. If you want to run quickly through the tutorial, + you can install the completed SpinnerTest application first, and then follow the + text. You may get more from the tutorial, however, if you create the test application + as you go. The instructions for installing the completed test application are in the + section + <a href="#InstallCompletedTestApp">Installing the Completed Test Application File</a>. + </li> + </ul> +<p> + The sample applications are described in more detail in the + <a href="{@docRoot}tools/samples/index.html">Samples</a> topic. Follow the instructions to + download the version of the samples that's appropriate for the platform you're working with. +</p> +<h2 id="SetupEmulator">Setting Up the Emulator</h2> +<p> + In this tutorial, you will use the Android emulator to run applications. The emulator needs + an Android Virtual Device (AVD) with an API level equal to or higher than the one you set for the projects in the previous step. + To find out how to check this and create the right AVD if necessary, + see <a href="{@docRoot}tools/devices/managing-avds.html">Creating an AVD</a>. +</p> +<p> + As a test of the AVD and emulator, run the SpinnerActivity application in Eclipse with ADT. When it starts, + click the large downward-pointing arrow to the right of the spinner text. You see the spinner expand and display the title "Select a planet" at the top. + Click one of the other planets. The spinner closes, and your selection appears below it on the screen. +</p> +<h2 id="SetupProjects">Setting Up the Projects</h2> +<p> + When you are ready to get started with the tutorial, begin by setting up Eclipse projects for + both Spinner (the application under test) and SpinnerTest (the test application). +</p> +<p> + You'll be using the Spinner application as-is, without modification, so you'll be loading it + into Eclipse as a new Android project from existing source. In the process, you'll be + creating a new test project associated with Spinner that will contain the SpinnerTest + application. The SpinnerTest application will be completely new and you'll be + using the code examples in this tutorial to add test classes and tests to it. +</p> +<p> + To install the Spinner app in a new Android project from existing source, following these steps: +</p> +<ol> + <li> + In Eclipse, select <strong>File</strong> > <strong>New</strong> > <strong>Project</strong> > <strong>Android</strong> > <strong>Android Project</strong>, + then click Next. The <strong>New Android Project</strong> dialog appears. + </li> + <li> + In the <em>Project name</em> text box, enter "SpinnerActivity". The <em>Properties</em> area is filled in automatically. + </li> + <li> + In the <em>Contents</em> area, set "Create project from existing source". + </li> + <li> + For <em>Location</em>, click <strong>Browse</strong>, navigate to the directory <code><SDK_path>/samples/android-8/Spinner</code>, + then click Open. The directory name <code><SDK_path>/samples/android-8/Spinner</code> now appears in the <em>Location</em> text box. + </li> + <li> + In the <em>Build Target</em> area, set a API level of 3 or higher. If you are already developing with a particular target, and it is API level 3 or higher, then use that target. + </li> + <li> + In the <em>Properties</em> area, in the <em>Min SDK Version:</em>, enter "3". + </li> + <li> + You should now see these values: + <ul> + <li><em>Project Name:</em> "SpinnerActivity"</li> + <li><em>Create project from existing source:</em> set</li> + <li><em>Location:</em> "<code><SDK_path>/samples/android-8/Spinner</code>"</li> + <li><em>Build Target:</em> "API level of 3 or higher" (<em>Target Name</em> "Android 1.5 or higher")</li> + <li><em>Package name:</em> (disabled, set to "<code>com.android.example.spinner</code>")</li> + <li><em>Create Activity:</em> (disabled, set to ".SpinnerActivity")</li> + <li><em>Min SDK Version:</em> "3"</li> + </ul> + <p> + The following screenshot summarizes these values: + </p> + <a href="{@docRoot}images/testing/eclipse_new_android_project_complete_callouts.png"> + <img src="{@docRoot}images/testing/eclipse_new_android_project_complete_callouts.png" alt="New Android Project dialog with filled-in values" style="height:230px"/> + </a> + + </li> +</ol> +<p> + To create a new test project for the SpinnerTest application, follow these steps: +</p> +<ol> + <li> + Click Next. The <strong>New Android Test Project</strong> dialog appears. + </li> + <li> + Set "Create a Test Project". + </li> + <li> + Leave the other values unchanged. The result should be: + <ul> + <li><em>Create a Test Project:</em> checked</li> + <li><em>Test Project Name:</em> "SpinnerActivityTest"</li> + <li><em>Use default location:</em> checked (this should contain the directory name "<code>workspace/SpinnerActivityTest</code>").</li> + <li><em>Build Target:</em> Use the same API level you used in the previous step.</li> + <li><em>Application name:</em> "SpinnerActivityTest"</li> + <li><em>Package name:</em> "<code>com.android.example.spinner.test</code>"</li> + <li><em>Min SDK Version:</em> "3"</li> + </ul> + <p> + The following screenshot summarizes these values: + </p> + <a href="{@docRoot}images/testing/eclipse_new_android_testproject_complete_callouts.png"> + <img src="{@docRoot}images/testing/eclipse_new_android_testproject_complete_callouts.png" alt="New Android Test Project dialog with filled-in values" style="height:230px"/> + </a> + </li> + <li> + Click Finish. Entries for SpinnerActivity and SpinnerActivityTest should appear in the + <strong>Package Explorer</strong>. + <p class="note"> + <strong>Note:</strong> If you set <em>Build Target</em> to an API level higher than "3", you will see the warning + "The API level for the selected SDK target does not match the Min SDK version". You do not need to change the API level or the Min SDK version. + The message tells you that you are building the projects with one particular API level, but specifying that a lower API level is required. This may + occur if you have chosen not to install the optional earlier API levels. + </p> + <p> + If you see errors listed in the <strong>Problems</strong> pane at the bottom of the Eclipse window, or if a red error marker appears next to + the entry for SpinnerActivity in the Package Explorer, highlight the SpinnerActivity entry and then select + <strong>Project</strong> > <strong>Clean</strong>. This should fix any errors. + </p> + </li> +</ol> +<p> + You now have the application under test in the SpinnerActivity project, + and an empty test project in SpinnerActivityTest. You may + notice that the two projects are in different directories, but Eclipse with + ADT handles this automatically. You should have no problem in either building or running them. +</p> +<p> + Notice that Eclipse and ADT have already done some initial setup for your test application. + Expand the SpinnerActivityTest project, and notice that it already has an + Android manifest file <code>AndroidManifest.xml</code>. + Eclipse with ADT created this when you added the test project. + Also, the test application is already set up to use instrumentation. You can see this + by examining <code>AndroidManifest.xml</code>. + Open it, then at the bottom of the center pane click <strong>AndroidManifest.xml</strong> + to display the XML contents: +</p> +<pre> +<?xml version="1.0" encoding="utf-8"?> +<manifest xmlns:android="http://schemas.android.com/apk/res/android" + package="com.android.example.spinner.test" + android:versionCode="1" + android:versionName="1.0"> + <uses-sdk android:minSdkVersion="3" /> + <instrumentation + android:targetPackage="com.android.example.spinner" + android:name="android.test.InstrumentationTestRunner" /> + <application android:icon="@drawable/icon" android:label="@string/app_name"> + <uses-library android:name="android.test.runner" /> + ... + </application> +</manifest> +</pre> +<p> + Notice the <code><instrumentation></code> element. The attribute + <code>android:targetPackage="com.android.example.spinner"</code> tells Android that the + application under test is defined in the Android package + <code>com.android.example.spinner</code>. Android now knows to use that + package's <code>AndroidManifest.xml</code> file to launch the application under test. + The <code><instrumentation></code> element also contains the attribute + <code>android:name="android.test.InstrumentationTestRunner"</code>, which tells Android + instrumentation to run the test application with Android's instrumentation-enabled test runner. +</p> +<h2 id="CreateTestCaseClass">Creating the Test Case Class</h2> + +<p> + You now have a test project SpinnerActivityTest, and the basic structure of a test + application also called SpinnerActivityTest. The basic structure includes all the files and + directories you need to build and run a test application, except for the class that + contains your tests (the test case class). +</p> +<p> + The next step is to define the test case class. In this tutorial, you'll be creating a + test case class that includes: +</p> +<ul> + <li> + Test setup. This use of the JUnit {@link junit.framework.TestCase#setUp() setUp()} + method demonstrates some of the tasks you might perform before running an Android test. + </li> + <li> + Testing initial conditions. This test demonstrates a good testing technique. + It also demonstrates that with Android instrumentation you can look at the application + under test <em>before</em> the main activity starts. The test checks that the application's + important objects have been initialized. + If the test fails, you then know that any other tests against the application are + unreliable, since the application was running in an incorrect state. + <p class="note"> + <strong>Note:</strong> The purpose of testing initial conditions is not the same as + using <code>setUp()</code>. The JUnit {@link junit.framework.TestCase#setUp()} runs once + before <strong>each test method</strong>, and its purpose is to create a clean test + environment. The initial conditions test runs once, and its purpose is to verify that the + application under test is ready to be tested. + </p> + </li> + <li> + Testing the UI. This test shows how to control the main application's UI + with instrumentation, a powerful automation feature of Android testing. + </li> + <li> + Testing state management. This test shows some techniques for testing how + well the application maintains state in the Android environment. Remember that to + provide a satisfactory user experience, your application must never lose its current state, + even if it's interrupted by a phone call or destroyed because of memory constraints. + The Android activity lifecycle provides ways to maintain state, and the + <code>SpinnerActivity</code> application uses them. The test shows the techniques for + verifying that they work. + </li> +</ul> +<p> + Android tests are contained in a special type of Android application that contains one or more test class definitions. Each of these contains + one or more test methods that do the actual tests. In this tutorial, you will first add a test case class, and then add tests to it. +</p> +<p> + You first choose an Android test case class to extend. You choose from the base test case classes according to the Android component you are testing and the types of tests you are doing. + In this tutorial, the application under test has a single simple activity, so the test case class will be for an Activity component. Android offers several, but the one that tests in + the most realistic environment is {@link android.test.ActivityInstrumentationTestCase2}, so you will use it as the base class. Like all activity test case classes, + <code>ActivityInstrumentationTestCase2</code> offers convenience methods for interacting directly with the UI of the application under test. +</p> +<h3 id="AddTestCaseClass">Adding the test case class file</h3> +<p> + To add <code>ActivityInstrumentationTestCase2</code> as the base test case class, follow these steps: +</p> +<ol> + <li> + In the Package Explorer, expand the test project SpinnerActivityTest if it is not open already. + </li> + <li> + Within SpinnerActivityTest, expand the <code>src/</code> folder and then the package marker for + <code>com.android.example.spinner.test</code>. Right-click on the package name and select <strong>New</strong> > <strong>Class</strong>:<br/> + <a href="{@docRoot}images/testing/spinner_create_test_class_callouts.png"> + <img alt="Menu for creating a new class in the test application" src="{@docRoot}images/testing/spinner_create_test_class_callouts.png" style="height:230px"/> + </a> + <p> + The <strong>New Java Class</strong> wizard appears: + </p> + <a href="{@docRoot}images/testing/spinnertest_new_class_callouts.png"> + <img alt="New Java Class wizard dialog" src="{@docRoot}images/testing/spinnertest_new_class_callouts.png" style="height:230px"/> + </a> + </li> + <li> + In the wizard, enter the following: + <ul> + <li> + <em>Name:</em> "SpinnerActivityTest". This becomes the name of your test class. + </li> + <li> + <em>Superclass:</em> "<code>android.test.ActivityInstrumentationTestCase2<SpinnerActivity></code>". The superclass is parameterized, so + you have to provide it your main application's class name. + </li> + </ul> + <p> + Do not change any of the other settings. Click Finish. + </p> + </li> + <li> + You now have a new file <code>SpinnerActivityTest.java</code> in the project. + </li> + <li> + To resolve the reference to SpinnerActivity, add the following import: +<pre> +import com.android.example.spinner.SpinnerActivity; +</pre> + </li> +</ol> +<h3 id="AddConstructor">Adding the test case constructor</h3> + <p> + To ensure that the test application is instantiated correctly, you must set up a constructor that the test + runner will call when it instantiates your test class. This constructor has no parameters, and its sole + purpose is to pass information to the superclass's default constructor. To set up this constructor, enter the + following code in the class: + </p> +<pre> + public SpinnerActivityTest() { + super("com.android.example.spinner", SpinnerActivity.class); + } // end of SpinnerActivityTest constructor definition +</pre> +<p> + This calls the superclass constructor with the Android package name (<code>com.android.example.spinner</code>)and main activity's class + (<code>SpinnerActivity.class</code>) for the application under test. Android uses this information to find the application and activity to test. +</p> +<p> + You are now ready to add tests, by adding test methods to the class. +</p> +<h3 id="AddSetupMethod">Adding the setup method</h3> +<p> + The <code>setUp()</code> method is invoked before every test. You use it to initialize variables and clean up from previous tests. You can also use + the JUnit {@link junit.framework.TestCase#tearDown() tearDown()} method, which runs <strong>after</strong> every test method. The tutorial does not use it. +</p> +<p> + The method you are going to add does the following: +</p> +<ul> + <li> + <code>super.setUp()</code>. Invokes the superclass constructor for <code>setUp()</code>, which is required by JUnit. + </li> + <li> + Calls {@link android.test.ActivityInstrumentationTestCase2#setActivityInitialTouchMode(boolean) setActivityInitialTouchMode(false)}. + This turns off <strong>touch mode</strong> in the device or emulator. If any of your test methods send key events to the application, + you must turn off touch mode <em>before</em> you start any activities; otherwise, the call is ignored. + </li> + <li> + Stores references to system objects. Retrieves and stores a reference to the activity under test, the <code>Spinner</code> + widget used by the activity, the <code>SpinnerAdapter</code> that backs the widget, and the string value of the selection that is + set when the application is first installed. These objects are used in the state management test. The methods invoked are: + <ul> + <li> + {@link android.test.ActivityInstrumentationTestCase2#getActivity()}. Gets a reference to the activity under test (<code>SpinnerActivity</code>). + This call also starts the activity if it is not already running. + </li> + <li> + {@link android.app.Activity#findViewById(int)}. Gets a reference to the <code>Spinner</code> widget of the application under test. + </li> + <li> + {@link android.widget.AbsSpinner#getAdapter()}. Gets a reference to the adapter (an array of strings) backing the spinner. + </li> + </ul> + </li> +</ul> +<p> + Add this code to the definition of <code>SpinnerActivityTest</code>, after the constructor definition: +</p> +<pre> + @Override + protected void setUp() throws Exception { + super.setUp(); + + setActivityInitialTouchMode(false); + + mActivity = getActivity(); + + mSpinner = + (Spinner) mActivity.findViewById( + com.android.example.spinner.R.id.Spinner01 + ); + + mPlanetData = mSpinner.getAdapter(); + + } // end of setUp() method definition +</pre> +<p> + Add these members to the test case class: +</p> +<pre> + private SpinnerActivity mActivity; + private Spinner mSpinner; + private SpinnerAdapter mPlanetData; +</pre> +<p> + Add these imports: +</p> +<pre> +import android.widget.Spinner; +import android.widget.SpinnerAdapter; +</pre> +<p> + You now have the the complete <code>setUp()</code> method. +</p> +<h3 id="AddPreConditionsTest">Adding an initial conditions test</h3> +<p> + The initial conditions test verifies that the application under test is initialized correctly. It is an illustration of the types of tests you can run, so it is not comprehensive. + It verifies the following: +</p> +<ul> + <li> + The item select listener is initialized. This listener is called when a selection is made from the spinner. + </li> + <li> + The adapter that provides values to the spinner is initialized. + </li> + <li> + The adapter contains the right number of entries. + </li> +</ul> +<p> + The actual initialization of the application under test is done in <code>setUp()</code>, which the test runner calls automatically before every test. The verifications are + done with JUnit {@link junit.framework.Assert} calls. As a useful convention, the method name is <code>testPreConditions()</code>: +</p> +<pre> + public void testPreConditions() { + assertTrue(mSpinner.getOnItemSelectedListener() != null); + assertTrue(mPlanetData != null); + assertEquals(mPlanetData.getCount(),ADAPTER_COUNT); + } // end of testPreConditions() method definition +</pre> +<p> + Add this member: +</p> +<pre> + public static final int ADAPTER_COUNT = 9; +</pre> +<h3 id="AddUITest">Adding a UI test</h3> +<p> + Now create a UI test that selects an item from the <code>Spinner</code> widget. The test sends key events to the UI with key events. + The test confirms that the selection matches the result you expect. +</p> +<p> + This test demonstrates the power of using instrumentation in Android testing. Only an instrumentation-based test class allows you to send key events (or touch events) + to the application under test. With instrumentation, you can test your UI without having to take screenshots, record the screen, or do human-controlled testing. +</p> +<p> + To work with the spinner, the test has to request focus for it and then set it to a known position. The test uses {@link android.view.View#requestFocus() requestFocus()} and + {@link android.widget.AbsSpinner#setSelection(int) setSelection()} to do this. Both of these methods interact with a View in the application under test, so you have to call them + in a special way. +</p> +<p> + Code in a test application that interacts with a View of the application under test must run in the main application's thread, also + known as the <em>UI thread</em>. To do this, you use the {@link android.app.Activity#runOnUiThread(java.lang.Runnable) Activity.runOnUiThread()} + method. You pass the code to <code>runOnUiThread()</code>in an anonymous {@link java.lang.Runnable Runnable} object. To set + the statements in the <code>Runnable</code> object, you override the object's {@link java.lang.Runnable#run()} method. +</p> +<p> + To send key events to the UI of the application under test, you use the <a href="{@docRoot}reference/android/test/InstrumentationTestCase.html#sendKeys(int...)">sendKeys</a>() method. + This method does not have to run on the UI thread, since Android uses instrumentation to pass the key events to the application under test. +</p> +<p> + The last part of the test compares the selection made by sending the key events to a pre-determined value. This tests that the spinner is working as intended. +</p> +<p> + The following sections show you how to add the code for this test. +</p> +<ol> + <li> + Get focus and set selection. Create a new method <code>public void testSpinnerUI()</code>. Add + code to to request focus for the spinner and set its position to default or initial position, "Earth". This code is run on the UI thread of + the application under test: +<pre> + public void testSpinnerUI() { + + mActivity.runOnUiThread( + new Runnable() { + public void run() { + mSpinner.requestFocus(); + mSpinner.setSelection(INITIAL_POSITION); + } // end of run() method definition + } // end of anonymous Runnable object instantiation + ); // end of invocation of runOnUiThread +</pre> + <p> + Add the following member to the test case class. + </p> +<pre> + public static final int INITIAL_POSITION = 0; +</pre> + </li> + <li> + Make a selection. Send key events to the spinner to select one of the items. To do this, open the spinner by + "clicking" the center keypad button (sending a DPAD_CENTER key event) and then clicking (sending) the down arrow keypad button five times. Finally, + click the center keypad button again to highlight the desired item. Add the following code: +<pre> + this.sendKeys(KeyEvent.KEYCODE_DPAD_CENTER); + for (int i = 1; i <= TEST_POSITION; i++) { + this.sendKeys(KeyEvent.KEYCODE_DPAD_DOWN); + } // end of for loop + + this.sendKeys(KeyEvent.KEYCODE_DPAD_CENTER); +</pre> + <p> + Add the following member to the test case class: + </p> +<pre> + public static final int TEST_POSITION = 5; +</pre> + <p> + This sets the final position of the spinner to "Saturn" (the spinner's backing adapter is 0-based). + </p> + </li> + <li> + Check the result. Query the current state of the spinner, and compare its current selection to the expected value. + Call the method {@link android.widget.AdapterView#getSelectedItemPosition() getSelectedItemPosition()} to find out the current selection position, and then + {@link android.widget.AdapterView#getItemAtPosition(int) getItemAtPosition()} to get the object corresponding to that position (casting it to a String). Assert that + this string value matches the expected value of "Saturn": +<pre> + mPos = mSpinner.getSelectedItemPosition(); + mSelection = (String)mSpinner.getItemAtPosition(mPos); + TextView resultView = + (TextView) mActivity.findViewById( + com.android.example.spinner.R.id.SpinnerResult + ); + + String resultText = (String) resultView.getText(); + + assertEquals(resultText,mSelection); + + } // end of testSpinnerUI() method definition +</pre> +<p> + Add the following members to the test case class: +</p> +<pre> + private String mSelection; + private int mPos; +</pre> + <p> + Add the following imports to the test case class: + </p> +<pre> + import android.view.KeyEvent; + import android.widget.TextView; +</pre> + </li> +</ol> +<p> + Pause here to run the tests you have. The procedure for running a test application is different + from running a regular Android application. You run a test application as an Android JUnit + application. To see how to do this, see <a href="#RunTests">Running the Tests and Seeing the Results</a>. +</p> +<p> + Eventually, you will see the <code>SpinnerActivity</code> application start, and the test + application controlling it by sending it key events. You will also see a new + <strong>JUnit</strong> view in the Explorer pane, showing the results of the + test. The JUnit view is documented in a following section, + <a href="#RunTests">Running the Test and Seeing the Results</a>. +</p> +<h3 id="StateManagementTests">Adding state management tests</h3> +<p> + You now write two tests that verify that SpinnerActivity maintains its state when it is paused or terminated. + The state, in this case, is the current selection in the spinner. When users make a selection, + pause or terminate the application, and then resume or restart it, they should see + the same selection. +</p> +<p> + Maintaining state is an important feature of an application. Users may switch from the current + application temporarily to answer the phone, and then switch back. Android may decide to + terminate and restart an activity to change the screen orientation, or terminate an unused + activity to regain storage. In each case, users are best served by having the UI return to its + previous state (except where the logic of the application dictates otherwise). +</p> +<p> + SpinnerActivity manages its state in these ways: +</p> + <ul> + <li> + Activity is hidden. When the spinner screen (the activity) is running but hidden by some other screen, it + stores the spinner's position and value in a form that persists while the application is running. + </li> + <li> + Application is terminated. When the activity is terminated, it stores the spinner's position and value in + a permanent form. The activity can read the position and value when it restarts, and restore the spinner to its previous state. + </li> + <li> + Activity re-appears. When the user returns to the spinner screen, the previous selection is restored. + </li> + <li> + Application is restarted. When the user starts the application again, the previous selection is restored. + </li> + </ul> +<p class="note"> + <strong>Note:</strong> An application can manage its state in other ways as well, but these are + not covered in this tutorial. +</p> +<p> + When an activity is hidden, it is <strong>paused</strong>. When it re-appears, it + <strong>resumes</strong>. Recognizing that these are key points in an activity's life cycle, + the Activity class provides two callback methods {@link android.app.Activity#onPause()} and + {@link android.app.Activity#onResume()} for handling pauses and resumes. + SpinnerActivity uses them for code that saves and restores state. +</p> +<p> + <strong>Note:</strong> If you would like to learn more about the difference between losing + focus/pausing and killing an application, + read about the <a href="{@docRoot}guide/components/activities.html#Lifecycle">activity +lifecycle</a>. +</p> +<p> + The first test verifies that the spinner selection is maintained after the entire application is shut down and then restarted. The test uses instrumentation to + set the spinner's variables outside of the UI. It then terminates the activity by calling {@link android.app.Activity#finish() Activity.finish()}, and restarts it + using the instrumentation method {@link android.test.ActivityInstrumentationTestCase2#getActivity()}. The test then asserts that the current spinner state matches + the test values. +</p> +<p> + The second test verifies that the spinner selection is maintained after the activity is paused and then resumed. The test uses instrumentation to + set the spinner's variables outside of the UI and then force calls to the <code>onPause()</code> and <code>onResume()</code> methods. The test then + asserts that the current spinner state matches the test values. +</p> +<p> + Notice that these tests make limited assumptions about the mechanism by which the activity manages state. The tests use the activity's getters and + setters to control the spinner. The first test also knows that hiding an activity calls <code>onPause()</code>, and bringing it back to the foreground + calls <code>onResume()</code>. Other than this, the tests treat the activity as a "black box". +</p> +<p> + To add the code for testing state management across shutdown and restart, follow these steps: +</p> + <ol> + <li> + Add the test method <code>testStateDestroy()</code>, then + set the spinner selection to a test value: +<pre> + public void testStateDestroy() { + mActivity.setSpinnerPosition(TEST_STATE_DESTROY_POSITION); + mActivity.setSpinnerSelection(TEST_STATE_DESTROY_SELECTION); +</pre> + </li> + <li> + Terminate the activity and restart it: +<pre> + mActivity.finish(); + mActivity = this.getActivity(); +</pre> + </li> + <li> + Get the current spinner settings from the activity: +<pre> + int currentPosition = mActivity.getSpinnerPosition(); + String currentSelection = mActivity.getSpinnerSelection(); +</pre> + </li> + <li> + Test the current settings against the test values: +<pre> + assertEquals(TEST_STATE_DESTROY_POSITION, currentPosition); + assertEquals(TEST_STATE_DESTROY_SELECTION, currentSelection); + } // end of testStateDestroy() method definition +</pre> +<p> + Add the following members to the test case class: +<pre> + public static final int TEST_STATE_DESTROY_POSITION = 2; + public static final String TEST_STATE_DESTROY_SELECTION = "Earth"; +</pre> + </li> + </ol> +<p> + To add the code for testing state management across a pause and resume, follow these steps: +</p> +<ol> + <li> + Add the test method <code>testStatePause()</code>: +<pre> + @UiThreadTest + public void testStatePause() { +</pre> + <p> + The <code>@UiThreadTest</code> annotation tells Android to build this method so that it runs + on the UI thread. This allows the method to change the state of the spinner widget in the + application under test. This use of <code>@UiThreadTest</code> shows that, if necessary, you + can run an entire method on the UI thread. + </p> + </li> + <li> + Set up instrumentation. Get the instrumentation object + that is controlling the application under test. This is used later to + invoke the <code>onPause()</code> and <code>onResume()</code> methods: +<pre> + Instrumentation mInstr = this.getInstrumentation(); +</pre> + </li> + <li> + Set the spinner selection to a test value: +<pre> + mActivity.setSpinnerPosition(TEST_STATE_PAUSE_POSITION); + mActivity.setSpinnerSelection(TEST_STATE_PAUSE_SELECTION); +</pre> + </li> + <li> + Use instrumentation to call the Activity's <code>onPause()</code>: +<pre> + mInstr.callActivityOnPause(mActivity); +</pre> + <p> + Under test, the activity is waiting for input. The invocation of + {@link android.app.Instrumentation#callActivityOnPause(android.app.Activity)} + performs a call directly to the activity's <code>onPause()</code> instead + of manipulating the activity's UI to force it into a paused state. + </p> + </li> + <li> + Force the spinner to a different selection: +<pre> + mActivity.setSpinnerPosition(0); + mActivity.setSpinnerSelection(""); +</pre> + <p> + This ensures that resuming the activity actually restores the + spinner's state rather than simply leaving it as it was. + </p> + </li> + <li> + Use instrumentation to call the Activity's <code>onResume()</code>: +<pre> + mInstr.callActivityOnResume(mActivity); +</pre> + <p> + Invoking {@link android.app.Instrumentation#callActivityOnResume(android.app.Activity)} + affects the activity in a way similar to <code>callActivityOnPause</code>. The + activity's <code>onResume()</code> method is invoked instead of manipulating the + activity's UI to force it to resume. + </p> + </li> + <li> + Get the current state of the spinner: +<pre> + int currentPosition = mActivity.getSpinnerPosition(); + String currentSelection = mActivity.getSpinnerSelection(); +</pre> + </li> + <li> + Test the current spinner state against the test values: +<pre> + assertEquals(TEST_STATE_PAUSE_POSITION,currentPosition); + assertEquals(TEST_STATE_PAUSE_SELECTION,currentSelection); + } // end of testStatePause() method definition +</pre> + <p> + Add the following members to the test case class: + </p> +<pre> + public static final int TEST_STATE_PAUSE_POSITION = 4; + public static final String TEST_STATE_PAUSE_SELECTION = "Jupiter"; +</pre> + </li> + <li> + Add the following imports: +<pre> + import android.app.Instrumentation; + import android.test.UiThreadTest; +</pre> + </li> +</ol> +<h2 id="RunTests">Running the Tests and Seeing the Results</h2> + <p> + The most simple way to run the <code>SpinnerActivityTest</code> test case is to run it directly from the Package Explorer. + </p> + <p> + To run the <code>SpinnerActivityTest</code> test, follow these steps: +</p> + <ol> + <li> + In the Package Explorer, right-click the project SpinnerActivityTest at the top level, and then + select <strong>Run As</strong> > <strong>Android JUnit Test</strong>:<br/> + <a href="{@docRoot}images/testing/spinnertest_runas_menu_callouts.png"> + <img alt="Menu to run a test as an Android JUnit test" src="{@docRoot}images/testing/spinnertest_runas_menu_callouts.png" style="height:230px"> + </a> + </li> + <li> + You will see the emulator start. When the unlock option is displayed (its appearance depends on the API level you specified for the AVD), + unlock the home screen. + </li> + <li> + The test application starts. You see a new tab for the <strong>JUnit</strong> view, next to the Package Explorer tab:<br/> + <a href="{@docRoot}images/testing/spinnertest_junit_panel.png"> + <img alt="The JUnit window" src="{@docRoot}images/testing/spinnertest_junit_panel.png" style="height:230px"> + </a> + </li> +</ol> +<p> + This view contains two sub-panes. The top pane summarizes the tests that were run, and the bottom pane shows failure traces for + highlighted tests. +</p> +<p> + At the conclusion of a successful test run, this is the view's appearance:<br/> + <a href="{@docRoot}images/testing/spinnertest_junit_success.png"> + <img src="{@docRoot}images/testing/spinnertest_junit_success.png" alt="JUnit test run success" style="height:230px"/> + </a> +</p> +<p> + The upper pane summarizes the test: +</p> + <ul> + <li> + Total time elapsed for the test application(labeled <em>Finished after <x> seconds</em>). + </li> + <li> + Number of runs (<em>Runs:</em>) - the number of tests in the entire test class. + </li> + <li> + Number of errors (<em>Errors:</em>) - the number of program errors and exceptions encountered during + the test run. + </li> + <li> + Number of failures (<em>Failures:</em>) - the number of test failures encountered during the test + run. This is the number of assertion failures. A test can fail even if the program does not encounter an error. + </li> + <li> + A progress bar. The progress bar extends from left to right as the tests run. + <p> + If all the tests succeed, the bar remains green. If a test fails, the bar turns from green to red. + </p> + </li> + <li> + A test method summary. Below the bar, you see a line for each class in the test application. To look at the results for the individual + methods in a test, click the arrow at the left to expand the line. You see the name of each test method. To the + right of the name, you see the time taken by the test. You can look at the test's code + by double-clicking its name. + </li> + </ul> +<p> + The lower pane contains the failure trace. If all the tests are successful, this pane is empty. If some tests fail, + then if you highlight a failed test in the upper pane, the lower view contains a stack trace for the test. This is + demonstrated in the next section. +</p> +<p class="note"> + <strong>Note:</strong> If you run the test application and nothing seems to happen, look for + the JUnit view. If you do not see it, you may have run the test application + as a regular Android application. + Remember that you need to run it as an Android <strong>JUnit</strong> + application. +</p> +<h2 id="TestFailure">Forcing Some Tests to Fail</h2> +<p> + A test is as useful when it fails as when it succeeds. This section shows what happens in Eclipse with ADT when a test fails. You + can quickly see that a test class has failed, find the method or methods that failed, and then use a failure trace to find + the exact problem. +</p> +<p> + The example application SpinnerActivity that you downloaded passes all the tests in the test application SpinnerActivityTest. + To force the test to fail, you must modify the example application. You change a line of setup code in the application under test. This + causes the <code>testPreConditions()</code> and <code>testTextView()</code> test methods to fail. +</p> +<p> + To force the tests to fail, follow these steps: +</p> +<ol> + <li> + In Eclipse with ADT, go to the SpinnerActivity project and open the file <code>SpinnerActivity.java</code>. + </li> + <li> + At the top of <code>SpinnerActivity.java</code>, at the end of the <code>onCreate()</code> method, find the following line: +<pre> + // mySpinner.setOnItemSelectedListener(null); +</pre> + <p>Remove the forward slash characters at the beginning of the line to + uncomment the line. This sets the listener callback to null: + </p> +<pre> + mySpinner.setOnItemSelectedListener(null); +</pre> + </li> + <li> + The <code>testPreConditions()</code> method in <code>SpinnerActivityTest</code> contains the following test: + <code>assertTrue(mSpinner.getOnItemSelectedListener() != null);</code>. This test asserts that the listener callback is <em>not</em> null. + Since you have modified the application under test, this assertion now fails. + </li> + <li> + Run the test, as described in the previous section <a href="#RunTests">Running the Tests and Seeing the Results</a>. + </li> +</ol> +<p> + The JUnit view is either created or updated with the results of the test. Now, however, the progress bar is red, + the number of failures is 2, and small "x" icons appear in the list icons next to the testPreConditions and + TestSpinnerUI tests. This indicates that the tests have failed. The display is similar to this:<br/> + <a href="{@docRoot}images/testing/spinnertest_junit_panel_fail_callouts.png"> + <img src="{@docRoot}images/testing/spinnertest_junit_panel_fail_callouts.png" alt="The JUnit Failure window" style="height:230px"/> + </a> +</p> +<p> + You now want to look at the failures to see exactly where they occurred. +</p> +<p> + To examine the failures, follow these steps: +</p> +<ol> + <li> + Click the testPreconditions entry. In the lower pane entitled <strong>Failure Trace</strong>, + you see a stack trace of the calls that led to the failure. This trace is similar to the following screenshot:<br/> + <a href="{@docRoot}images/testing/spinnertest_junit_panel_failtrace_callouts.png"> + <img src="{@docRoot}images/testing/spinnertest_junit_panel_failtrace_callouts.png" alt="The JUnit failure trace" style="height:230px"/> + </a> + </li> + <li> + The first line of the trace tells you the error. In this case, a JUnit assertion failed. To look at the + assertion in the test code, double-click the next line (the first line of the trace). In the center pane + a new tabbed window opens, containing the code for the test application <code>SpinnerActivityTest</code>. The failed assertion + is highlighted in the middle of the window. + </li> +</ol> +<p> + The assertion failed because you modified the main application to set the <code>getOnItemSelectedListener</code> callback to <code>null</code>. +</p> +<p> + You can look at the failure in <code>testTextView</code> if you want. Remember, though, that <code>testPreConditions</code> is meant to verify the + initial setup of the application under test. If testPreConditions() fails, then succeeding tests can't be trusted. The best strategy to follow is to + fix the problem and re-run all the tests. +</p> +<p> + Remember to go back to <code>SpinnerActivity.java</code> and re-comment the line you uncommented in an earlier step. +</p> +<p> + You have now completed the tutorial. +</p> +<h2 id="NextSteps">Next Steps</h2> +<p> + This example test application has shown you how to create a test project and link it to + the application you want to test, how to choose and add a test case class, how to write + UI and state management tests, and how to run the tests against the application under + test. Now that you are familiar with the basics of testing Android applications, here + are some suggested next steps: +</p> +<p> + <strong>Learn more about testing on Android</strong> +</p> +<ul> + <li> + If you haven't done so already, read the + <a href="{@docRoot}tools/testing/testing_android.html">Testing Fundamentals</a> + document in the <em>Dev Guide</em>. It provides an overview of how testing on Android + works. If you are just getting started with Android testing, reading that document will + help you understand the tools available to you, so that you can develop effective + tests. + </li> +</ul> +<p> + <strong>Review the main Android test case classes</strong> +</p> +<ul> + <li> + {@link android.test.ActivityInstrumentationTestCase2} + </li> + <li> + {@link android.test.ActivityUnitTestCase} + </li> + <li> + {@link android.test.ProviderTestCase2} + </li> + <li> + {@link android.test.ServiceTestCase} + </li> +</ul> +<p> + <strong>Learn more about the assert and utility classes</strong> +</p> +<ul> + <li> + {@link junit.framework.Assert}, the JUnit Assert class. + </li> + <li> + {@link android.test.MoreAsserts}, additional Android assert methods. + </li> + <li> + {@link android.test.ViewAsserts}, useful assertion methods for testing Views. + </li> + <li> + {@link android.test.TouchUtils}, utility methods for simulating touch events in an Activity. + </li> +</ul> +<p> + <strong>Learn about instrumentation and the instrumented test runner</strong> +</p> +<ul> + <li> + {@link android.app.Instrumentation}, the base instrumentation class. + </li> + <li> + {@link android.test.InstrumentationTestCase}, the base instrumentation test case. + </li> + <li> + {@link android.test.InstrumentationTestRunner}, the standard Android test runner. + </li> +</ul> +<h2 id="Appendix">Appendix</h2> +<h3 id="InstallCompletedTestApp">Installing the Completed Test Application File</h3> +<p> + The recommended approach to this tutorial is to follow the instructions step-by-step and + write the test code as you go. However, if you want to do this tutorial quickly, + you can install the entire file for the test application into the test project. +</p> +<p> + To do this, you first create a test project with the necessary structure and files by using + the automated tools in Eclipse. Then you exit Eclipse and copy the test application's file + from the SpinnerTest sample project into your test project. The SpinnerTest sample project is + part of the Samples component of the SDK. +</p> +<p> + The result is a complete test application, ready to run against the Spinner sample application. +</p> +<p> + To install the test application file, follow these steps: +</p> +<ol> + <li> + Set up the projects for the application under test and the test application, as described + in the section section <a href="#SetupProjects">Setting Up the Projects</a>. + </li> + <li> + Set up the emulator, as described in the section <a href="#SetupEmulator">Setting Up the Emulator</a>. + </li> + <li> + Add the test case class, as described in the section <a href="#AddTestCaseClass">Adding the test case class file</a>. + </li> + <li> + Close Eclipse with ADT. + </li> + <li> + Copy the file <code><SDK_path>/samples/android-8/SpinnerTest/src/com/android/example/spinner/test/SpinnerActivityTest.java</code> + to the directory <code>workspace/SpinnerActivityTest/src/com/android/example/spinner/test/</code>. + </li> + <li> + Restart Eclipse with ADT. + </li> + <li> + In Eclipse with ADT, re-build the project <code>SpinnerActivityTest</code> by selecting it in the Package Explorer, right-clicking, + and selecting <em>Project</em> > <em>Clean</em>. + </li> + <li> + The complete, working test application should now be in the <code>SpinnerActivityTest</code> project. + </li> +</ol> +<p> + You can now continue with the tutorial, starting at the section <a href="#AddConstructor">Adding the test case constructor</a> and + following along in the text. +</p> +<h3 id="EditorCommandLine">For Users Not Developing In Eclipse</h3> +<p> + If you are not developing in Eclipse, you can still do this tutorial. Android provides tools for + creating test applications using a code editor and command-line tools. You use the following tools: +</p> +<ul> + <li> + <a href="{@docRoot}tools/help/adb.html">adb</a> - Installs and uninstalls applications and test applications to a device or the emulator. You + also use this tool to run the test application from the command line. + </li> + <li> + <a href="{@docRoot}tools/help/android.html">android</a> - Manages projects and test projects. This tool also manages AVDs and Android platforms. + </li> +</ul> + <p> + You use the <code>emulator</code> tool to run the emulator from the command line. + </p> + <p> + Here are the general steps for doing this tutorial using an editor and the command line: + </p> +<ol> + <li> + As described in the section <a href="#DownloadCode">Installing the Tutorial Sample Code</a>, get the sample code. You will then + have a directory <code><SDK_path>/samples/android-8</code>, containing (among others) the directories <code>Spinner</code> + and <code>SpinnerTest</code>: + <ul> + <li> + <code>Spinner</code> contains the main application, also known as the <strong>application under test</strong>. This tutorial focuses on the + common situation of writing tests for an application that already exists, so the main application is provided to you. + </li> + <li> + <code>SpinnerTest</code> contains all the code for the test application. If you want to run quickly through the tutorial, you can + install the test code and then follow the text. You may get more from the tutorial, however, if you write the code as you go. The instructions + for installing the test code are in the section <a href="#InstallCompletedTestApp">Appendix: Installing the Completed Test Application File</a>. + </li> + </ul> + </li> + <li> + Navigate to the directory <code><SDK_path>/samples/android-8</code>. + </li> + <li> + Create a new Android application project using <code>android create project</code>: +<pre> +$ android create project -t <APItarget> -k com.android.example.spinner -a SpinnerActivity -n SpinnerActivity -p Spinner +</pre> + <p> + The value of <code><APItarget></code> should be "3" (API level 3) or higher. If you are already developing with a particular API level, and it is + higher than 3, then use that API level. + </p> + <p> + This a new Android project <code>SpinnerActivity</code> in the existing <code>Spinner</code> directory. The existing source and + resource files are not touched, but the <code>android</code> tool adds the necessary build files. + </p> + </li> + <li> + Create a new Android test project using <code>android create test-project</code>: +<pre> +$ android create test-project -m ../Spinner -n SpinnerActivityTest -p SpinnerActivityTest +</pre> + <p> + This will create a new Android test project in the <em>new</em> directory <code>SpinnerActivityTest</code>. You do this + so that the solution to the tutorial that is in <code>SpinnerTest</code> is left untouched. If you want to use the solution + code instead of entering it as you read through the tutorial, refer to the section + <a href="#InstallCompletedTestApp">Appendix: Installing the Completed Test Application File</a>. + </p> + <p class="Note"> + <strong>Note:</strong> Running <code>android create test-project</code> will automatically create + the file <code>AndroidManifest.xml</code> with the correct <code><instrumentation></code> element. + </p> + </li> + <li> + Build the sample application. If you are building with Ant, then it is easiest to use the command <code>ant debug</code> to build a debug version, since the SDK comes + with a debug signing key. The result will be the file <code>Spinner/bin/SpinnerActivity-debug.apk</code>. + You can install this to your device or emulator. Attach your device or start the emulator if you haven't already, and run the command: +<pre> +$ adb install Spinner/bin/SpinnerActivity-debug.apk +</pre> + </li> + <li> + To create the test application, create a file <code>SpinnerActivityTest.java</code> in the directory + <code>SpinnerActivityTest/src/com/android/example/spinner/test/</code>. + </li> + <li> + Follow the tutorial, starting with the section <a href="#CreateTestCaseClass">Creating the Test Case Class</a>. When you are prompted to + run the sample application, go the the Launcher screen in your device or emulator and select SpinnerActivity. + When you are prompted to run the test application, return here to continue with the following instructions. + </li> + <li> + Build the test application. If you are building with Ant, then it is easiest to use the command <code>ant debug</code> to build a + debug version, since the SDK comes with a debug signing key. The result will be the Android file + <code>SpinnerActivityTest/bin/SpinnerActivityTest-debug.apk</code>. You can install this to your device or emulator. + Attach your device or start the emulator if you haven't already, and run the command: +<pre> +$ adb install SpinnerActivityTest/bin/SpinnerActivityTest-debug.apk +</pre> + </li> + <li> + In your device or emulator, check that both the main application <code>SpinnerActivity</code> and the test application + <code>SpinnerActivityTest</code> are installed. + </li> + <li> + To run the test application, enter the following at the command line: +<pre> +$ adb shell am instrument -w com.android.example.spinner.test/android.test.InstrumentationTestRunner + </pre> + </li> +</ol> +<p> + The result of a successful test looks like this: +</p> +<pre> +com.android.example.spinner.test.SpinnerActivityTest:.... +Test results for InstrumentationTestRunner=.... +Time: 10.098 +OK (4 tests) +</pre> +<p> + If you force the test to fail, as described in the previous section <a href="#TestFailure">Forcing Some Tests to Fail</a>, then + the output looks like this: +</p> +<pre> +com.android.example.spinner.test.SpinnerActivityTest: +Failure in testPreConditions: +junit.framework.AssertionFailedError + at com.android.example.spinner.test.SpinnerActivityTest.testPreConditions(SpinnerActivityTest.java:104) + at java.lang.reflect.Method.invokeNative(Native Method) + at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:205) + at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:195) + at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:175) + at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169) + at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154) + at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:430) + at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447) +Failure in testSpinnerUI: +junit.framework.ComparisonFailure: expected:<Result> but was:<Saturn> + at com.android.example.spinner.test.SpinnerActivityTest.testSpinnerUI(SpinnerActivityTest.java:153) + at java.lang.reflect.Method.invokeNative(Native Method) + at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:205) + at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:195) + at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:175) + at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169) + at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154) + at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:430) + at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447) +.. +Test results for InstrumentationTestRunner=.F.F.. +Time: 9.377 +FAILURES!!! +Tests run: 4, Failures: 2, Errors: 0 +</pre> diff --git a/docs/html/tools/testing/activity_testing.jd b/docs/html/tools/testing/activity_testing.jd new file mode 100644 index 0000000..7190b98 --- /dev/null +++ b/docs/html/tools/testing/activity_testing.jd @@ -0,0 +1,375 @@ +page.title=Activity Testing +parent.title=Testing +parent.link=index.html +@jd:body + +<div id="qv-wrapper"> + <div id="qv"> + <h2>In this document</h2> + <ol> + <li> + <a href="#ActivityTestAPI">The Activity Testing API</a> + <ol> + <li> + <a href="#ActivityInstrumentationTestCase2">ActivityInstrumentationTestCase2</a> + </li> + <li> + <a href="#ActivityUnitTestCase">ActivityUnitTestCase</a> + </li> + <li> + <a href="#SingleLaunchActivityTestCase">SingleLaunchActivityTestCase</a> + </li> + <li> + <a href="#MockObjectNotes">Mock objects and activity testing</a> + </li> + <li> + <a href="#AssertionNotes">Assertions for activity testing</a> + </li> + </ol> + </li> + <li> + <a href="#WhatToTest">What to Test</a> + </li> + <li> + <a href="#NextSteps">Next Steps</a> + </li> + <li> + <a href="#UITesting">Appendix: UI Testing Notes</a> + <ol> + <li> + <a href="#RunOnUIThread">Testing on the UI thread</a> + </li> + <li> + <a href="#NotouchMode">Turning off touch mode</a> + </li> + <li> + <a href="#UnlockDevice">Unlocking the Emulator or Device</a> + </li> + <li> + <a href="#UITestTroubleshooting">Troubleshooting UI tests</a> + </li> + </ol> + </li> + </ol> +<h2>Key Classes</h2> + <ol> + <li>{@link android.test.InstrumentationTestRunner}</li> + <li>{@link android.test.ActivityInstrumentationTestCase2}</li> + <li>{@link android.test.ActivityUnitTestCase}</li> + </ol> +<h2>Related Tutorials</h2> + <ol> + <li> + <a href="{@docRoot}tools/testing/activity_test.html">Activity Testing Tutorial</a> + </li> + </ol> +<h2>See Also</h2> + <ol> + <li> + <a href="{@docRoot}tools/testing/testing_eclipse.html"> + Testing from Eclipse with ADT</a> + </li> + <li> + <a href="{@docRoot}tools/testing/testing_otheride.html"> + Testing from Other IDEs</a> + </li> + </ol> + </div> +</div> +<p> + Activity testing is particularly dependent on the the Android instrumentation framework. + Unlike other components, activities have a complex lifecycle based on callback methods; these + can't be invoked directly except by instrumentation. Also, the only way to send events to the + user interface from a program is through instrumentation. +</p> +<p> + This document describes how to test activities using instrumentation and other test + facilities. The document assumes you have already read + <a href="{@docRoot}tools/testing/testing_android.html">Testing Fundamentals</a>, + the introduction to the Android testing and instrumentation framework. +</p> +<h2 id="ActivityTestAPI">The Activity Testing API</h2> +<p> + The activity testing API base class is {@link android.test.InstrumentationTestCase}, + which provides instrumentation to the test case subclasses you use for Activities. +</p> +<p> + For activity testing, this base class provides these functions: +</p> +<ul> + <li> + Lifecycle control: With instrumentation, you can start the activity under test, pause it, + and destroy it, using methods provided by the test case classes. + </li> + <li> + Dependency injection: Instrumentation allows you to create mock system objects such as + Contexts or Applications and use them to run the activity under test. This + helps you control the test environment and isolate it from production systems. You can + also set up customized Intents and start an activity with them. + </li> + <li> + User interface interaction: You use instrumentation to send keystrokes or touch events + directly to the UI of the activity under test. + </li> +</ul> +<p> + The activity testing classes also provide the JUnit framework by extending + {@link junit.framework.TestCase} and {@link junit.framework.Assert}. +</p> +<p> + The two main testing subclasses are {@link android.test.ActivityInstrumentationTestCase2} and + {@link android.test.ActivityUnitTestCase}. To test an Activity that is launched in a mode + other than <code>standard</code>, you use {@link android.test.SingleLaunchActivityTestCase}. +</p> +<h3 id="ActivityInstrumentationTestCase2">ActivityInstrumentationTestCase2</h3> +<p> + The {@link android.test.ActivityInstrumentationTestCase2} test case class is designed to do + functional testing of one or more Activities in an application, using a normal system + infrastructure. It runs the Activities in a normal instance of the application under test, + using a standard system Context. It allows you to send mock Intents to the activity under + test, so you can use it to test an activity that responds to multiple types of intents, or + an activity that expects a certain type of data in the intent, or both. Notice, though, that it + does not allow mock Contexts or Applications, so you can not isolate the test from the rest of + a production system. +</p> +<h3 id="ActivityUnitTestCase">ActivityUnitTestCase</h3> +<p> + The {@link android.test.ActivityUnitTestCase} test case class tests a single activity in + isolation. Before you start the activity, you can inject a mock Context or Application, or both. + You use it to run activity tests in isolation, and to do unit testing of methods + that do not interact with Android. You can not send mock Intents to the activity under test, + although you can call + {@link android.app.Activity#startActivity(Intent) Activity.startActivity(Intent)} and then + look at arguments that were received. +</p> +<h3 id="SingleLaunchActivityTestCase">SingleLaunchActivityTestCase</h3> +<p> + The {@link android.test.SingleLaunchActivityTestCase} class is a convenience class for + testing a single activity in an environment that doesn't change from test to test. + It invokes {@link junit.framework.TestCase#setUp() setUp()} and + {@link junit.framework.TestCase#tearDown() tearDown()} only once, instead of once per + method call. It does not allow you to inject any mock objects. +</p> +<p> + This test case is useful for testing an activity that runs in a mode other than + <code>standard</code>. It ensures that the test fixture is not reset between tests. You + can then test that the activity handles multiple calls correctly. +</p> +<h3 id="MockObjectNotes">Mock objects and activity testing</h3> +<p> + This section contains notes about the use of the mock objects defined in + {@link android.test.mock} with activity tests. +</p> +<p> + The mock object {@link android.test.mock.MockApplication} is only available for activity + testing if you use the {@link android.test.ActivityUnitTestCase} test case class. + By default, <code>ActivityUnitTestCase</code>, creates a hidden <code>MockApplication</code> + object that is used as the application under test. You can inject your own object using + {@link android.test.ActivityUnitTestCase#setApplication(Application) setApplication()}. +</p> +<h3 id="AssertionNotes">Assertions for activity testing</h3> +<p> + {@link android.test.ViewAsserts} defines assertions for Views. You use it to verify the + alignment and position of View objects, and to look at the state of ViewGroup objects. +</p> +<h2 id="WhatToTest">What To Test</h2> +<ul> + <li> + Input validation: Test that an activity responds correctly to input values in an + EditText View. Set up a keystroke sequence, send it to the activity, and then + use {@link android.view.View#findViewById(int)} to examine the state of the View. You can + verify that a valid keystroke sequence enables an OK button, while an invalid one leaves the + button disabled. You can also verify that the Activity responds to invalid input by + setting error messages in the View. + </li> + <li> + Lifecycle events: Test that each of your application's activities handles lifecycle events + correctly. In general, lifecycle events are actions, either from the system or from the + user, that trigger a callback method such as <code>onCreate()</code> or + <code>onClick()</code>. For example, an activity should respond to pause or destroy events + by saving its state. Remember that even a change in screen orientation causes the current + activity to be destroyed, so you should test that accidental device movements don't + accidentally lose the application state. + </li> + <li> + Intents: Test that each activity correctly handles the intents listed in the intent + filter specified in its manifest. You can use + {@link android.test.ActivityInstrumentationTestCase2} to send mock Intents to the + activity under test. + </li> + <li> + Runtime configuration changes: Test that each activity responds correctly to the + possible changes in the device's configuration while your application is running. These + include a change to the device's orientation, a change to the current language, and so + forth. Handling these changes is described in detail in the topic + <a href="{@docRoot}guide/topics/resources/runtime-changes.html">Handling Runtime + Changes</a>. + </li> + <li> + Screen sizes and resolutions: Before you publish your application, make sure to test it on + all of the screen sizes and densities on which you want it to run. You can test the + application on multiple sizes and densities using AVDs, or you can test your application + directly on the devices that you are targeting. For more information, see the topic + <a href="{@docRoot}guide/practices/screens_support.html">Supporting Multiple Screens</a>. + </li> +</ul> +<h2 id="NextSteps">Next Steps</h2> +<p> + To learn how to set up and run tests in Eclipse, please refer to +<a href="{@docRoot}tools/testing/testing_eclipse.html">Testing from Eclipse with ADT</a>. + If you're not working in Eclipse, refer to +<a href="{@docRoot}tools/testing/testing_otheride.html">Testing from Other IDEs</a>. +</p> +<p> + If you want a step-by-step introduction to testing activities, try the + <a href="{@docRoot}tools/testing/activity_test.html">Activity Testing Tutorial</a>, which + guides you through a testing scenario that you develop against an activity-oriented application. +</p> +<h2 id="UITesting">Appendix: UI Testing Notes</h2> +<p> + The following sections have tips for testing the UI of your Android application, specifically + to help you handle actions that run in the UI thread, touch screen and keyboard events, and home + screen unlock during testing. +</p> +<h3 id="RunOnUIThread">Testing on the UI thread</h3> +<p> + An application's activities run on the application's <strong>UI thread</strong>. Once the + UI is instantiated, for example in the activity's <code>onCreate()</code> method, then all + interactions with the UI must run in the UI thread. When you run the application normally, it + has access to the thread and does not have to do anything special. +</p> +<p> + This changes when you run tests against the application. With instrumentation-based classes, + you can invoke methods against the UI of the application under test. The other test classes + don't allow this. To run an entire test method on the UI thread, you can annotate the thread + with <code>@UIThreadTest</code>. Notice that this will run <em>all</em> of the method statements + on the UI thread. Methods that do not interact with the UI are not allowed; for example, you + can't invoke <code>Instrumentation.waitForIdleSync()</code>. +</p> +<p> + To run a subset of a test method on the UI thread, create an anonymous class of type + <code>Runnable</code>, put the statements you want in the <code>run()</code> method, and + instantiate a new instance of the class as a parameter to the method + <code><em>appActivity</em>.runOnUiThread()</code>, where <code><em>appActivity</em></code> is + the instance of the application you are testing. +</p> +<p> + For example, this code instantiates an activity to test, requests focus (a UI action) for the + Spinner displayed by the activity, and then sends a key to it. Notice that the calls to + <code>waitForIdleSync</code> and <code>sendKeys</code> aren't allowed to run on the UI thread: +</p> +<pre> + private MyActivity mActivity; // MyActivity is the class name of the app under test + private Spinner mSpinner; + + ... + + protected void setUp() throws Exception { + super.setUp(); + mInstrumentation = getInstrumentation(); + + mActivity = getActivity(); // get a references to the app under test + + /* + * Get a reference to the main widget of the app under test, a Spinner + */ + mSpinner = (Spinner) mActivity.findViewById(com.android.demo.myactivity.R.id.Spinner01); + + ... + + public void aTest() { + /* + * request focus for the Spinner, so that the test can send key events to it + * This request must be run on the UI thread. To do this, use the runOnUiThread method + * and pass it a Runnable that contains a call to requestFocus on the Spinner. + */ + mActivity.runOnUiThread(new Runnable() { + public void run() { + mSpinner.requestFocus(); + } + }); + + mInstrumentation.waitForIdleSync(); + + this.sendKeys(KeyEvent.KEYCODE_DPAD_CENTER); +</pre> + +<h3 id="NotouchMode">Turning off touch mode</h3> +<p> + To control the emulator or a device with key events you send from your tests, you must turn off + touch mode. If you do not do this, the key events are ignored. +</p> +<p> + To turn off touch mode, you invoke + <code>ActivityInstrumentationTestCase2.setActivityTouchMode(false)</code> + <em>before</em> you call <code>getActivity()</code> to start the activity. You must invoke the + method in a test method that is <em>not</em> running on the UI thread. For this reason, you + can't invoke the touch mode method from a test method that is annotated with + <code>@UIThread</code>. Instead, invoke the touch mode method from <code>setUp()</code>. +</p> +<h3 id="UnlockDevice">Unlocking the emulator or device</h3> +<p> + You may find that UI tests don't work if the emulator's or device's home screen is disabled with + the keyguard pattern. This is because the application under test can't receive key events sent + by <code>sendKeys()</code>. The best way to avoid this is to start your emulator or device + first and then disable the keyguard for the home screen. +</p> +<p> + You can also explicitly disable the keyguard. To do this, + you need to add a permission in the manifest file (<code>AndroidManifest.xml</code>) and + then disable the keyguard in your application under test. Note, though, that you either have to + remove this before you publish your application, or you have to disable it with code in + the published application. +</p> +<p> + To add the the permission, add the element + <code><uses-permission android:name="android.permission.DISABLE_KEYGUARD"/></code> + as a child of the <code><manifest></code> element. To disable the KeyGuard, add the + following code to the <code>onCreate()</code> method of activities you intend to test: +</p> +<pre> + mKeyGuardManager = (KeyguardManager) getSystemService(KEYGUARD_SERVICE); + mLock = mKeyGuardManager.newKeyguardLock("<em>activity_classname</em>"); + mLock.disableKeyguard(); +</pre> +<p>where <code><em>activity_classname</em></code> is the class name of the activity.</p> +<h3 id="UITestTroubleshooting">Troubleshooting UI tests</h3> +<p> + This section lists some of the common test failures you may encounter in UI testing, and their + causes: +</p> +<dl> + <dt><code>WrongThreadException</code></dt> + <dd> + <p><strong>Problem:</strong></p> + For a failed test, the Failure Trace contains the following error message: + <code> + android.view.ViewRoot$CalledFromWrongThreadException: Only the original thread that created + a view hierarchy can touch its views. + </code> + <p><strong>Probable Cause:</strong></p> + This error is common if you tried to send UI events to the UI thread from outside the UI + thread. This commonly happens if you send UI events from the test application, but you don't + use the <code>@UIThread</code> annotation or the <code>runOnUiThread()</code> method. The + test method tried to interact with the UI outside the UI thread. + <p><strong>Suggested Resolution:</strong></p> + Run the interaction on the UI thread. Use a test class that provides instrumentation. See + the previous section <a href="#RunOnUIThread">Testing on the UI Thread</a> + for more details. + </dd> + <dt><code>java.lang.RuntimeException</code></dt> + <dd> + <p><strong>Problem:</strong></p> + For a failed test, the Failure Trace contains the following error message: + <code> + java.lang.RuntimeException: This method can not be called from the main application thread + </code> + <p><strong>Probable Cause:</strong></p> + This error is common if your test method is annotated with <code>@UiThreadTest</code> but + then tries to do something outside the UI thread or tries to invoke + <code>runOnUiThread()</code>. + <p><strong>Suggested Resolution:</strong></p> + Remove the <code>@UiThreadTest</code> annotation, remove the <code>runOnUiThread()</code> + call, or re-factor your tests. + </dd> +</dl> diff --git a/docs/html/tools/testing/contentprovider_testing.jd b/docs/html/tools/testing/contentprovider_testing.jd new file mode 100644 index 0000000..a6440df --- /dev/null +++ b/docs/html/tools/testing/contentprovider_testing.jd @@ -0,0 +1,217 @@ +page.title=Content Provider Testing +parent.title=Testing +parent.link=index.html +@jd:body + +<div id="qv-wrapper"> + <div id="qv"> + <h2>In this document</h2> + <ol> + <li> + <a href="#DesignAndTest">Content Provider Design and Testing</a> + </li> + <li> + <a href="#ContentProviderTestAPI">The Content Provider Testing API</a> + <ol> + <li> + <a href="#ProviderTestCase2">ProviderTestCase2 </a> + </li> + <li> + <a href="#MockObjects">Mock object classes</a> + </li> + </ol> + </li> + <li> + <a href="#WhatToTest">What To Test</a> + </li> + <li> + <a href="#NextSteps">Next Steps</a> + </li> + </ol> + <h2>Key Classes</h2> + <ol> + <li>{@link android.test.InstrumentationTestRunner}</li> + <li>{@link android.test.ProviderTestCase2}</li> + <li>{@link android.test.IsolatedContext}</li> + <li>{@link android.test.mock.MockContentResolver}</li> + </ol> + <h2>Related Tutorials</h2> + <ol> + <li> + <a href="{@docRoot}tools/testing/activity_test.html">Activity Testing Tutorial</a> + </li> + </ol> + <h2>See Also</h2> + <ol> + <li> + <a + href="{@docRoot}tools/testing/testing_android.html"> + Testing Fundamentals</a> + </li> + <li> + <a href="{@docRoot}tools/testing/testing_eclipse.html"> + Testing From Eclipse with ADT</a> + </li> + <li> + <a href="{@docRoot}tools/testing/testing_otheride.html"> + Testing From Other IDEs</a> + </li> + </ol> + </div> +</div> +<p> + Content providers, which store and retrieve data and make it accessible across applications, + are a key part of the Android API. As an application developer you're allowed to provide your + own public providers for use by other applications. If you do, then you should test them + using the API you publish. +</p> +<p> + This document describes how to test public content providers, although the information is + also applicable to providers that you keep private to your own application. If you aren't + familiar with content providers or the Android testing framework, please read + <a href="{@docRoot}guide/topics/providers/content-providers.html">Content Providers</a>, + the guide to developing content providers, and + <a href="{@docRoot}tools/testing/testing_android.html">Testing Fundamentals</a>, + the introduction to the Android testing and instrumentation framework. +</p> +<h2 id="DesignAndTest">Content Provider Design and Testing</h2> +<p> + In Android, content providers are viewed externally as data APIs that provide + tables of data, with their internals hidden from view. A content provider may have many + public constants, but it usually has few if any public methods and no public variables. + This suggests that you should write your tests based only on the provider's public members. + A content provider that is designed like this is offering a contract between itself and its + users. +</p> +<p> + The base test case class for content providers, + {@link android.test.ProviderTestCase2}, allows you to test your content provider in an + isolated environment. Android mock objects such as {@link android.test.IsolatedContext} and + {@link android.test.mock.MockContentResolver} also help provide an isolated test environment. +</p> +<p> + As with other Android tests, provider test packages are run under the control of the test + runner {@link android.test.InstrumentationTestRunner}. The section + <a href="{@docRoot}tools/testing/testing_android.html#InstrumentationTestRunner"> + Running Tests With InstrumentationTestRunner</a> describes the test runner in + more detail. The topic <a href="{@docRoot}tools/testing/testing_eclipse.html"> + Testing From Eclipse with ADT</a> shows you how to run a test package in Eclipse, and the + topic <a href="{@docRoot}tools/testing/testing_otheride.html"> + Testing From Other IDEs</a> + shows you how to run a test package from the command line. +</p> +<h2 id="ContentProviderTestAPI">Content Provider Testing API</h2> +<p> + The main focus of the provider testing API is to provide an isolated testing environment. This + ensures that tests always run against data dependencies set explicitly in the test case. It + also prevents tests from modifying actual user data. For example, you want to avoid writing + a test that fails because there was data left over from a previous test, and you want to + avoid adding or deleting contact information in a actual provider. +</p> +<p> + The test case class and mock object classes for provider testing set up this isolated testing + environment for you. +</p> +<h3 id="ProviderTestCase2">ProviderTestCase2</h3> +<p> + You test a provider with a subclass of {@link android.test.ProviderTestCase2}. This base class + extends {@link android.test.AndroidTestCase}, so it provides the JUnit testing framework as well + as Android-specific methods for testing application permissions. The most important + feature of this class is its initialization, which creates the isolated test environment. +</p> +<p> + The initialization is done in the constructor for {@link android.test.ProviderTestCase2}, which + subclasses call in their own constructors. The {@link android.test.ProviderTestCase2} + constructor creates an {@link android.test.IsolatedContext} object that allows file and + database operations but stubs out other interactions with the Android system. + The file and database operations themselves take place in a directory that is local to the + device or emulator and has a special prefix. +</p> +<p> + The constructor then creates a {@link android.test.mock.MockContentResolver} to use as the + resolver for the test. The {@link android.test.mock.MockContentResolver} class is described in + detail in the section + <a href="{@docRoot}tools/testing/testing_android.html#MockObjectClasses">Mock object +classes</a>. +</p> +<p> + Lastly, the constructor creates an instance of the provider under test. This is a normal + {@link android.content.ContentProvider} object, but it takes all of its environment information + from the {@link android.test.IsolatedContext}, so it is restricted to + working in the isolated test environment. All of the tests done in the test case class run + against this isolated object. +</p> +<h3 id="MockObjects">Mock object classes</h3> +<p> + {@link android.test.ProviderTestCase2} uses {@link android.test.IsolatedContext} and + {@link android.test.mock.MockContentResolver}, which are standard mock object classes. To + learn more about them, please read + <a href="{@docRoot}tools/testing/testing_android.html#MockObjectClasses"> + Testing Fundamentals</a>. +</p> +<h2 id="WhatToTest">What To Test</h2> +<p> + The topic <a href="{@docRoot}tools/testing/what_to_test.html">What To Test</a> + lists general considerations for testing Android components. + Here are some specific guidelines for testing content providers. +</p> +<ul> + <li> + Test with resolver methods: Even though you can instantiate a provider object in + {@link android.test.ProviderTestCase2}, you should always test with a resolver object + using the appropriate URI. This ensures that you are testing the provider using the same + interaction that a regular application would use. + </li> + <li> + Test a public provider as a contract: If you intent your provider to be public and + available to other applications, you should test it as a contract. This includes + the following ideas: + <ul> + <li> + Test with constants that your provider publicly exposes. For + example, look for constants that refer to column names in one of the provider's + data tables. These should always be constants publicly defined by the provider. + </li> + <li> + Test all the URIs offered by your provider. Your provider may offer several URIs, + each one referring to a different aspect of the data. The + <a href="{@docRoot}resources/samples/NotePad/index.html">Note Pad</a> sample, + for example, features a provider that offers one URI for retrieving a list of notes, + another for retrieving an individual note by it's database ID, and a third for + displaying notes in a live folder. + </li> + <li> + Test invalid URIs: Your unit tests should deliberately call the provider with an + invalid URI, and look for errors. Good provider design is to throw an + IllegalArgumentException for invalid URIs. + + </li> + </ul> + </li> + <li> + Test the standard provider interactions: Most providers offer six access methods: + query, insert, delete, update, getType, and onCreate(). Your tests should verify that all + of these methods work. These are described in more detail in the topic + <a href="{@docRoot}guide/topics/providers/content-providers.html">Content Providers</a>. + </li> + <li> + Test business logic: Don't forget to test the business logic that your provider should + enforce. Business logic includes handling of invalid values, financial or arithmetic + calculations, elimination or combining of duplicates, and so forth. A content provider + does not have to have business logic, because it may be implemented by activities that + modify the data. If the provider does implement business logic, you should test it. + </li> +</ul> +<h2 id="NextSteps">Next Steps</h2> +<p> + To learn how to set up and run tests in Eclipse, please refer to +<a href="{@docRoot}tools/testing/testing_eclipse.html">Testing from Eclipse with ADT</a>. + If you're not working in Eclipse, refer to +<a href="{@docRoot}tools/testing/testing_otheride.html">Testing From Other IDEs</a>. +</p> +<p> + If you want a step-by-step introduction to testing activities, try the + <a href="{@docRoot}tools/testing/activity_test.html">Activity Testing Tutorial</a>, which + guides you through a testing scenario that you develop against an activity-oriented application. +</p> + diff --git a/docs/html/tools/testing/index.jd b/docs/html/tools/testing/index.jd new file mode 100644 index 0000000..56de4cf --- /dev/null +++ b/docs/html/tools/testing/index.jd @@ -0,0 +1,40 @@ +page.title=Testing +@jd:body + +<p> The Android framework includes an integrated testing framework that helps you test all aspects +of your application and the SDK tools include tools for setting up and running test applications. +Whether you are working in Eclipse with ADT or working from the command line, the SDK tools help you +set up and run your tests within an emulator or the device you are targeting. </p> + +<p>If you aren't yet familiar with the Android testing framework, start by reading <a +href="{@docRoot}tools/testing/testing_android.html">Testing Fundamentals</a>. For a step-by-step +introduction to Android testing, try the <a +href="{@docRoot}tools/testing/activity_test.html">Activity Testing Tutorial</a>. </p> + + + +<div class="landing-docs"> + + <div class="col-13" style="margin-left:0"> + <h3>Blog Articles</h3> + + <a href="http://android-developers.blogspot.com/2010/12/new-gingerbread-api-strictmode.html"> + <h4>New Gingerbread API: StrictMode</h4> + <p>StrictMode is a new API in Gingerbread which primarily lets you set a policy on a thread +declaring what you’re not allowed to do on that thread, and what the penalty is if you violate the +policy. Implementation-wise, this policy is simply a thread-local integer bitmask.</p> + </a> + + <a href="http://android-developers.blogspot.com/2010/10/traceview-war-story.html"> + <h4>Traceview War Story</h4> + <p>I recently took my first serious look at Traceview, and it occurred to me, first, that +there are probably a few other Android developers who haven’t used it and, second, that this is an +opportunity to lecture sternly on one of my favorite subjects: performance improvement and +profiling.</p> + </a> + </div> + + +</div> + + diff --git a/docs/html/tools/testing/service_testing.jd b/docs/html/tools/testing/service_testing.jd new file mode 100644 index 0000000..7c56fd9 --- /dev/null +++ b/docs/html/tools/testing/service_testing.jd @@ -0,0 +1,176 @@ +page.title=Service Testing +parent.title=Testing +parent.link=index.html +@jd:body + +<div id="qv-wrapper"> + <div id="qv"> + <h2>In this document</h2> + <ol> + <li> + <a href="#DesignAndTest">Service Design and Testing</a> + </li> + <li> + <a href="#ServiceTestCase">ServiceTestCase</a> + </li> + <li> + <a href="#MockObjects">Mock object classes</a> + </li> + <li> + <a href="#TestAreas">What to Test</a> + </li> + </ol> + <h2>Key Classes</h2> + <ol> + <li>{@link android.test.InstrumentationTestRunner}</li> + <li>{@link android.test.ServiceTestCase}</li> + <li>{@link android.test.mock.MockApplication}</li> + <li>{@link android.test.RenamingDelegatingContext}</li> + </ol> + <h2>Related Tutorials</h2> + <ol> + <li> + <a href="{@docRoot}tools/testing/activity_test.html">Activity Testing Tutorial</a> + </li> + </ol> + <h2>See Also</h2> + <ol> + <li> + <a href="{@docRoot}tools/testing/testing_eclipse.html"> + Testing From Eclipse with ADT</a> + </li> + <li> + <a href="{@docRoot}tools/testing/testing_otheride.html"> + Testing From Other IDEs</a> + </li> + </ol> + </div> +</div> +<p> + Android provides a testing framework for Service objects that can run them in + isolation and provides mock objects. The test case class for Service objects is + {@link android.test.ServiceTestCase}. Since the Service class assumes that it is separate + from its clients, you can test a Service object without using instrumentation. +</p> +<p> + This document describes techniques for testing Service objects. If you aren't familiar with the + Service class, please read the <a href="{@docRoot}guide/components/services.html"> + Services</a> document. If you aren't familiar with Android testing, please read + <a href="{@docRoot}tools/testing/testing_android.html">Testing Fundamentals</a>, + the introduction to the Android testing and instrumentation framework. +</p> +<h2 id="DesignAndTest">Service Design and Testing</h2> +<p> + When you design a Service, you should consider how your tests can examine the various states + of the Service lifecycle. If the lifecycle methods that start up your Service, such as + {@link android.app.Service#onCreate() onCreate()} or + {@link android.app.Service#onStartCommand(Intent, int, int) onStartCommand()} do not normally + set a global variable to indicate that they were successful, you may want to provide such a + variable for testing purposes. +</p> +<p> + Most other testing is facilitated by the methods in the {@link android.test.ServiceTestCase} + test case class. For example, the {@link android.test.ServiceTestCase#getService()} method + returns a handle to the Service under test, which you can test to confirm that the Service is + running even at the end of your tests. +</p> +<h2 id="ServiceTestCase">ServiceTestCase</h2> +<p> + {@link android.test.ServiceTestCase} extends the JUnit {@link junit.framework.TestCase} class + with with methods for testing application permissions and for controlling the application and + Service under test. It also provides mock application and Context objects that isolate your + test from the rest of the system. +</p> +<p> + {@link android.test.ServiceTestCase} defers initialization of the test environment until you + call {@link android.test.ServiceTestCase#startService(Intent) ServiceTestCase.startService()} or + {@link android.test.ServiceTestCase#bindService(Intent) ServiceTestCase.bindService()}. This + allows you to set up your test environment, particularly your mock objects, before the Service + is started. +</p> +<p> + Notice that the parameters to <code>ServiceTestCase.bindService()</code>are different from + those for <code>Service.bindService()</code>. For the <code>ServiceTestCase</code> version, + you only provide an Intent. Instead of returning a boolean, + <code>ServiceTestCase.bindService()</code> returns an object that subclasses + {@link android.os.IBinder}. +</p> +<p> + The {@link android.test.ServiceTestCase#setUp()} method for {@link android.test.ServiceTestCase} + is called before each test. It sets up the test fixture by making a copy of the current system + Context before any test methods touch it. You can retrieve this Context by calling + {@link android.test.ServiceTestCase#getSystemContext()}. If you override this method, you must + call <code>super.setUp()</code> as the first statement in the override. +</p> +<p> + The methods {@link android.test.ServiceTestCase#setApplication(Application) setApplication()} + and {@link android.test.AndroidTestCase#setContext(Context)} setContext()} allow you to set + a mock Context or mock Application (or both) for the Service, before you start it. These mock + objects are described in <a href="#MockObjects">Mock object classes</a>. +</p> +<p> + By default, {@link android.test.ServiceTestCase} runs the test method + {@link android.test.AndroidTestCase#testAndroidTestCaseSetupProperly()}, which asserts that + the base test case class successfully set up a Context before running. +</p> +<h2 id="MockObjects">Mock object classes</h2> +<p> + <code>ServiceTestCase</code> assumes that you will use a mock Context or mock Application + (or both) for the test environment. These objects isolate the test environment from the + rest of the system. If you don't provide your own instances of these objects before you + start the Service, then {@link android.test.ServiceTestCase} will create its own internal + instances and inject them into the Service. You can override this behavior by creating and + injecting your own instances before starting the Service +</p> +<p> + To inject a mock Application object into the Service under test, first create a subclass of + {@link android.test.mock.MockApplication}. <code>MockApplication</code> is a subclass of + {@link android.app.Application} in which all the methods throw an Exception, so to use it + effectively you subclass it and override the methods you need. You then inject it into the + Service with the + {@link android.test.ServiceTestCase#setApplication(Application) setApplication()} method. + This mock object allows you to control the application values that the Service sees, and + isolates it from the real system. In addition, any hidden dependencies your Service has on + its application reveal themselves as exceptions when you run the test. +</p> +<p> + You inject a mock Context into the Service under test with the + {@link android.test.AndroidTestCase#setContext(Context) setContext()} method. The mock + Context classes you can use are described in more detail in + <a href="{@docRoot}tools/testing/testing_android.html#MockObjectClasses"> + Testing Fundamentals</a>. +</p> +<h2 id="TestAreas">What to Test</h2> +<p> + The topic <a href="{@docRoot}tools/testing/what_to_test.html">What To Test</a> + lists general considerations for testing Android components. + Here are some specific guidelines for testing a Service: +</p> +<ul> + <li> + Ensure that the {@link android.app.Service#onCreate()} is called in response to + {@link android.content.Context#startService(Intent) Context.startService()} or + {@link android.content.Context#bindService(Intent,ServiceConnection,int) Context.bindService()}. + Similarly, you should ensure that {@link android.app.Service#onDestroy()} is called in + response to {@link android.content.Context#stopService(Intent) Context.stopService()}, + {@link android.content.Context#unbindService(ServiceConnection) Context.unbindService()}, + {@link android.app.Service#stopSelf()}, or + {@link android.app.Service#stopSelfResult(int) stopSelfResult()}. + </li> + <li> + Test that your Service correctly handles multiple calls from + <code>Context.startService()</code>. Only the first call triggers + <code>Service.onCreate()</code>, but all calls trigger a call to + <code>Service.onStartCommand()</code>. + <p> + In addition, remember that <code>startService()</code> calls don't + nest, so a single call to <code>Context.stopService()</code> or + <code>Service.stopSelf()</code> (but not <code>stopSelf(int)</code>) + will stop the Service. You should test that your Service stops at the correct point. + </p> + </li> + <li> + Test any business logic that your Service implements. Business logic includes checking for + invalid values, financial and arithmetic calculations, and so forth. + </li> +</ul> diff --git a/docs/html/tools/testing/testing_android.jd b/docs/html/tools/testing/testing_android.jd new file mode 100755 index 0000000..acf5ec2 --- /dev/null +++ b/docs/html/tools/testing/testing_android.jd @@ -0,0 +1,640 @@ +page.title=Testing Fundamentals +parent.title=Testing +parent.link=index.html +@jd:body + +<div id="qv-wrapper"> + <div id="qv"> + <h2>In this document</h2> + <ol> + <li> + <a href="#TestStructure">Test Structure</a> + </li> + <li> + <a href="#TestProjects">Test Projects</a> + </li> + <li> + <a href="#TestAPI">The Testing API</a> + <ol> + <li> + <a href="#JUnit">JUnit</a> + </li> + <li> + <a href="#Instrumentation">Instrumentation</a> + </li> + <li> + <a href="#TestCaseClasses">Test case classes</a> + </li> + <li> + <a href="#AssertionClasses">Assertion classes</a> + </li> + <li> + <a href="#MockObjectClasses">Mock object classes</a> + </li> + </ol> + </li> + <li> + <a href="#InstrumentationTestRunner">Running Tests</a> + </li> + <li> + <a href="#TestResults">Seeing Test Results</a> + </li> + <li> + <a href="#Monkeys">monkey and monkeyrunner</a> + </li> + <li> + <a href="#PackageNames">Working With Package Names</a> + </li> + <li> + <a href="#WhatToTest">What To Test</a> + </li> + <li> + <a href="#NextSteps">Next Steps</a> + </li> + </ol> + <h2>Key classes</h2> + <ol> + <li>{@link android.test.InstrumentationTestRunner}</li> + <li>{@link android.test}</li> + <li>{@link android.test.mock}</li> + <li>{@link junit.framework}</li> + </ol> + <h2>Related tutorials</h2> + <ol> + <li> + <a href="{@docRoot}tools/testing/activity_test.html">Activity Testing Tutorial</a> + </li> + </ol> + <h2>See also</h2> + <ol> + <li> + <a href="{@docRoot}tools/testing/testing_eclipse.html"> + Testing from Eclipse with ADT</a> + </li> + <li> + <a href="{@docRoot}tools/testing/testing_otheride.html"> + Testing from Other IDEs</a> + </li> + <li> + <a href="{@docRoot}tools/help/monkeyrunner_concepts.html"> + monkeyrunner</a> + </li> + <li> + <a href="{@docRoot}tools/help/monkey.html">UI/Application Exerciser Monkey</a> + </li> + </ol> + </div> +</div> +<p> + The Android testing framework, an integral part of the development environment, + provides an architecture and powerful tools that help you test every aspect of your application + at every level from unit to framework. +</p> +<p> + The testing framework has these key features: +</p> +<ul> + <li> + Android test suites are based on JUnit. You can use plain JUnit to test a class that doesn't + call the Android API, or Android's JUnit extensions to test Android components. If you're + new to Android testing, you can start with general-purpose test case classes such as {@link + android.test.AndroidTestCase} and then go on to use more sophisticated classes. + </li> + <li> + The Android JUnit extensions provide component-specific test case classes. These classes + provide helper methods for creating mock objects and methods that help you control the + lifecycle of a component. + </li> + <li> + Test suites are contained in test packages that are similar to main application packages, so + you don't need to learn a new set of tools or techniques for designing and building tests. + </li> + <li> + The SDK tools for building and tests are available in Eclipse with ADT, and also in + command-line form for use with other IDES. These tools get information from the project of + the application under test and use this information to automatically create the build files, + manifest file, and directory structure for the test package. + </li> + <li> + The SDK also provides + <a href="{@docRoot}tools/help/monkeyrunner_concepts.html">monkeyrunner</a>, an API + testing devices with Python programs, and <a + href="{@docRoot}tools/help/monkey.html">UI/Application Exerciser Monkey</a>, + a command-line tool for stress-testing UIs by sending pseudo-random events to a device. + </li> +</ul> +<p> + This document describes the fundamentals of the Android testing framework, including the + structure of tests, the APIs that you use to develop tests, and the tools that you use to run + tests and view results. The document assumes you have a basic knowledge of Android application + programming and JUnit testing methodology. +</p> +<p> + The following diagram summarizes the testing framework: +</p> +<div style="width: 70%; margin-left:auto; margin-right:auto;"> +<a href="{@docRoot}images/testing/test_framework.png"> + <img src="{@docRoot}images/testing/test_framework.png" + alt="The Android testing framework"/> +</a> +</div> +<h2 id="TestStructure">Test Structure</h2> +<p> + Android's build and test tools assume that test projects are organized into a standard + structure of tests, test case classes, test packages, and test projects. +</p> +<p> + Android testing is based on JUnit. In general, a JUnit test is a method whose + statements test a part of the application under test. You organize test methods into classes + called test cases (or test suites). Each test is an isolated test of an individual module in + the application under test. Each class is a container for related test methods, although it + often provides helper methods as well. +</p> +<p> + In JUnit, you build one or more test source files into a class file. Similarly, in Android you + use the SDK's build tools to build one or more test source files into class files in an + Android test package. In JUnit, you use a test runner to execute test classes. In Android, you + use test tools to load the test package and the application under test, and the tools then + execute an Android-specific test runner. +</p> +<h2 id="TestProjects">Test Projects</h2> +<p> + Tests, like Android applications, are organized into projects. +</p> +<p> + A test project is a directory or Eclipse project in which you create the source code, manifest + file, and other files for a test package. The Android SDK contains tools for Eclipse with ADT + and for the command line that create and update test projects for you. The tools create the + directories you use for source code and resources and the manifest file for the test package. + The command-line tools also create the Ant build files you need. +</p> +<p> + You should always use Android tools to create a test project. Among other benefits, + the tools: +</p> + <ul> + <li> + Automatically set up your test package to use + {@link android.test.InstrumentationTestRunner} as the test case runner. You must use + <code>InstrumentationTestRunner</code> (or a subclass) to run JUnit tests. + </li> + <li> + Create an appropriate name for the test package. If the application + under test has a package name of <code>com.mydomain.myapp</code>, then the + Android tools set the test package name to <code>com.mydomain.myapp.test</code>. This + helps you identify their relationship, while preventing conflicts within the system. + </li> + <li> + Automatically create the proper build files, manifest file, and directory + structure for the test project. This helps you to build the test package without + having to modify build files and sets up the linkage between your test package and + the application under test. + The + </li> + </ul> +<p> + You can create a test project anywhere in your file system, but the best approach is to + add the test project so that its root directory <code>tests/</code> is at the same level + as the <code>src/</code> directory of the main application's project. This helps you find the + tests associated with an application. For example, if your application project's root directory + is <code>MyProject</code>, then you should use the following directory structure: +</p> +<pre class="classic no-pretty-print"> + MyProject/ + AndroidManifest.xml + res/ + ... (resources for main application) + src/ + ... (source code for main application) ... + tests/ + AndroidManifest.xml + res/ + ... (resources for tests) + src/ + ... (source code for tests) +</pre> +<h2 id="TestAPI">The Testing API</h2> +<p> + The Android testing API is based on the JUnit API and extended with a instrumentation + framework and Android-specific testing classes. +</p> +<h3 id="JUnit">JUnit</h3> +<p> + You can use the JUnit {@link junit.framework.TestCase TestCase} class to do unit testing on + a class that doesn't call Android APIs. <code>TestCase</code> is also the base class for + {@link android.test.AndroidTestCase}, which you can use to test Android-dependent objects. + Besides providing the JUnit framework, AndroidTestCase offers Android-specific setup, + teardown, and helper methods. +</p> +<p> + You use the JUnit {@link junit.framework.Assert} class to display test results. + The assert methods compare values you expect from a test to the actual results and + throw an exception if the comparison fails. Android also provides a class of assertions that + extend the possible types of comparisons, and another class of assertions for testing the UI. + These are described in more detail in the section <a href="#AssertionClasses"> + Assertion classes</a> +</p> +<p> + To learn more about JUnit, you can read the documentation on the + <a href="http://www.junit.org">junit.org</a> home page. + Note that the Android testing API supports JUnit 3 code style, but not JUnit 4. Also, you must + use Android's instrumented test runner {@link android.test.InstrumentationTestRunner} to run + your test case classes. This test runner is described in the + section <a href="#InstrumentationTestRunner">Running Tests</a>. +</p> +<h3 id="Instrumentation">Instrumentation</h3> +<p> + Android instrumentation is a set of control methods or "hooks" in the Android system. These hooks + control an Android component independently of its normal lifecycle. They also control how + Android loads applications. +</p> +<p> + Normally, an Android component runs in a lifecycle determined by the system. For example, an + Activity object's lifecycle starts when the Activity is activated by an Intent. The object's + <code>onCreate()</code> method is called, followed by <code>onResume()</code>. When the user + starts another application, the <code>onPause()</code> method is called. If the Activity + code calls the <code>finish()</code> method, the <code>onDestroy()</code> method is called. + The Android framework API does not provide a way for your code to invoke these callback + methods directly, but you can do so using instrumentation. +</p> +<p> + Also, the system runs all the components of an application into the same + process. You can allow some components, such as content providers, to run in a separate process, + but you can't force an application to run in the same process as another application that is + already running. +</p> +<p> + With Android instrumentation, though, you can invoke callback methods in your test code. + This allows you to run through the lifecycle of a component step by step, as if you were + debugging the component. The following test code snippet demonstrates how to use this to + test that an Activity saves and restores its state: +</p> +<a name="ActivitySnippet"></a> +<pre> + // Start the main activity of the application under test + mActivity = getActivity(); + + // Get a handle to the Activity object's main UI widget, a Spinner + mSpinner = (Spinner)mActivity.findViewById(com.android.example.spinner.R.id.Spinner01); + + // Set the Spinner to a known position + mActivity.setSpinnerPosition(TEST_STATE_DESTROY_POSITION); + + // Stop the activity - The onDestroy() method should save the state of the Spinner + mActivity.finish(); + + // Re-start the Activity - the onResume() method should restore the state of the Spinner + mActivity = getActivity(); + + // Get the Spinner's current position + int currentPosition = mActivity.getSpinnerPosition(); + + // Assert that the current position is the same as the starting position + assertEquals(TEST_STATE_DESTROY_POSITION, currentPosition); +</pre> +<p> + The key method used here is + {@link android.test.ActivityInstrumentationTestCase2#getActivity()}, which is a + part of the instrumentation API. The Activity under test is not started until you call this + method. You can set up the test fixture in advance, and then call this method to start the + Activity. +</p> +<p> + Also, instrumentation can load both a test package and the application under test into the + same process. Since the application components and their tests are in the same process, the + tests can invoke methods in the components, and modify and examine fields in the components. +</p> +<h3 id="TestCaseClasses">Test case classes</h3> +<p> + Android provides several test case classes that extend {@link junit.framework.TestCase} and + {@link junit.framework.Assert} with Android-specific setup, teardown, and helper methods. +</p> +<h4 id="AndroidTestCase">AndroidTestCase</h4> +<p> + A useful general test case class, especially if you are + just starting out with Android testing, is {@link android.test.AndroidTestCase}. It extends + both {@link junit.framework.TestCase} and {@link junit.framework.Assert}. It provides the + JUnit-standard <code>setUp()</code> and <code>tearDown()</code> methods, as well as + all of JUnit's Assert methods. In addition, it provides methods for testing permissions, and a + method that guards against memory leaks by clearing out certain class references. +</p> +<h4 id="ComponentTestCase">Component-specific test cases</h4> +<p> + A key feature of the Android testing framework is its component-specific test case classes. + These address specific component testing needs with methods for fixture setup and + teardown and component lifecycle control. They also provide methods for setting up mock objects. + These classes are described in the component-specific testing topics: +</p> +<ul> + <li> + <a href="{@docRoot}tools/testing/activity_testing.html">Activity Testing</a> + </li> + <li> + <a href="{@docRoot}tools/testing/contentprovider_testing.html"> + Content Provider Testing</a> + </li> + <li> + <a href="{@docRoot}tools/testing/service_testing.html">Service Testing</a> + </li> +</ul> +<p> + Android does not provide a separate test case class for BroadcastReceiver. Instead, test a + BroadcastReceiver by testing the component that sends it Intent objects, to verify that the + BroadcastReceiver responds correctly. +</p> +<h4 id="ApplicationTestCase">ApplicationTestCase</h4> +<p> + You use the {@link android.test.ApplicationTestCase} test case class to test the setup and + teardown of {@link android.app.Application} objects. These objects maintain the global state of + information that applies to all the components in an application package. The test case can + be useful in verifying that the <application> element in the manifest file is correctly + set up. Note, however, that this test case does not allow you to control testing of the + components within your application package. +</p> +<h4 id="InstrumentationTestCase">InstrumentationTestCase</h4> +<p> + If you want to use instrumentation methods in a test case class, you must use + {@link android.test.InstrumentationTestCase} or one of its subclasses. The + {@link android.app.Activity} test cases extend this base class with other functionality that + assists in Activity testing. +</p> + +<h3 id="AssertionClasses">Assertion classes</h3> +<p> + Because Android test case classes extend JUnit, you can use assertion methods to display the + results of tests. An assertion method compares an actual value returned by a test to an + expected value, and throws an AssertionException if the comparison test fails. Using assertions + is more convenient than doing logging, and provides better test performance. +</p> +<p> + Besides the JUnit {@link junit.framework.Assert} class methods, the testing API also provides + the {@link android.test.MoreAsserts} and {@link android.test.ViewAsserts} classes: +</p> +<ul> + <li> + {@link android.test.MoreAsserts} contains more powerful assertions such as + {@link android.test.MoreAsserts#assertContainsRegex}, which does regular expression + matching. + </li> + <li> + {@link android.test.ViewAsserts} contains useful assertions about Views. For example + it contains {@link android.test.ViewAsserts#assertHasScreenCoordinates} that tests if a View + has a particular X and Y position on the visible screen. These asserts simplify testing of + geometry and alignment in the UI. + </li> +</ul> +<h3 id="MockObjectClasses">Mock object classes</h3> +<p> + To facilitate dependency injection in testing, Android provides classes that create mock system + objects such as {@link android.content.Context} objects, + {@link android.content.ContentProvider} objects, {@link android.content.ContentResolver} + objects, and {@link android.app.Service} objects. Some test cases also provide mock + {@link android.content.Intent} objects. You use these mocks both to isolate tests + from the rest of the system and to facilitate dependency injection for testing. These classes + are found in the packages {@link android.test} and {@link android.test.mock}. +</p> +<p> + Mock objects isolate tests from a running system by stubbing out or overriding + normal operations. For example, a {@link android.test.mock.MockContentResolver} + replaces the normal resolver framework with its own local framework, which is isolated + from the rest of the system. MockContentResolver also stubs out the + {@link android.content.ContentResolver#notifyChange(Uri, ContentObserver, boolean)} method + so that observer objects outside the test environment are not accidentally triggered. +</p> +<p> + Mock object classes also facilitate dependency injection by providing a subclass of the + normal object that is non-functional except for overrides you define. For example, the + {@link android.test.mock.MockResources} object provides a subclass of + {@link android.content.res.Resources} in which all the methods throw Exceptions when called. + To use it, you override only those methods that must provide information. +</p> +<p> + These are the mock object classes available in Android: +</p> +<h4 id="SimpleMocks">Simple mock object classes</h4> +<p> + {@link android.test.mock.MockApplication}, {@link android.test.mock.MockContext}, + {@link android.test.mock.MockContentProvider}, {@link android.test.mock.MockCursor}, + {@link android.test.mock.MockDialogInterface}, {@link android.test.mock.MockPackageManager}, and + {@link android.test.mock.MockResources} provide a simple and useful mock strategy. They are + stubbed-out versions of the corresponding system object class, and all of their methods throw an + {@link java.lang.UnsupportedOperationException} exception if called. To use them, you override + the methods you need in order to provide mock dependencies. +</p> +<p class="Note"><strong>Note:</strong> + {@link android.test.mock.MockContentProvider} + and {@link android.test.mock.MockCursor} are new as of API level 8. +</p> +<h4 id="ResolverMocks">Resolver mock objects</h4> +<p> + {@link android.test.mock.MockContentResolver} provides isolated testing of content providers by + masking out the normal system resolver framework. Instead of looking in the system to find a + content provider given an authority string, MockContentResolver uses its own internal table. You + must explicitly add providers to this table using + {@link android.test.mock.MockContentResolver#addProvider(String,ContentProvider)}. +</p> +<p> + With this feature, you can associate a mock content provider with an authority. You can create + an instance of a real provider but use test data in it. You can even set the provider for an + authority to <code>null</code>. In effect, a MockContentResolver object isolates your test + from providers that contain real data. You can control the + function of the provider, and you can prevent your test from affecting real data. +</p> +<h3 id="ContextMocks">Contexts for testing</h3> +<p> + Android provides two Context classes that are useful for testing: +</p> +<ul> + <li> + {@link android.test.IsolatedContext} provides an isolated {@link android.content.Context}, + File, directory, and database operations that use this Context take place in a test area. + Though its functionality is limited, this Context has enough stub code to respond to + system calls. + <p> + This class allows you to test an application's data operations without affecting real + data that may be present on the device. + </p> + </li> + <li> + {@link android.test.RenamingDelegatingContext} provides a Context in which + most functions are handled by an existing {@link android.content.Context}, but + file and database operations are handled by a {@link android.test.IsolatedContext}. + The isolated part uses a test directory and creates special file and directory names. + You can control the naming yourself, or let the constructor determine it automatically. + <p> + This object provides a quick way to set up an isolated area for data operations, + while keeping normal functionality for all other Context operations. + </p> + </li> +</ul> +<h2 id="InstrumentationTestRunner">Running Tests</h2> +<p> + Test cases are run by a test runner class that loads the test case class, set ups, + runs, and tears down each test. An Android test runner must also be instrumented, so that + the system utility for starting applications can control how the test package + loads test cases and the application under test. You tell the Android platform + which instrumented test runner to use by setting a value in the test package's manifest file. +</p> +<p> + {@link android.test.InstrumentationTestRunner} is the primary Android test runner class. It + extends the JUnit test runner framework and is also instrumented. It can run any of the test + case classes provided by Android and supports all possible types of testing. +</p> +<p> + You specify <code>InstrumentationTestRunner</code> or a subclass in your test package's + manifest file, in the +<code><a href="{@docRoot}guide/topics/manifest/instrumentation-element.html"><instrumentation></a></code> + element. Also, <code>InstrumentationTestRunner</code> code resides + in the shared library <code>android.test.runner</code>, which is not normally linked to + Android code. To include it, you must specify it in a +<code><a href="{@docRoot}guide/topics/manifest/uses-library-element.html"><uses-library></a></code> + element. You do not have to set up these elements yourself. Both Eclipse with ADT and the + <code>android</code> command-line tool construct them automatically and add them to your + test package's manifest file. +</p> +<p class="Note"> + <strong>Note:</strong> If you use a test runner other than + <code>InstrumentationTestRunner</code>, you must change the <instrumentation> + element to point to the class you want to use. +</p> +<p> + To run {@link android.test.InstrumentationTestRunner}, you use internal system classes called by + Android tools. When you run a test in Eclipse with ADT, the classes are called automatically. + When you run a test from the command line, you run these classes with + <a href="{@docRoot}tools/help/adb.html">Android Debug Bridge (adb)</a>. +</p> +<p> + The system classes load and start the test package, kill any processes that + are running an instance of the application under test, and then load a new instance of the + application under test. They then pass control to + {@link android.test.InstrumentationTestRunner}, which runs + each test case class in the test package. You can also control which test cases and + methods are run using settings in Eclipse with ADT, or using flags with the command-line tools. +</p> +<p> + Neither the system classes nor {@link android.test.InstrumentationTestRunner} run + the application under test. Instead, the test case does this directly. It either calls methods + in the application under test, or it calls its own methods that trigger lifecycle events in + the application under test. The application is under the complete control of the test case, + which allows it to set up the test environment (the test fixture) before running a test. This + is demonstrated in the previous <a href="#ActivitySnippet">code snippet</a> that tests an + Activity that displays a Spinner widget. +</p> +<p> + To learn more about running tests, please read the topics + <a href="{@docRoot}tools/testing/testing_eclipse.html"> + Testing from Eclipse with ADT</a> or + <a href="{@docRoot}tools/testing/testing_otheride.html"> + Testing from Other IDEs</a>. +</p> +<h2 id="TestResults">Seeing Test Results</h2> +<p> + The Android testing framework returns test results back to the tool that started the test. + If you run a test in Eclipse with ADT, the results are displayed in a new JUnit view pane. If + you run a test from the command line, the results are displayed in <code>STDOUT</code>. In + both cases, you see a test summary that displays the name of each test case and method that + was run. You also see all the assertion failures that occurred. These include pointers to the + line in the test code where the failure occurred. Assertion failures also list the expected + value and actual value. +</p> +<p> + The test results have a format that is specific to the IDE that you are using. The test + results format for Eclipse with ADT is described in + <a href="{@docRoot}tools/testing/testing_eclipse.html#RunTestEclipse"> + Testing from Eclipse with ADT</a>. The test results format for tests run from the + command line is described in + <a href="{@docRoot}tools/testing/testing_otheride.html#RunTestsCommand"> + Testing from Other IDEs</a>. +</p> +<h2 id="Monkeys">monkey and monkeyrunner</h2> +<p> + The SDK provides two tools for functional-level application testing: +</p> + <ul> + <li> +The <a href="{@docRoot}tools/help/monkey.html">UI/Application Exerciser Monkey</a>, + usually called "monkey", is a command-line tool that sends pseudo-random streams of + keystrokes, touches, and gestures to a device. You run it with the + <a href="{@docRoot}tools/help/adb.html">Android Debug Bridge</a> (adb) tool. + You use it to stress-test your application and report back errors that are encountered. + You can repeat a stream of events by running the tool each time with the same random + number seed. + </li> + <li> + The <a href="{@docRoot}tools/help/monkeyrunner_concepts.html">monkeyrunner</a> tool + is an API and execution environment for test programs written in Python. The API + includes functions for connecting to a device, installing and uninstalling packages, + taking screenshots, comparing two images, and running a test package against an + application. Using the API, you can write a wide range of large, powerful, and complex + tests. You run programs that use the API with the <code>monkeyrunner</code> command-line + tool. + </li> + </ul> +<h2 id="PackageNames">Working With Package names</h2> +<p> + In the test environment, you work with both Android application package names and + Java package identifiers. Both use the same naming format, but they represent substantially + different entities. You need to know the difference to set up your tests correctly. +</p> +<p> + An Android package name is a unique system name for a <code>.apk</code> file, set by the + "android:package" attribute of the <manifest> element in the package's + manifest. The Android package name of your test package must be different from the + Android package name of the application under test. By default, Android tools create the + test package name by appending ".test" to the package name of the application under test. +</p> +<p> + The test package also uses an Android package name to target the application package it + tests. This is set in the "android:targetPackage" attribute of the + <instrumentation> element in the test package's manifest. +</p> +<p> + A Java package identifier applies to a source file. This package name reflects the directory + path of the source file. It also affects the visibility of classes and members to each other. +</p> +<p> + Android tools that create test projects set up an Android test package name for you. + From your input, the tools set up the test package name and the target package name for the + application under test. For these tools to work, the application project must already exist. +</p> +<p> + By default, these tools set the Java package identifier for the test class to be the same + as the Android package identifier. You may want to change this if you want to expose + members in the application under test by giving them package visibility. If you do this, + change only the Java package identifier, not the Android package names, and change only the + test case source files. Do not change the Java package name of the generated + <code>R.java</code> class in your test package, because it will then conflict with the + <code>R.java</code> class in the application under test. Do not change the Android package name + of your test package to be the same as the application it tests, because then their names + will no longer be unique in the system. +</p> +<h2 id="WhatToTest">What to Test</h2> +<p> + The topic <a href="{@docRoot}tools/testing/what_to_test.html">What To Test</a> + describes the key functionality you should test in an Android application, and the key + situations that might affect that functionality. +</p> +<p> + Most unit testing is specific to the Android component you are testing. + The topics <a href="{@docRoot}tools/testing/activity_testing.html">Activity Testing</a>, + <a href="{@docRoot}tools/testing/contentprovider_testing.html"> + Content Provider Testing</a>, and <a href="{@docRoot}tools/testing/service_testing.html"> + Service Testing</a> each have a section entitled "What To Test" that lists possible testing + areas. +</p> +<p> + When possible, you should run these tests on an actual device. If this is not possible, you can + use the <a href="{@docRoot}tools/devices/emulator.html">Android Emulator</a> with + Android Virtual Devices configured for the hardware, screens, and versions you want to test. +</p> +<h2 id="NextSteps">Next Steps</h2> +<p> + To learn how to set up and run tests in Eclipse, please refer to +<a href="{@docRoot}tools/testing/testing_eclipse.html">Testing from Eclipse with ADT</a>. + If you're not working in Eclipse, refer to +<a href="{@docRoot}tools/testing/testing_otheride.html">Testing from Other IDEs</a>. +</p> +<p> + If you want a step-by-step introduction to Android testing, try the + <a href="{@docRoot}tools/testing/activity_test.html">Activity Testing Tutorial</a>. +</p> diff --git a/docs/html/tools/testing/testing_eclipse.jd b/docs/html/tools/testing/testing_eclipse.jd new file mode 100644 index 0000000..7d3be47 --- /dev/null +++ b/docs/html/tools/testing/testing_eclipse.jd @@ -0,0 +1,535 @@ +page.title=Testing from Eclipse with ADT +parent.title=Testing +parent.link=index.html +@jd:body +<div id="qv-wrapper"> + <div id="qv"> + <h2>In this document</h2> + <ol> + <li><a href="#CreateTestProjectEclipse">Creating a Test Project</a></li> + <li><a href="#CreateTestAppEclipse">Creating a Test Package</a></li> + <li><a href="#RunTestEclipse">Running Tests</a></li> + </ol> + </div> +</div> +<p> + This topic explains how create and run tests of Android applications in Eclipse with ADT. + Before you read this topic, you should read about how to create an Android application with the + basic processes for creating and running applications with ADT, as described in + <a href="{@docRoot}tools/projects/projects-eclipse.html">Managing Projects from +Eclipse</a> + and <a href="{@docRoot}tools/building/building-eclipse.html">Building and Running +from Eclipse</a>. + You may also want to read + <a href="{@docRoot}tools/testing/testing_android.html">Testing Fundamentals</a>, + which provides an overview of the Android testing framework. +</p> +<p> + ADT provides several features that help you set up and manage your testing environment + effectively: +</p> + <ul> + <li> + It lets you quickly create a test project and link it to the application under test. + When it creates the test project, it automatically inserts the necessary + <code><instrumentation></code> element in the test package's manifest file. + </li> + <li> + It lets you quickly import the classes of the application under test, so that your + tests can inspect them. + </li> + <li> + It lets you create run configurations for your test package and include in + them flags that are passed to the Android testing framework. + </li> + <li> + It lets you run your test package without leaving Eclipse. ADT builds both the + application under test and the test package automatically, installs them if + necessary to your device or emulator, runs the test package, and displays the + results in a separate window in Eclipse. + </li> + </ul> +<p> + If you are not developing in Eclipse or you want to learn how to create and run tests from the + command line, see + <a href="{@docRoot}tools/testing/testing_otheride.html">Testing from Other IDEs</a>. +</p> +<h2 id="CreateTestProjectEclipse">Creating a Test Project</h2> +<p> + To set up a test environment for your Android application, you must first create a separate + project that holds the test code. The new project follows the directory structure + used for any Android application. It includes the same types of content and files, such as + source code, resources, a manifest file, and so forth. The test package you + create is connected to the application under test by an + <a href="{@docRoot}guide/topics/manifest/instrumentation-element.html"> + <code><instrumentation></code></a> element in its manifest file. +</p> +<p> + The <em>New Android Test Project</em> dialog makes it easy for you to generate a + new test project that has the proper structure, including the + <code><instrumentation></code> element in the manifest file. You can use the New + Android Test Project dialog to generate the test project at any time. The dialog appears + just after you create a new Android main application project, but you can also run it to + create a test project for a project that you created previously. +</p> +<p> + To create a test project in Eclipse with ADT: +</p> +<ol> + <li> + In Eclipse, select <strong>File > New > Other</strong>. This opens the <em>Select a + Wizard</em> dialog. + </li> + <li> + In the dialog, in the <em>Wizards</em> drop-down list, find the entry for Android, then + click the toggle to the left. Select <strong>Android Test Project</strong>, then at the + bottom of the dialog click <strong>Next</strong>. The <em>New Android Test Project</em> + wizard appears. + </li> + <li> + Next to <em>Test Project Name</em>, enter a name for the project. You may use any name, + but you may want to associate the name with the project name for the application under test. + One way to do this is to take the application's project name, append the string "Test" to + it, and then use this as the test package project name. + <p> + The name becomes part of the suggested project path, but you can change this in the + next step. + </p> + </li> + <li> + In the <em>Content</em> panel, examine the suggested path to the project. + If <em>Use default location</em> is set, then the wizard will suggest a path that is + a concatenation of the workspace path and the project name you entered. For example, + if your workspace path is <code>/usr/local/workspace</code> and your project name is + <code>MyTestApp</code>, then the wizard will suggest + <code>/usr/local/workspace/MyTestApp</code>. To enter your own + choice for a path, unselect <em>Use default location</em>, then enter or browse to the + path where you want your project. + <p> + To learn more about choosing the location of test projects, please read + <a href="{@docRoot}tools/testing/testing_android.html#TestProjectPaths"> + Testing Fundamentals</a>. + </p> + </li> + <li> + In the Test Target panel, set An Existing Android Project, click Browse, then select your + Android application from the list. You now see that the wizard has completed the Test + Target Package, Application Name, and Package Name fields for you (the latter two are in + the Properties panel). + </li> + <li> + In the Build Target panel, select the Android SDK platform that the application under test + uses. + </li> + <li> + Click Finish to complete the wizard. If Finish is disabled, look for error messages at the + top of the wizard dialog, and then fix any problems. + </li> +</ol> +<h2 id="CreateTestAppEclipse">Creating a Test Package</h2> +<p> + Once you have created a test project, you populate it with a test package. This package does not + require an Activity, although you can define one if you wish. Although your test package can + combine Activity classes, test case classes, or ordinary classes, your main test case + should extend one of the Android test case classes or JUnit classes, because these provide the + best testing features. +</p> +<p> + Test packages do not need to have an Android GUI. When you run the package in + Eclipse with ADT, its results appear in the JUnit view. Running tests and seeing the results is + described in more detail in the section <a href="#RunTestEclipse">Running Tests</a>. +</p> + +<p> + To create a test package, start with one of Android's test case classes defined in + {@link android.test android.test}. These extend the JUnit + {@link junit.framework.TestCase TestCase} class. The Android test classes for Activity objects + also provide instrumentation for testing an Activity. To learn more about test case + classes, please read the topic <a href="{@docRoot}tools/testing/testing_android.html"> + Testing Fundamentals</a>. +</p> +<p> + Before you create your test package, you choose the Java package identifier you want to use + for your test case classes and the Android package name you want to use. To learn more + about this, please read + <a href="{@docRoot}tools/testing/testing_android.html#PackageNames"> + Testing Fundamentals</a>. +</p> +<p> + To add a test case class to your project: +</p> +<ol> + <li> + In the <em>Project Explorer</em> tab, open your test project, then open the <em>src</em> + folder. + </li> + <li> + Find the Java package identifier set by the projection creation wizard. If you haven't + added classes yet, this node won't have any children, and its icon will not be filled in. + If you want to change the identifier value, right-click the identifier and select + <strong>Refactor</strong> > <strong>Rename</strong>, then enter the new name. + </li> + <li> + When you are ready, right-click the Java package identifier again and select + <strong>New</strong> > <strong>Class</strong>. This displays the <em>New Java Class</em> + dialog, with the <em>Source folder</em> and <em>Package</em> values already set. + </li> + <li> + In the <em>Name</em> field, enter a name for the test case class. One way to choose a + class name is to append the string "Test" to the class of the component you are testing. + For example, if you are testing the class MyAppActivity, your test case class + name would be MyAppActivityTest. Leave the modifiers set to <em>public</em>. + </li> + <li> + In the <em>Superclass</em> field, enter the name of the Android test case class you + are extending. You can also browse the available classes. + </li> + <li> + In <em>Which method stubs would you like to create?</em>, unset all the options, then + click <strong>Finish</strong>. You will set up the constructor manually. + </li> + <li> + Your new class appears in a new Java editor pane. + </li> +</ol> +<p> + You now have to ensure that the constructor is set up correctly. Create a constructor for your + class that has no arguments; this is required by JUnit. As the first statement in this + constructor, add a call to the base class' constructor. Each base test case class has its + own constructor signature. Refer to the class documentation in the documentation for + {@link android.test} for more information. +</p> +<p> + To control your test environment, you will want to override the <code>setUp()</code> and + <code>tearDown()</code> methods: +</p> +<ul> + <li> + <code>setUp()</code>: This method is invoked before any of the test methods in the class. + Use it to set up the environment for the test (the test fixture. You can use + <code>setUp()</code> to instantiate a new Intent with the action <code>ACTION_MAIN</code>. + You can then use this intent to start the Activity under test. + </li> + <li> + <code>tearDown()</code>: This method is invoked after all the test methods in the class. Use + it to do garbage collection and to reset the test fixture. + </li> +</ul> +<p> + Another useful convention is to add the method <code>testPreconditions()</code> to your test + class. Use this method to test that the application under test is initialized correctly. If this + test fails, you know that that the initial conditions were in error. When this happens, further + test results are suspect, regardless of whether or not the tests succeeded. +</p> +<p> + The Resources tab contains an + <a href="{@docRoot}tools/testing/activity_test.html">Activity Testing</a> + tutorial with more information about creating test classes and methods. +</p> +<h2 id="RunTestEclipse">Running Tests</h2> + <div class="sidebox-wrapper"> + <div class="sidebox"> + <h2>Running tests from the command line</h2> + <p> + If you've created your tests in Eclipse, you can still run your tests and test + suites by using command-line tools included with the Android SDK. You may want + to do this, for example, if you have a large number of tests to run, if you + have a large test case, or if you want a fine level of control over which + tests are run at a particular time. + </p> + <p> + To run tests created in Eclipse with ADT with command-line tools, you must first + install additional files into the test project using the <code>android</code> + tool's "create test-project" option. To see how to do this, read + <a href="{@docRoot}tools/testing/testing_otheride.html#CreateProject"> + Testing in Other IDEs</a>. + </p> + </div> + </div> +<p> + When you run a test package in Eclipse with ADT, the output appears in the Eclipse JUnit view. + You can run the entire test package or one test case class. To do run tests, Eclipse runs the + <code>adb</code> command for running a test package, and displays the output, so there is no + difference between running tests inside Eclipse and running them from the command line. +</p> +<p> + As with any other package, to run a test package in Eclipse with ADT you must either attach a + device to your computer or use the Android emulator. If you use the emulator, you must have an + Android Virtual Device (AVD) that uses the same target as the test package. +</p> +<p> + To run a test in Eclipse, you have two choices:</p> +<ul> + <li> + Run a test just as you run an application, by selecting + <strong>Run As... > Android JUnit Test</strong> from the project's context menu or + from the main menu's <strong>Run</strong> item. + </li> + <li> + Create an Eclipse run configuration for your test project. This is useful if you want + multiple test suites, each consisting of selected tests from the project. To run + a test suite, you run the test configuration. + <p> + Creating and running test configurations is described in the next section. + </p> + </li> +</ul> +<p> + To create and run a test suite using a run configuration: +</p> +<ol> + <li> + In the Package Explorer, select the test project, then from the main menu, select + <strong>Run > Run Configurations...</strong>. The Run Configurations dialog appears. + </li> + <li> + In the left-hand pane, find the Android JUnit Test entry. In the right-hand pane, click the + Test tab. The Name: text box shows the name of your project. The Test class: dropdown box + shows one of the test classes in your project. + </li> + <li> + To run one test class, click Run a single test, then enter your project name in the + Project: text box and the class name in the Test class: text box. + <p> + To run all the test classes, click Run all tests in the selected project or package, + then enter the project or package name in the text box. + </p> + </li> + <li> + Now click the Target tab. + <ul> + <li> + Optional: If you are using the emulator, click Automatic, then in the Android + Virtual Device (AVD) selection table, select an existing AVD. + </li> + <li> + In the Emulator Launch Parameters pane, set the Android emulator flags you want to + use. These are documented in the topic + <a href="{@docRoot}tools/help/emulator.html#startup-options"> + Android Emulator</a>. + </li> + </ul> + </li> + <li> + Click the Common tab. In the Save As pane, click Local to save this run configuration + locally, or click Shared to save it to another project. + </li> + <li> + Optional: Add the configuration to the Run toolbar and the <strong>Favorites</strong> + menu: in the Display in Favorites pane click the checkbox next to Run. + </li> + <li> + Optional: To add this configuration to the <strong>Debug</strong> menu and toolbar, click + the checkbox next to Debug. + </li> + <li> + To save your settings, click Close.<br/> + <p class="note"><strong>Note:</strong> + Although you can run the test immediately by clicking Run, you should save the test + first and then run it by selecting it from the Eclipse standard toolbar. + </p> + </li> + <li> + On the Eclipse standard toolbar, click the down arrow next to the green Run arrow. This + displays a menu of saved Run and Debug configurations. + </li> + <li> + Select the test run configuration you just created. The test starts. + </li> +</ol> +<p> + The progress of your test appears in the Console view as a series of messages. Each message is + preceded by a timestamp and the <code>.apk</code> filename to which it applies. For example, + this message appears when you run a test to the emulator, and the emulator is not yet started: +</p> +<div class="sidebox-wrapper"> + <div class="sidebox"> + <h2>Message Examples</h2> + <p> + The examples shown in this section come from the + <a href="{@docRoot}resources/samples/SpinnerTest/index.html">SpinnerTest</a> + sample test package, which tests the + <a href="{@docRoot}resources/samples/Spinner/index.html">Spinner</a> + sample application. This test package is also featured in the + <a href="{@docRoot}tools/testing/activity_test.html">Activity Testing</a> + tutorial. + </p> + </div> +</div> +<pre> + [<em>yyyy-mm-dd hh:mm:ss</em> - <em>testfile</em>] Waiting for HOME ('android.process.acore') to be launched... +</pre> +<p> + In the following description of these messages, <code><em>devicename</em></code> is the name of + the device or emulator you are using to run the test, and <code><em>port</em></code> is the + port number for the device. The name and port number are in the format used by the + <code><a href="{@docRoot}tools/help/adb.html#devicestatus">adb devices</a></code> + command. Also, <code><em>testfile</em></code> is the <code>.apk</code> filename of the test + package you are running, and <em>appfile</em> is the filename of the application under test. +</p> +<ul> + <li> + If you are using an emulator and you have not yet started it, then Eclipse + first starts the emulator. When this is complete, you see + the message: + <p> + <code>HOME is up on device '<em>devicename</em>-<em>port</em>'</code> + </p> + </li> + <li> + If you have not already installed your test package, then you see + the message: + <p> + <code>Uploading <em>testfile</em> onto device '<em>devicename</em>-<em>port</em>' + </code> + </p> + <p> + then the message <code>Installing <em>testfile</em></code>. + </p> + <p> + and finally the message <code>Success!</code> + </p> + </li> +</ul> +<p> + The following lines are an example of this message sequence: +</p> +<code> +[2010-07-01 12:44:40 - MyTest] HOME is up on device 'emulator-5554'<br> +[2010-07-01 12:44:40 - MyTest] Uploading MyTest.apk onto device 'emulator-5554'<br> +[2010-07-01 12:44:40 - MyTest] Installing MyTest.apk...<br> +[2010-07-01 12:44:49 - MyTest] Success!<br> +</code> +<br> +<ul> + <li> + Next, if you have not yet installed the application under test to the device or + emulator, you see the message + <p> + <code>Project dependency found, installing: <em>appfile</em></code> + </p> + <p> + then the message <code>Uploading <em>appfile</em></code> onto device + '<em>devicename</em>-<em>port</em>' + </p> + <p> + then the message <code>Installing <em>appfile</em></code> + </p> + <p> + and finally the message <code>Success!</code> + </p> + </li> +</ul> +<p> + The following lines are an example of this message sequence: +</p> +<code> +[2010-07-01 12:44:49 - MyTest] Project dependency found, installing: MyApp<br> +[2010-07-01 12:44:49 - MyApp] Uploading MyApp.apk onto device 'emulator-5554'<br> +[2010-07-01 12:44:49 - MyApp] Installing MyApp.apk...<br> +[2010-07-01 12:44:54 - MyApp] Success!<br> +</code> +<br> +<ul> + <li> + Next, you see the message + <code>Launching instrumentation <em>instrumentation_class</em> on device + <em>devicename</em>-<em>port</em></code> + <p> + <code>instrumentation_class</code> is the fully-qualified class name of the + instrumentation test runner you have specified (usually + {@link android.test.InstrumentationTestRunner}. + </p> + </li> + <li> + Next, as {@link android.test.InstrumentationTestRunner} builds a list of tests to run, + you see the message + <p> + <code>Collecting test information</code> + </p> + <p> + followed by + </p> + <p> + <code>Sending test information to Eclipse</code> + </p> + </li> + <li> + Finally, you see the message <code>Running tests</code>, which indicates that your tests + are running. At this point, you should start seeing the test results in the JUnit view. + When the tests are finished, you see the console message <code>Test run complete</code>. + This indicates that your tests are finished. + </li> +</ul> +<p> + The following lines are an example of this message sequence: +</p> +<code> +[2010-01-01 12:45:02 - MyTest] Launching instrumentation android.test.InstrumentationTestRunner on device emulator-5554<br> +[2010-01-01 12:45:02 - MyTest] Collecting test information<br> +[2010-01-01 12:45:02 - MyTest] Sending test information to Eclipse<br> +[2010-01-01 12:45:02 - MyTest] Running tests...<br> +[2010-01-01 12:45:22 - MyTest] Test run complete<br> +</code> +<br> +<p> + The test results appear in the JUnit view. This is divided into an upper summary pane, + and a lower stack trace pane. +</p> +<p> + The upper pane contains test information. In the pane's header, you see the following + information: +</p> +<ul> + <li> + Total time elapsed for the test package (labeled Finished after <em>x</em> seconds). + </li> + <li> + Number of runs (Runs:) - the number of tests in the entire test class. + </li> + <li> + Number of errors (Errors:) - the number of program errors and exceptions encountered + during the test run. + </li> + <li> + Number of failures (Failures:) - the number of test failures encountered during the test + run. This is the number of assertion failures. A test can fail even if the program does + not encounter an error. + </li> + <li> + A progress bar. The progress bar extends from left to right as the tests run. If all the + tests succeed, the bar remains green. If a test fails, the bar turns from green to red. + </li> +</ul> +<p> + The body of the upper pane contains the details of the test run. For each test case class + that was run, you see a line with the class name. To look at the results for the individual + test methods in that class, you click the left arrow to expand the line. You now see a + line for each test method in the class, and to its right the time it took to run. + If you double-click the method name, Eclipse opens the test class source in an editor view + pane and moves the focus to the first line of the test method. +</p> +<p> + The results of a successful test are shown in figure 1. +</p> +<a href="{@docRoot}images/testing/eclipse_test_results.png"> + <img src="{@docRoot}images/testing/eclipse_test_results.png" + alt="Messages for a successful test" height="327px" id="TestResults"/> +</a> +<p class="img-caption"> + <strong>Figure 1.</strong> Messages for a successful test. +</p> +<p> + The lower pane is for stack traces. If you highlight a failed test in the upper pane, the + lower pane contains a stack trace for the test. If a line corresponds to a point in your + test code, you can double-click it to display the code in an editor view pane, with the + line highlighted. For a successful test, the lower pane is empty. +</p> +<p>The results of a failed test are shown in figure 2.</p> +<a href="{@docRoot}images/testing/eclipse_test_run_failure.png"> + <img src="{@docRoot}images/testing/eclipse_test_run_failure.png" + alt="" height="372px" id="TestRun"/> +</a> +<p class="img-caption"> + <strong>Figure 2.</strong> Messages for a test failure. +</p> diff --git a/docs/html/tools/testing/testing_otheride.jd b/docs/html/tools/testing/testing_otheride.jd new file mode 100644 index 0000000..0678f52 --- /dev/null +++ b/docs/html/tools/testing/testing_otheride.jd @@ -0,0 +1,690 @@ +page.title=Testing from Other IDEs +parent.title=Testing +parent.link=index.html +@jd:body + +<div id="qv-wrapper"> + <div id="qv"> + <h2>In this document</h2> + <ol> + <li> + <a href="#CreateTestProjectCommand">Working with Test Projects</a> + <ol> + <li> + <a href="#CreateTestProject">Creating a test project</a> + </li> + <li> + <a href="#UpdateTestProject">Updating a test project</a> + </li> + </ol> + </li> + <li> + <a href="#CreateTestApp">Creating a Test Package</a> + </li> + <li> + <a href="#RunTestsCommand">Running Tests</a> + <ol> + <li> + <a href="#RunTestsAnt">Quick build and run with Ant</a> + </li> + <li> + <a href="#RunTestsDevice">Running tests on a device or emulator</a> + </li> + </ol> + </li> + <li> + <a href="#AMSyntax">Using the Instrument Command</a> + <ol> + <li> + <a href="#AMOptionsSyntax">Instrument options</a> + </li> + <li> + <a href="#RunTestExamples">Instrument examples</a> + </li> + </ol> + </li> + </ol> + <h2>See Also</h2> + <ol> + <li> + <a href="{@docRoot}tools/testing/testing_android.html"> + Testing Fundamentals</a> + </li> + <li> + <a href="{@docRoot}tools/help/adb.html">Android Debug Bridge</a> + </li> + </ol> + </div> +</div> +<p> + This document describes how to create and run tests directly from the command line. + You can use the techniques described here if you are developing in an IDE other than Eclipse + or if you prefer to work from the command line. This document assumes that you already know how + to create a Android application in your programming environment. Before you start this + document, you should read the topic + <a href="{@docRoot}tools/testing/testing_android.html">Testing Fundamentals</a>, + which provides an overview of Android testing. +</p> +<p> + If you are developing in Eclipse with ADT, you can set up and run your tests + directly in Eclipse. For more information, please read + <a href="{@docRoot}tools/testing/testing_eclipse.html"> + Testing from Eclipse with ADT</a>. +</p> +<h2 id="CreateTestProjectCommand">Working with Test Projects</h2> +<p> + You use the <code>android</code> tool to create test projects. + You also use <code>android</code> to convert existing test code into an Android test project, + or to add the <code>run-tests</code> Ant target to an existing Android test project. + These operations are described in more detail in the section <a href="#UpdateTestProject"> + Updating a test project</a>. The <code>run-tests</code> target is described in + <a href="#RunTestsAnt">Quick build and run with Ant</a>. +</p> +<h3 id="CreateTestProject">Creating a test project</h3> +<p> + To create a test project with the <code>android</code> tool, enter: +</p> +<pre> +android create test-project -m <main_path> -n <project_name> -p <test_path> +</pre> +<p> + You must supply all the flags. The following table explains them in detail: +</p> +<table> + <tr> + <th>Flag</th> + <th>Value</th> + <th>Description</th> + </tr> + <tr> + <td><code>-m, --main</code></td> + <td> + Path to the project of the application under test, relative to the test package + directory. + </td> + <td> + For example, if the application under test is in <code>source/HelloAndroid</code>, and + you want to create the test project in <code>source/HelloAndroidTest</code>, then the + value of <code>--main</code> should be <code>../HelloAndroid</code>. + <p> + To learn more about choosing the location of test projects, please read + <a href="{@docRoot}tools/testing/testing_android.html#TestProjects"> + Testing Fundamentals</a>. + </p> + </td> + </tr> + <tr> + <td><code>-n, --name</code></td> + <td>Name that you want to give the test project.</td> + <td> </td> + </tr> + <tr> + <td><code>-p, --path</code></td> + <td>Directory in which you want to create the new test project.</td> + <td> + The <code>android</code> tool creates the test project files and directory structure + in this directory. If the directory does not exist, <code>android</code> creates it. + </td> + </tr> +</table> +<p> + If the operation is successful, <code>android</code> lists to STDOUT the names of the files + and directories it has created. +</p> +<p> + This creates a new test project with the appropriate directories and build files. The directory + structure and build file contents are identical to those in a regular Android application + project. They are described in detail in the topic + <a href="{@docRoot}tools/projects/index.html">Managing Projects</a>. +</p> +<p> + The operation also creates an <code>AndroidManifest.xml</code> file with instrumentation + information. When you run the test, Android uses this information to load the application you + are testing and control it with instrumentation. +</p> +<p> + For example, suppose you create a project in the directory <code>~/source/HelloAndroid</code>, +with the package name <code>com.example.helloandroid</code>, + and the activity name <code>HelloAndroid</code>. You can to create the test for this in + <code>~/source/HelloAndroidTest</code>. To do so, you enter: +</p> +<pre> +$ cd ~/source +$ android create test-project -m ../HelloAndroid -n HelloAndroidTest -p HelloAndroidTest +</pre> +<p> + This creates a directory called <code>~/src/HelloAndroidTest</code>. In the new directory you + see the file <code>AndroidManifest.xml</code>. This file contains the following + instrumentation-related elements and attributes: +</p> +<ul> + <li> + <code><application></code>: to contain the + <code><uses-library></code> element. + </li> + <li> + <code><uses-library android:name="android.test.runner"</code>: + specifies this testing application uses the <code>android.test.runner</code> library. + </li> + <li> + <code><instrumentation></code>: contains attributes that control Android + instrumentation. The attributes are: + <ul> + <li> + <code>android:name="android.test.InstrumentationTestRunner"</code>: + {@link android.test.InstrumentationTestRunner} runs test cases. It extends both + JUnit test case runner classes and Android instrumentation classes. + </li> + <li> + <code>android:targetPackage="com.example.helloandroid"</code>: specifies + that the tests in HelloAndroidTest should be run against the application with the + <em>Android</em> package name <code>com.example.helloandroid</code>. + </li> + <li> + <code>android:label="Tests for .HelloAndroid"</code>: specifies a + user-readable label for the instrumentation class. By default, + the <code>android</code> tool gives it the value "Tests for " plus + the name of the main Activity of the application under test. + </li> + </ul> + </li> +</ul> +<h3 id="UpdateTestProject">Updating a test project</h3> +<p> + You use the <code>android</code> tool when you need to change the path to the + project of the application under test. If you are changing an existing test project created in + Eclipse with ADT so that you can also build and run it from the command line, you must use the + "create" operation. See the section <a href="#CreateTestProject">Creating a test project</a>. +</p> +<p class="note"> + <strong>Note:</strong> If you change the Android package name of the application under test, + you must <em>manually</em> change the value of the <code><android:targetPackage></code> + attribute within the <code>AndroidManifest.xml</code> file of the test package. + Running <code>android update test-project</code> does not do this. +</p> +<p> + To update a test project with the <code>android</code> tool, enter: +</p> +<pre>android update test-project -m <main_path> -p <test_path></pre> + +<table> + <tr> + <th>Flag</th> + <th>Value</th> + <th>Description</th> + </tr> + <tr> + <td><code>-m, --main</code></td> + <td>The path to the project of the application under test, relative to the test project</td> + <td> + For example, if the application under test is in <code>source/HelloAndroid</code>, and + the test project is in <code>source/HelloAndroidTest</code>, then the value for + <code>--main</code> is <code>../HelloAndroid</code>. + </td> + </tr> + <tr> + <td><code>-p, --path</code></td> + <td>The of the test project.</td> + <td> + For example, if the test project is in <code>source/HelloAndroidTest</code>, then the + value for <code>--path</code> is <code>HelloAndroidTest</code>. + </td> + </tr> +</table> +<p> + If the operation is successful, <code>android</code> lists to STDOUT the names of the files + and directories it has created. +</p> +<h2 id="CreateTestApp">Creating a Test Package</h2> +<p> + Once you have created a test project, you populate it with a test package. + The application does not require an {@link android.app.Activity Activity}, + although you can define one if you wish. Although your test package can + combine Activities, Android test class extensions, JUnit extensions, or + ordinary classes, you should extend one of the Android test classes or JUnit classes, + because these provide the best testing features. +</p> +<p> + If you run your tests with {@link android.test.InstrumentationTestRunner} + (or a related test runner), then it will run all the methods in each class. You can modify + this behavior by using the {@link junit.framework.TestSuite TestSuite} class. +</p> + +<p> + To create a test package, start with one of Android's test classes in the Java package + {@link android.test android.test}. These extend the JUnit + {@link junit.framework.TestCase TestCase} class. With a few exceptions, the Android test + classes also provide instrumentation for testing. +</p> +<p> + For test classes that extend {@link junit.framework.TestCase TestCase}, you probably want to + override the <code>setUp()</code> and <code>tearDown()</code> methods: +</p> +<ul> + <li> + <code>setUp()</code>: This method is invoked before any of the test methods in the class. + Use it to set up the environment for the test. You can use <code>setUp()</code> + to instantiate a new <code>Intent</code> object with the action <code>ACTION_MAIN</code>. + You can then use this intent to start the Activity under test. + <p class="note"> + <strong>Note:</strong> If you override this method, call + <code>super.setUp()</code> as the first statement in your code. + </p> + </li> + <li> + <code>tearDown()</code>: This method is invoked after all the test methods in the class. Use + it to do garbage collection and re-setting before moving on to the next set of tests. + <p class="note"><strong>Note:</strong> If you override this method, you must call + <code>super.tearDown()</code> as the <em>last</em> statement in your code.</p> + </li> +</ul> +<p> + Another useful convention is to add the method <code>testPreConditions()</code> to your test + class. Use this method to test that the application under test is initialized correctly. If this + test fails, you know that that the initial conditions were in error. When this happens, further + test results are suspect, regardless of whether or not the tests succeeded. +</p> +<p> + To learn more about creating test packages, see the topic <a + href="{@docRoot}tools/testing/testing_android.html">Testing Fundamentals</a>, + which provides an overview of Android testing. If you prefer to follow a tutorial, + try the <a href="{@docRoot}tools/testing/activity_test.html">Activity Testing</a> + tutorial, which leads you through the creation of tests for an actual Android application. +</p> +<h2 id="RunTestsCommand">Running Tests</h2> +<p> + You run tests from the command line, either with Ant or with an + <a href="{@docRoot}tools/help/adb.html"> + Android Debug Bridge (adb)</a> shell. +</p> +<h3 id="RunTestsAnt">Quick build and run with Ant</h3> +<p> + You can use Ant to run all the tests in your test project, using the target + <code>run-tests</code>, which is created automatically when you create a test project with + the <code>android</code> tool. +</p> +<p> + This target re-builds your main project and test project if necessary, installs the test + application to the current AVD or device, and then runs all the test classes in the test + application. The results are directed to <code>STDOUT</code>. +</p> +<p> + You can update an existing test project to use this feature. To do this, use the + <code>android</code> tool with the <code>update test-project</code> option. This is described + in the section <a href="#UpdateTestProject">Updating a test project</a>. +</p> +<h3 id="RunTestsDevice">Running tests on a device or emulator</h3> +<p> + When you run tests from the command line with + <a href="{@docRoot}tools/help/adb.html"> + Android Debug Bridge (adb)</a>, you get more options for choosing the tests + to run than with any other method. You can select individual test methods, filter tests + according to their annotation, or specify testing options. Since the test run is controlled + entirely from a command line, you can customize your testing with shell scripts in various ways. +</p> +<p> + To run a test from the command line, you run <code>adb shell</code> to start a command-line + shell on your device or emulator, and then in the shell run the <code>am instrument</code> + command. You control <code>am</code> and your tests with command-line flags. +</p> +<p> + As a shortcut, you can start an <code>adb</code> shell, call <code>am instrument</code>, and + specify command-line flags all on one input line. The shell opens on the device or emulator, + runs your tests, produces output, and then returns to the command line on your computer. +</p> +<p> + To run a test with <code>am instrument</code>: +</p> +<ol> + <li> + If necessary, rebuild your main application and test package. + </li> + <li> + Install your test package and main application Android package files + (<code>.apk</code> files) to your current Android device or emulator</li> + <li> + At the command line, enter: +<pre> +$ adb shell am instrument -w <test_package_name>/<runner_class> +</pre> + <p> + where <code><test_package_name></code> is the Android package name of your test + application, and <code><runner_class></code> is the name of the Android test + runner class you are using. The Android package name is the value of the + <code>package</code> attribute of the <code>manifest</code> element in the manifest file + (<code>AndroidManifest.xml</code>) of your test package. The Android test runner + class is usually {@link android.test.InstrumentationTestRunner}. + </p> + <p> + Your test results appear in <code>STDOUT</code>. + </p> + </li> +</ol> +<p> + This operation starts an <code>adb</code> shell, then runs <code>am instrument</code> + with the specified parameters. This particular form of the command will run all of the tests + in your test package. You can control this behavior with flags that you pass to + <code>am instrument</code>. These flags are described in the next section. +</p> +<h2 id="AMSyntax">Using the am instrument Command</h2> +<p> + The general syntax of the <code>am instrument</code> command is: +</p> +<pre> + am instrument [flags] <test_package>/<runner_class> +</pre> +<p> + The main input parameters to <code>am instrument</code> are described in the following table: +</p> +<table> + <tr> + <th> + Parameter + </th> + <th> + Value + </th> + <th> + Description + </th> + </tr> + <tr> + <td> + <code><test_package></code> + </td> + <td> + The Android package name of the test package. + </td> + <td> + The value of the <code>package</code> attribute of the <code>manifest</code> + element in the test package's manifest file. + </td> + </tr> + <tr> + <td> + <code><runner_class></code> + </td> + <td> + The class name of the instrumented test runner you are using. + </td> + <td> + This is usually {@link android.test.InstrumentationTestRunner}. + </td> + </tr> +</table> +<p> + The flags for <code>am instrument</code> are described in the following table: +</p> +<table> + <tr> + <th> + Flag + </th> + <th> + Value + </th> + <th> + Description + </th> + </tr> + <tr> + <td> + <code>-w</code> + </td> + <td> + (none) + </td> + <td> + Forces <code>am instrument</code> to wait until the instrumentation terminates + before terminating itself. The net effect is to keep the shell open until the tests + have finished. This flag is not required, but if you do not use it, you will not + see the results of your tests. + </td> + </tr> + <tr> + <td> + <code>-r</code> + </td> + <td> + (none) + </td> + <td> + Outputs results in raw format. Use this flag when you want to collect + performance measurements, so that they are not formatted as test results. This flag is + designed for use with the flag <code>-e perf true</code> (documented in the section + <a href="#AMOptionsSyntax">Instrument options</a>). + </td> + </tr> + <tr> + <td> + <code>-e</code> + </td> + <td> + <test_options> + </td> + <td> + Provides testing options as key-value pairs. The + <code>am instrument</code> tool passes these to the specified instrumentation class + via its <code>onCreate()</code> method. You can specify multiple occurrences of + <code>-e <test_options></code>. The keys and values are described in the + section <a href="#AMOptionsSyntax">am instrument options</a>. + <p> + The only instrumentation class that uses these key-value pairs is + {@link android.test.InstrumentationTestRunner} (or a subclass). Using them with + any other class has no effect. + </p> + </td> + </tr> +</table> + +<h3 id="AMOptionsSyntax">am instrument options</h3> +<p> + The <code>am instrument</code> tool passes testing options to + <code>InstrumentationTestRunner</code> or a subclass in the form of key-value pairs, + using the <code>-e</code> flag, with this syntax: +</p> +<pre> + -e <key> <value> +</pre> +<p> + Some keys accept multiple values. You specify multiple values in a comma-separated list. + For example, this invocation of <code>InstrumentationTestRunner</code> provides multiple + values for the <code>package</code> key: +</p> +<pre> +$ adb shell am instrument -w -e package com.android.test.package1,com.android.test.package2 \ +> com.android.test/android.test.InstrumentationTestRunner +</pre> +<p> + The following table describes the key-value pairs and their result. Please review the + <strong>Usage Notes</strong> following the table. +</p> +<table> + <tr> + <th>Key</th> + <th>Value</th> + <th>Description</th> + </tr> + <tr> + <td> + <code>package</code> + </td> + <td> + <Java_package_name> + </td> + <td> + The fully-qualified <em>Java</em> package name for one of the packages in the test + application. Any test case class that uses this package name is executed. Notice that + this is not an <em>Android</em> package name; a test package has a single + Android package name but may have several Java packages within it. + </td> + </tr> + <tr> + <td rowspan="2"><code>class</code></td> + <td><class_name></td> + <td> + The fully-qualified Java class name for one of the test case classes. Only this test + case class is executed. + </td> + </tr> + <tr> + <td><class_name><strong>#</strong>method name</td> + <td> + A fully-qualified test case class name, and one of its methods. Only this method is + executed. Note the hash mark (#) between the class name and the method name. + </td> + </tr> + <tr> + <td><code>func</code></td> + <td><code>true</code></td> + <td> + Runs all test classes that extend {@link android.test.InstrumentationTestCase}. + </td> + </tr> + <tr> + <td><code>unit</code></td> + <td><code>true</code></td> + <td> + Runs all test classes that do <em>not</em> extend either + {@link android.test.InstrumentationTestCase} or + {@link android.test.PerformanceTestCase}. + </td> + </tr> + <tr> + <td><code>size</code></td> + <td> + [<code>small</code> | <code>medium</code> | <code>large</code>] + </td> + <td> + Runs a test method annotated by size. The annotations are <code>@SmallTest</code>, + <code>@MediumTest</code>, and <code>@LargeTest</code>. + </td> + </tr> + <tr> + <td><code>perf</code></td> + <td><code>true</code></td> + <td> + Runs all test classes that implement {@link android.test.PerformanceTestCase}. + When you use this option, also specify the <code>-r</code> flag for + <code>am instrument</code>, so that the output is kept in raw format and not + re-formatted as test results. + </td> + </tr> + <tr> + <td><code>debug</code></td> + <td><code>true</code></td> + <td> + Runs tests in debug mode. + </td> + </tr> + <tr> + <td><code>log</code></td> + <td><code>true</code></td> + <td> + Loads and logs all specified tests, but does not run them. The test + information appears in <code>STDOUT</code>. Use this to verify combinations of other + filters and test specifications. + </td> + </tr> + <tr> + <td><code>emma</code></td> + <td><code>true</code></td> + <td> + Runs an EMMA code coverage analysis and writes the output to + <code>/data//coverage.ec</code> on the device. To override the file location, use the + <code>coverageFile</code> key that is described in the following entry. + <p class="note"> + <strong>Note:</strong> This option requires an EMMA-instrumented build of the test + application, which you can generate with the <code>coverage</code> target. + </p> + </td> + </tr> + <tr> + <td><code>coverageFile</code></td> + <td><code><filename></code></td> + <td> + Overrides the default location of the EMMA coverage file on the device. Specify this + value as a path and filename in UNIX format. The default filename is described in the + entry for the <code>emma</code> key. + </td> + </tr> +</table> +<strong><code>-e</code> Flag Usage Notes</strong> +<ul> + <li> + <code>am instrument</code> invokes + {@link android.test.InstrumentationTestRunner#onCreate(Bundle)} + with a {@link android.os.Bundle} containing the key-value pairs. + </li> + <li> + The <code>package</code> key takes precedence over the <code>class</code> key. If you + specifiy a package, and then separately specify a class within that package, Android + will run all the tests in the package and ignore the <code>class</code> key. + </li> + <li> + The <code>func</code> key and <code>unit</code> key are mutually exclusive. + </li> +</ul> +<h3 id="RunTestExamples">Usage examples</h3> +<p> +The following sections provide examples of using <code>am instrument</code> to run tests. +They are based on the following structure:</p> +<ul> + <li> + The test package has the Android package name <code>com.android.demo.app.tests</code> + </li> + <li> + There are three test classes: + <ul> + <li> + <code>UnitTests</code>, which contains the methods + <code>testPermissions</code> and <code>testSaveState</code>. + </li> + <li> + <code>FunctionTests</code>, which contains the methods + <code>testCamera</code>, <code>testXVGA</code>, and <code>testHardKeyboard</code>. + </li> + <li> + <code>IntegrationTests</code>, + which contains the method <code>testActivityProvider</code>. + </li> + </ul> + </li> + <li> + The test runner is {@link android.test.InstrumentationTestRunner}. + </li> +</ul> +<h4>Running the entire test package</h4> +<p> + To run all of the test classes in the test package, enter: +</p> +<pre> +$ adb shell am instrument -w com.android.demo.app.tests/android.test.InstrumentationTestRunner +</pre> +<h4>Running all tests in a test case class</h4> +<p> + To run all of the tests in the class <code>UnitTests</code>, enter: +</p> +<pre> +$ adb shell am instrument -w \ +> -e class com.android.demo.app.tests.UnitTests \ +> com.android.demo.app.tests/android.test.InstrumentationTestRunner +</pre> +<p> + <code>am instrument</code> gets the value of the <code>-e</code> flag, detects the + <code>class</code> keyword, and runs all the methods in the <code>UnitTests</code> class. +</p> +<h4>Selecting a subset of tests</h4> +<p> + To run all of the tests in <code>UnitTests</code>, and the <code>testCamera</code> method in + <code>FunctionTests</code>, enter: +</p> +<pre> +$ adb shell am instrument -w \ +> -e class com.android.demo.app.tests.UnitTests,com.android.demo.app.tests.FunctionTests#testCamera \ +> com.android.demo.app.tests/android.test.InstrumentationTestRunner +</pre> +<p> + You can find more examples of the command in the documentation for + {@link android.test.InstrumentationTestRunner}. +</p> diff --git a/docs/html/tools/testing/what_to_test.jd b/docs/html/tools/testing/what_to_test.jd new file mode 100644 index 0000000..77ae211 --- /dev/null +++ b/docs/html/tools/testing/what_to_test.jd @@ -0,0 +1,86 @@ +page.title=What To Test +parent.title=Testing +parent.link=index.html +@jd:body +<p> + As you develop Android applications, knowing what to test is as important as knowing how to + test. This document lists some most common Android-related situations that you should consider + when you test, even at the unit test level. This is not an exhaustive list, and you consult the + documentation for the features that you use for more ideas. The + <a href="http://groups.google.com/group/android-developers">android-developers</a> Google Groups + site is another resource for information about testing. +</p> +<h2 id="Tests">Ideas for Testing</h2> +<p> + The following sections are organized by behaviors or situations that you should test. Each + section contains a scenario that further illustrates the situation and the test or tests you + should do. +</p> +<h4>Change in orientation</h4> +<p> + For devices that support multiple orientations, Android detects a change in orientation when + the user turns the device so that the display is "landscape" (long edge is horizontal) instead + of "portrait" (long edge is vertical). +</p> +<p> + When Android detects a change in orientation, its default behavior is to destroy and then + re-start the foreground Activity. You should consider testing the following: +</p> +<ul> + <li> + Is the screen re-drawn correctly? Any custom UI code you have should handle changes in the + orientation. + </li> + <li> + Does the application maintain its state? The Activity should not lose anything that the + user has already entered into the UI. The application should not "forget" its place in the + current transaction. + </li> +</ul> +<h4>Change in configuration</h4> +<p> + A situation that is more general than a change in orientation is a change in the device's + configuration, such as a change in the availability of a keyboard or a change in system + language. +</p> +<p> + A change in configuration also triggers the default behavior of destroying and then restarting + the foreground Activity. Besides testing that the application maintains the UI and its + transaction state, you should also test that the application updates itself to respond + correctly to the new configuration. +</p> +<h4>Battery life</h4> +<p> + Mobile devices primarily run on battery power. A device has finite "battery budget", and when it + is gone, the device is useless until it is recharged. You need to write your application to + minimize battery usage, you need to test its battery performance, and you need to test the + methods that manage battery usage. +</p> +<p> + Techniques for minimizing battery usage were presented at the 2010 Google I/O conference in the + presentation + <a href="http://code.google.com/events/io/2009/sessions/CodingLifeBatteryLife.html"> + Coding for Life -- Battery Life, That Is</a>. This presentation describes the impact on battery + life of various operations, and the ways you can design your application to minimize these + impacts. When you code your application to reduce battery usage, you also write the + appropriate unit tests. +</p> +<h4>Dependence on external resources</h4> +<p> + If your application depends on network access, SMS, Bluetooth, or GPS, then you should + test what happens when the resource or resources are not available. +</p> +<p> + For example, if your application uses the network,it can notify the user if access is + unavailable, or disable network-related features, or do both. For GPS, it can switch to + IP-based location awareness. It can also wait for WiFi access before doing large data transfers, + since WiFi transfers maximize battery usage compared to transfers over 3G or EDGE. +</p> +<p> + You can use the emulator to test network access and bandwidth. To learn more, please see + <a href="{@docRoot}tools/help/emulator.html#netspeed">Network Speed Emulation</a>. + To test GPS, you can use the emulator console and {@link android.location.LocationManager}. To + learn more about the emulator console, please see + <a href="{@docRoot}tools/help/emulator.html#console"> + Using the Emulator Console</a>. +</p> |