docs: M Preview Testing Guide

A general overview for developers on what to test during the  M Preview

Change-Id: Idbc080482afdaf08c665e4ff2b7e62a84f9b398a

1 files changed, 176 insertions, 0 deletions

diff --git a/docs/html/preview/testing/guide.jd b/docs/html/preview/testing/guide.jd
new file mode 100644
index 0000000..1879268
--- /dev/null
+++ b/docs/html/preview/testing/guide.jd
@@ -0,0 +1,176 @@
+page.title=Testing Guide
+page.image=images/cards/card-set-up_16-9_2x.png
+
+@jd:body
+
+<div id="qv-wrapper">
+  <div id="qv">
+    <h2>In this document</h2>
+      <ol>
+        <li><a href="#runtime-permissions">Testing Runtime Permissions</a></li>
+        <li><a href="#doze-standby">Testing Doze and App Standby</a></li>
+      </ol>
+  </div>
+</div>
+
+<p>
+  The Android M Developer Preview gives you an opportunity to ensure your apps work with the next
+  version of the platform. This preview includes a number of APIs and behavior changes that can
+  impact your app, as described in the <a href="{@docRoot}preview/api-overview.html">API
+  Overview</a> and <a href="{@docRoot}preview/api-changes.html">Behavior Changes</a>. In testing
+  your app with the preview, there are some specific system changes that you should focus on to
+  ensure that users have a good experience.
+</p>
+
+<p>
+  This guide describes the what and how to test preview features with your app. You should
+  prioritize testing of these specific preview features, due to their high potential impact on your
+  app's behavior:
+</p>
+
+<ul>
+  <li><a href="#runtime-permissions">Runtime Permissions</a>
+  </li>
+  <li><a href="#doze-mode">Doze and App Standby</a>
+  </li>
+</ul>
+
+<p>
+  For more information about how to set up devices or virtual devices with a preview system image
+  for testing, see <a href="{@docRoot}preview/setup-sdk.html">Set up the Preview SDK</a>.
+</p>
+
+
+<h2 id="runtime-permissions">Testing Runtime Permissions</h2>
+
+<p>
+  The <a href="{@docRoot}preview/features/runtime-permissions.html">Runtime Permissions</a> feature
+  changes the way that permissions are allocated to your app by the user. Instead of granting all
+  permissions during the install procedure, your app must ask the user for individual permissions
+  at runtime. For users this behavior provides more granular control over each app’s activities, as
+  well as better context for understanding why the app is requesting a specific permission. Users
+  can grant or revoke the permissions granted to an app individually at any time. This feature of
+  the preview is most likely to have an impact on your app's behavior and may prevent some of your
+  app features from working, or they may work in a degraded state.
+</p>
+
+<p>
+  This change that affects all apps running on the new platform, even those not targeting the new
+  platform version. The platform provides a limited compatibility behavior for legacy apps, but you
+  should begin planning your app’s migration to the new permissions model now, with a goal of
+  publishing an updated version of your app at the official platform launch.
+</p>
+
+
+<h3 id="permission-test-tips">Test tips</h3>
+
+<p>
+  Use the following test tips to help you plan and execute testing of your app with the new
+  permissions behavior.
+</p>
+
+<ul>
+  <li>Identify your app’s current permissions and the related code paths.</li>
+  <li>Test user flows across permission-protected services and data.</li>
+  <li>Test with various combinations of granted/revoked permission.</li>
+  <li>Use the {@code adb} tool to manage permssions from the command line:
+    <ul>
+      <li>List permissions and status by group:
+        <pre>adb shell pm list permissions -d -g</pre>
+      </li>
+      <li>Grant or revoke one or more permissions using the following syntax:<br>
+        <pre>adb shell pm [grant|revoke] &lt;permission.name&gt; ...</pre>
+      </li>
+    </ul>
+  </li>
+  <li>Analyze your app for services that use permissions.</li>
+</ul>
+
+<h3 id="permission-test-strategy">Test strategy</h3>
+
+<p>
+  The Runtime Permissions change affects the structure and design of your app, as well as
+  the user experience and flows you provide to users. You should assess your app’s current
+  permissions use and start planning for the new flows you want to offer. The official release of
+  the platform provides compatibility behavior, but you should plan on updating your app and not
+  rely on these behaviors.
+</p>
+
+<p>
+  Identify the permissions that your app actually needs and uses, and then find the various code
+  paths that use the permission-protected services. You can do this through a combination of
+  testing on the new platform and code analysis. In testing, you should focus on opting in to
+  runtime permissions by changing the app’s {@code targetSdkVersion} to the preview version. For
+  more information, see <a href="{@docRoot}preview/setup-sdk.html#">Set up the Preview SDK</a>.
+</p>
+
+<p>
+  Test with various combinations of permissions revoked and added, to highlight the user flows that
+  depend on permissions. Where a dependency is not obvious or logical you should consider
+  refactoring or compartmentalizing that flow to eliminate the dependency or make it clear why the
+  permission is needed.
+</p>
+
+<p>
+  For more information on the behavior of Runtime Permissions, testing, and best practices, see the
+  <a href="{@docRoot}preview/features/runtime-permissions.html">Runtime Permissions</a> developer
+  preview page.
+</p>
+
+
+<h2 id="doze-standby">Testing Doze and App Standby</h2>
+
+<p>
+  The power saving features of Doze and App Standby limits the amount of background processing that
+  your app can perform when a device is in an idle state or while your app is not in focus. The
+  restrictions the system may impose on apps include limited or no network access,
+  suspended background tasks, suspended Notifications, ignored wake requests, and alarms. To ensure
+  that your app behaves properly with these power saving optimizations, you should test your app by
+  simulating these low power states.
+</p>
+
+<h4 id="doze">Testing your app with Doze</h4>
+
+<p>To test Doze with your app:</p>
+
+<ol>
+<li>Configure a hardware device or virtual device with a M Preview system image.</li>
+<li>Connect the device to your development machine and install your app.</li>
+<li>Run your app and leave it active.</li>
+<li>Simulate the device going into Doze mode by running the following commands:
+
+<pre>
+$ adb shell dumpsys battery unplug
+$ adb shell dumpsys deviceidle step
+$ adb shell dumpsys deviceidle -h
+</pre>
+
+  </li>
+  <li>Observe the behavior of your app when the device is re-activated. Make sure it
+    recovers gracefully when the device exits Doze.</li>
+</ol>
+
+
+<h4 id="standby">Testing apps with App Standby</h4>
+
+<p>To test the App Standby mode with your app:</p>
+
+<ol>
+  <li>Configure a hardware device or virtual device with a M Preview system image.</li>
+  <li>Connect the device to your development machine and install your app.</li>
+  <li>Run your app and leave it active.</li>
+  <li>Simulate the app going into standby mode by running the following commands:
+
+<pre>
+$ adb shell am broadcast -a android.os.action.DISCHARGING
+$ adb shell am set-idle &lt;packageName&gt; true
+</pre>
+
+  </li>
+  <li>Simulate waking your app using the following command:
+    <pre>$ adb shell am set-idle &lt;packageName&gt; false</pre>
+  </li>
+  <li>Observe the behavior of your app when it is woken. Make sure it recovers gracefully
+    from standby mode. In particular, you should check if your app's Notifications and background
+    jobs continue to function as expected.</li>
+</ol>