page.title=3D Rendering and Computation with Renderscript parent.title=Graphics parent.link=index.html @jd:body

In this document

  1. Renderscript System Overview
  2. API Overview
    1. Native Renderscript APIs
    2. Reflective layer APIs
    3. Graphics APIs
  3. Developing a Renderscript application
    1. The Hello Graphics application

The Renderscript system offers high performance 3D rendering and mathematical computations at the native level. The Renderscript APIs are intended for developers who are comfortable with developing in C (C99 standard) and want to maximize performance in their applications. The Renderscript system improves performance by running as native code on the device, but it also features cross-platform functionality. To achieve this, the Android build tools compile your Renderscript .rs file to intermediate bytecode and package it inside your application's .apk file. On the device, the bytecode is compiled (just-in-time) to machine code that is further optimized for the device that it is running on. This eliminates the need to target a specific architecture during the development process. The compiled code on the device is cached, so subsequent uses of the Renderscript enabled application do not recompile the intermediate code.

The disadvantage of the Renderscript system is that it adds complexity to the development and debugging processes and is not a substitute for the Android system APIs. It is a portable native language with pointers and explicit resource management. The target use is for performance critical code where the existing Android APIs are not sufficient. If what you are rendering or computing is very simple and does not require much processing power, you should still use the Android APIs for ease of development. Debugging visibility can be limited, because the Renderscript system can execute on processors other than the main CPU (such as the GPU), so if this occurs, debugging becomes more difficult. Remember the tradeoffs between development and debugging complexity versus performance when deciding to use Renderscript.

For an example of Renderscript in action, see the 3D carousel view in the Android 3.0 versions of Google Books and YouTube or install the Renderscript sample applications that are shipped with the SDK in <sdk_root>/platforms/android-3.0/samples.

Renderscript System Overview

The Renderscript system adopts a control and slave architecture where the low-level native code is controlled by the higher level Android system that runs in the virtual machine (VM). When you use the Renderscript system, there are three layers of APIs that exist:

To fully understand how the Renderscript system works, you must understand how the reflected layer is generated and how it interacts with the native Renderscript layer and Android system layer. The reflected layer provides the entry points into the native code, enabling the Android system code to give high level commands like, "rotate the view" or "filter the bitmap." It delegates all the heavy lifting to the native layer. To accomplish this, you need to create logic to hook together all of these layers so that they can correctly communicate.

At the root of everything is your Renderscript, which is the actual C code that you write and save to a .rs file in your project. There are two kinds of Renderscripts: compute and graphics. A compute Renderscript does not do any graphics rendering while a graphics Renderscript does.

When you create a Renderscript .rs file, an equivalent, reflective layer class, {@link android.renderscript.ScriptC}, is generated by the build tools and exposes the native functions to the Android system. This class is named ScriptC_renderscript_filename. The following list describes the major components of your native Renderscript code that is reflected:

The Android system also has a corresponding Renderscript context object, {@link android.renderscript.RenderScript} (for a compute Renderscript) or {@link android.renderscript.RenderScriptGL} (for a graphics Renderscript). This context object allows you to bind to the reflected Renderscript class, so that the Renderscript context knows what its corresponding native Renderscript is. If you have a graphics Renderscript context, you can also specify a variety of Programs (stages in the graphics pipeline) to tweek how your graphics are rendered. A graphics Renderscript context also needs a surface to render on, {@link android.renderscript.RSSurfaceView}, which gets passed into its constructor. When all three of the layers are connected, the Renderscript system can compute or render graphics.

API overview

Renderscript code is compiled and executed in a compact and well defined runtime, which has access to a limited amount of functions. Renderscript cannot use the NDK or standard C functions, because these functions are assumed to be running on a standard CPU. The Renderscript runtime chooses the best processor to execute the code, which may not be the CPU, so it cannot guarantee support for standard C libraries. What Renderscript does offer is an API that supports intensive computation with an extensive collection of math APIs. Some key features of the Renderscript APIs are:

Native Renderscript APIs

The Renderscript headers are located in the include and clang-include directories in the <sdk_root>/platforms/android-3.0/renderscript directory of the Android SDK. The headers are automatically included for you, except for the graphics specific header, which you can define as follows:

#include "rs_graphics.rsh"

Some key features of the native Renderscript libraries include:

Reflective layer APIs

These classes are not generated by the reflection process, and are actually part of the Android system APIs, but they are mainly used by the reflective layer classes to handle memory allocation and management for your Renderscript. You normally do not need to be call these classes directly.

Because of the constraints of the Renderscript native layer, you cannot do any dynamic memory allocation in your Renderscript .rs file. The native Renderscript layer can request memory from the Android system layer, which allocates memory for you and does reference counting to figure out when to free the memory. A memory allocation is taken care of by the {@link android.renderscript.Allocation} class and memory is requested in your Renderscript code with the the rs_allocation type. All references to Renderscript objects are counted, so when your Renderscript native code or system code no longer references a particular {@link android.renderscript.Allocation}, it destroys itself. Alternatively, you can call {@link android.renderscript.Allocation#destroy destroy()} from the Android system level, which decreases the reference to the {@link android.renderscript.Allocation}. If no references exist after the decrease, the {@link android.renderscript.Allocation} destroys itself. The Android system object, which at this point is just an empty shell, is eventually garbage collected.

The following classes are mainly used by the reflective layer classes:

Android Object Type Renderscript Native Type Description
{@link android.renderscript.Element} rs_element An {@link android.renderscript.Element} is the most basic element of a memory type. An element represents one cell of a memory allocation. An element can have two forms: Basic or Complex. They are typically created from C structures that are used within Renderscript code and cannot contain pointers or nested arrays. The other common source of elements is bitmap formats.

A basic element contains a single component of data of any valid Renderscript data type. Examples of basic element data types include a single float value, a float4 vector, or a single RGB-565 color.

Complex elements contain a list of sub-elements and names that is basically a reflection of a C struct. You access the sub-elements by name from a script or vertex program. The most basic primitive type determines the data alignment of the structure. For example, a float4 vector is alligned to sizeof(float) and not sizeof(float4). The ordering of the elements in memory are the order in which they were added, with each component aligned as necessary.
{@link android.renderscript.Type} rs_type A Type is an allocation template that consists of an element and one or more dimensions. It describes the layout of the memory but does not allocate storage for the data that it describes. A Type consists of five dimensions: X, Y, Z, LOD (level of detail), and Faces (of a cube map). You can assign the X,Y,Z dimensions to any positive integer value within the constraints of available memory. A single dimension allocation has an X dimension of greater than zero while the Y and Z dimensions are zero to indicate not present. For example, an allocation of x=10, y=1 is considered two dimensional and x=10, y=0 is considered one dimensional. The LOD and Faces dimensions are booleans to indicate present or not present.
{@link android.renderscript.Allocation} rs_allocation An {@link android.renderscript.Allocation} provides the memory for applications. An {@link android.renderscript.Allocation} allocates memory based on a description of the memory that is represented by a {@link android.renderscript.Type}. The {@link android.renderscript.Type} describes an array of {@link android.renderscript.Element}s that represent the memory to be allocated. Allocations are the primary way data moves into and out of scripts.

Memory is user-synchronized and it's possible for allocations to exist in multiple memory spaces concurrently. For example, if you make a call to the graphics card to load a bitmap, you give it the bitmap to load from in the system memory. After that call returns, the graphics memory contains its own copy of the bitmap so you can choose whether or not to maintain the bitmap in the system memory. If the Renderscript system modifies an allocation that is used by other targets, it must call {@link android.renderscript#syncAll syncAll()} to push the updates to the memory. Otherwise, the results are undefined.

Allocation data is uploaded in one of two primary ways: type checked and type unchecked. For simple arrays there are copyFrom() functions that take an array from the Android system code and copy it to the native layer memory store. Both type checked and unchecked copies are provided. The unchecked variants allow the Android system to copy over arrays of structures because it not support inherently support structures. For example, if there is an allocation that is an array n floats, you can copy the data contained in a float[n] array or a byte[n*4] array.
{@link android.renderscript.Script} rs_script Renderscript scripts do much of the work in the native layer. This class is generated from a Renderscript file that has the .rs file extension. This class is named ScriptC_rendersript_filename when it gets generated.

Graphics API

Renderscript provides a number of graphics APIs for hardware-accelerated 3D rendering. The Renderscript graphics APIs include a stateful context, {@link android.renderscript.RenderScriptGL} that contains the current rendering state. The primary state consists of the objects that are attached to the rendering context, which are the graphics Renderscript and the four program types. The main working function of the graphics Renderscript is the code that is defined in the root() function. The root() function is called each time the surface goes through a frame refresh. The four program types mirror a traditional graphical rendering pipeline and are:

Graphical scripts have more properties beyond a basic computational script, and they call the 'rsg'-prefixed functions defined in the rs_graphics.rsh header file. A graphics Renderscript can also set four pragmas that control the default bindings to the {@link android.renderscript.RenderScriptGL} context when the script is executing:

The possible values are parent or default for each pragma. Using default says that when a script is executed, the bindings to the graphical context are the system defaults. Using parent says that the state should be the same as it is in the calling script. If this is a root script, the parent state is taken from the bind points as set in the {@link android.renderscript.RenderScriptGL} bind methods in the control environment (VM environment).

For example, you can define this at the top of your native Renderscript code:

#pragma stateVertex(parent)
#pragma stateStore(parent)

The following table describes the major graphics specific APIs that are available to you:

Android Object Type Renderscript Native Type Description
{@link android.renderscript.ProgramVertex} rs_program_vertex The Renderscript vertex program, also known as a vertex shader, describes the stage in the graphics pipeline responsible for manipulating geometric data in a user-defined way. The object is constructed by providing Renderscript with the following data:
  • An Element describing its varying inputs or attributes
  • GLSL shader string that defines the body of the program
  • a Type that describes the layout of an Allocation containing constant or uniform inputs

Once the program is created, bind it to the graphics context. It is then used for all subsequent draw calls until you bind a new program. If the program has constant inputs, the user needs to bind an allocation containing those inputs. The allocation’s type must match the one provided during creation. The Renderscript library then does all the necessary plumbing to send those constants to the graphics hardware. Varying inputs to the shader, such as position, normal, and texture coordinates are matched by name between the input Element and the Mesh object being drawn. The signatures don’t have to be exact or in any strict order. As long as the input name in the shader matches a channel name and size available on the mesh, the run-time would take care of connecting the two. Unlike OpenGL, there is no need to link the vertex and fragment programs.

To bind shader constructs to the Program, declare a struct containing the necessary shader constants in your native Renderscript code. This struct is generated into a reflected class that you can use as a constant input element during the Program's creation. It is an easy way to create an instance of this struct as an allocation. You would then bind this Allocation to the Program and the Renderscript system sends the data that is contained in the struct to the hardware when necessary. To update shader constants, you change the values in the Allocation and notify the native Renderscript code of the change.
{@link android.renderscript.ProgramFragment} rs_program_fragment The Renderscript fragment program, also known as the fragment shader, is responsible for manipulating pixel data in a user-defined way. It’s constructed from a GLSL shader string containing the program body, textures inputs, and a Type object describing the constants used by the program. Like the vertex programs, when an allocation with constant input values is bound to the shader, its values are sent to the graphics program automatically. Note that the values inside the allocation are not explicitly tracked. If they change between two draw calls using the same program object, notify the runtime of that change by calling rsgAllocationSyncAll so it could send the new values to hardware. Communication between the vertex and fragment programs is handled internally in the GLSL code. For example, if the fragment program is expecting a varying input called varTex0, the GLSL code inside the program vertex must provide it.

To bind shader constructs to the this Program, declare a struct containing the necessary shader constants in your native Renderscript code. This struct is generated into a reflected class that you can use as a constant input element during the Program's creation. It is an easy way to create an instance of this struct as an allocation. You would then bind this Allocation to the Program and the Renderscript system sends the data that is contained in the struct to the hardware when necessary. To update shader constants, you change the values in the Allocation and notify the native Renderscript code of the change.
{@link android.renderscript.ProgramStore} rs_program_store The Renderscript ProgramStore contains a set of parameters that control how the graphics hardware writes to the framebuffer. It could be used to enable/disable depth writes and testing, setup various blending modes for effects like transparency and define write masks for color components.
{@link android.renderscript.ProgramRaster} rs_program_raster Program raster is primarily used to specify whether point sprites are enabled and to control the culling mode. By default back faces are culled.
{@link android.renderscript.Sampler} rs_sampler A Sampler object defines how data is extracted from textures. Samplers are bound to Program objects (currently only a Fragment Program) alongside the texture whose sampling they control. These objects are used to specify such things as edge clamping behavior, whether mip-maps are used and the amount of anisotropy required. There may be situations where hardware limitations prevent the exact behavior from being matched. In these cases, the runtime attempts to provide the closest possible approximation. For example, the user requested 16x anisotropy, but only 8x was set because it’s the best available on the hardware.
{@link android.renderscript.Mesh} rs_mesh A collection of allocations that represent vertex data (positions, normals, texture coordinates) and index data such as triangles and lines. Vertex data can be interleaved within one allocation, provided separately as multiple allocation objects, or done as a combination of the above. The layout of these allocations will be extracted from their Elements. When a vertex channel name matches an input in the vertex program, Renderscript automatically connects the two. Moreover, even allocations that cannot be directly mapped to graphics hardware can be stored as part of the mesh. Such allocations can be used as a working area for vertex-related computation and will be ignored by the hardware. Parts of the mesh could be rendered with either explicit index sets or primitive types.
{@link android.renderscript.Font} rs_font

This class gives you a way to draw hardware accelerated text. Internally, the glyphs are rendered using the Freetype library, and an internal cache of rendered glyph bitmaps is maintained. Each font object represents a combination of a typeface and point sizes. Multiple font objects can be created to represent faces such as bold and italic and to create different font sizes. During creation, the framework determines the device screen's DPI to ensure proper sizing across multiple configurations.

Font rendering can impact performance. Even though though the state changes are transparent to the user, they are happening internally. It is more efficient to render large batches of text in sequence, and it is also more efficient to render multiple characters at once instead of one by one.

Font color and transparency are not part of the font object and can be freely modified in the script to suit the your needs. Font colors work as a state machine, and every new call to draw text will use the last color set in the script.

Developing a Renderscript application

The basic workflow of developing a Renderscript application is:

  1. Analyze your application's requirements and figure out what you want to develop with Renderscript. To take full advantage of Renderscript, you want to use it when the computation or graphics performance you're getting with the normal Android system APIs is insufficient.
  2. Design the interface of your Renderscript code and implement it using the native Renderscript APIs that are included in the Android SDK in <sdk_root>/platforms/android-3.0/renderscript.
  3. Create an Android project as you would normally, in Eclipse or with the android tool.
  4. Place your Renderscript files in src folder of the Android project so that the build tools can generate the reflective layer classes.
  5. Create your application, calling the Renderscript through the reflected class layer when you need to.
  6. Build, install, and run your application as you would normally.

To see how a simple Renderscript application is put together, see The Hello World Renderscript Graphics Application. The SDK also ships with many Renderscript samples in the<sdk_root>/samples/android-3.0/ directory.

The Hello Graphics Application

This small application demonstrates the structure of a simple Renderscript application. You can model your Renderscript application after the basic structure of this application. You can find the complete source in the SDK in the <android-sdk>/platforms/android-3.0/samples/HelloWorldRS directory. The application uses Renderscript to draw the string, "Hello World!" to the screen and redraws the text whenever the user touches the screen at the location of the touch. This application is only a demonstration and you should not use the Renderscript system to do something this trivial. The application contains the following source files:

Each file has its own distinct use. The following section demonstrates in detail how the sample works:

helloworld.rs
The native Renderscript code is contained in the helloworld.rs file. Every .rs file must contain two pragmas that define the version of Renderscript that it is using (1 is the only version for now), and the package name that the reflected classes should be generated with. For example: 
#pragma version(1)

#pragma rs java_package_name(com.my.package.name)

An .rs file can also declare two special functions:

ScriptC_helloworld
This class is generated by the Android build tools and is the reflected version of the helloworld.rs Renderscript. It provides a a high level entry point into the helloworld.rs native code by defining the corresponding methods that you can call from Android system APIs.
helloworld.bc bytecode
This file is the intermediate, platform-independent bytecode that gets compiled on the device when the Renderscript application runs. It is generated by the Android build tools and is packaged with the .apk file and subsequently compiled on the device at runtime. This file is located in the <project_root>/res/raw/ directory and is named rs_filename.bc. You need to bind these files to your Renderscript context before call any Renderscript code from your Android application. You can reference them in your code with R.id.rs_filename.
HelloWorldView class
This class represents the Surface View that the Renderscript graphics are drawn on. It does some administrative tasks in the ensureRenderScript() method that sets up the Renderscript system. This method creates a {@link android.renderscript.RenderScriptGL} object, which represents the context of the Renderscript and creates a default surface to draw on (you can set the surface properties such as alpha and bit depth in the {@link android.renderscript.RenderScriptGL.SurfaceConfig} class ). When a {@link android.renderscript.RenderScriptGL} is instantiated, this class calls the HelloRS class and creates the instance of the actual Renderscript graphics renderer. 
// Renderscipt context
private RenderScriptGL mRS;
// Script that does the rendering
private HelloWorldRS mRender;

    private void ensureRenderScript() {
        if (mRS == null) {
            // Initialize Renderscript with desired surface characteristics.
            // In this case, just use the defaults
            RenderScriptGL.SurfaceConfig sc = new RenderScriptGL.SurfaceConfig();
            mRS = createRenderScriptGL(sc);

            // Create an instance of the Renderscript that does the rendering
            mRender = new HelloWorldRS();
            mRender.init(mRS, getResources());
        }
    }

This class also handles the important lifecycle events and relays touch events to the Renderscript renderer. When a user touches the screen, it calls the renderer, HelloWorldRS and asks it to draw the text on the screen at the new location.

public boolean onTouchEvent(MotionEvent ev) {
    // Pass touch events from the system to the rendering script
    if (ev.getAction() == MotionEvent.ACTION_DOWN) {
        mRender.onActionDown((int)ev.getX(), (int)ev.getY());
        return true;
    }
    return false;
}
HelloWorldRS
This class represents the Renderscript renderer for the HelloWorldView Surface View. It interacts with the native Renderscript code that is defined in helloworld.rs through the interfaces exposed by ScriptC_helloworld. To be able to call the native code, it creates an instance of the Renderscript reflected class, ScriptC_helloworld. The reflected Renderscript object binds the Renderscript bytecode (R.raw.helloworld) and the Renderscript context, {@link android.renderscript.RenderScriptGL}, so the context knows to use the right Renderscript to render its surface. 
private Resources mRes;
private RenderScriptGL mRS;
private ScriptC_helloworld mScript;

private void initRS() {
    mScript = new ScriptC_helloworld(mRS, mRes, R.raw.helloworld);
    mRS.bindRootScript(mScript);
}