diff options
Diffstat (limited to 'emulator/opengl/DESIGN')
| -rw-r--r-- | emulator/opengl/DESIGN | 593 |
1 files changed, 593 insertions, 0 deletions
diff --git a/emulator/opengl/DESIGN b/emulator/opengl/DESIGN new file mode 100644 index 0000000..1da44b7 --- /dev/null +++ b/emulator/opengl/DESIGN @@ -0,0 +1,593 @@ +Android Hardware OpenGLES emulation design overview +=================================================== + +Introduction: +------------- + +Hardware GLES emulation in the Android platform is implemented with a mix +of components, which are: + + - Several host "translator" libraries. They implement the EGL, GLES 1.1 and + GLES 2.0 ABIs defined by Khronos, and translate the corresponding function + calls into calls to the appropriate desktop APIs, i.e.: + + - Xgl (Linux), AGL (OS X) or WGL (Windows) for EGL + - desktop GL 2.0 for GLES 1.1 and GLES 2.0 + + _________ __________ __________ + | | | | | | HOST + |TRANSLATOR |TRANSLATOR| |TRANSLATOR| HOST + | EGL | | GLES 1.1 | | GLES 2.0 | TRANSLATOR + |_________| |__________| |__________| LIBRARIES + | | | + - - - | - - - - - - - - - | - - - - - - - - - | - - - - - + | | | + ____v____ ____v_____ _____v____ HOST + | | | | | | SYSTEM + | Xgl | | GL 2.0 | | GL 2.0 | LIBRARIES + |_________| |__________| |__________| + + + + - Several system libraries inside the emulated guest system that implement + the same EGL / GLES 1.1 and GLES 2.0 ABIs. + + They collect the sequence of EGL/GLES function calls and translate then + into a custom wire protocol stream that is sent to the emulator program + through a high-speed communication channel called a "QEMU Pipe". + + For now, all you need to know is that the pipe is implemented with a + custom kernel driver, and provides for _very_ fast bandwidth. All read() + and writes() from/to the pipes are essentially instantaneous from the + guest's point of view. + + + _________ __________ __________ + | | | | | | + |EMULATION| |EMULATION | |EMULATION | GUEST + | EGL | | GLES 1.1 | | GLES 2.0 | SYSTEM + |_________| |__________| |__________| LIBRARIES + | | | + - - - | - - - - - - - - - | - - - - - - - - - | - - - - - + | | | + ____v____________________v____________________v____ GUEST + | | KERNEL + | QEMU PIPE | + |___________________________________________________| + | + - - - - - - - - - - - - - -|- - - - - - - - - - - - - - - - + | + v + EMULATOR + + - Specific code inside the emulator program that is capable of transmitting + the wire protocol stream to a special rendering library or process (called + the "renderer" here), which understands the format. + + | + | PROTOCOL BYTE STREAM + _____v_____ + | | + | EMULATOR | + |___________| + | + | UNMODIFIED PROTOCOL BYTE STREAM + _____v_____ + | | + | RENDERER | + |___________| + + + - The renderer decodes the EGL/GLES commands from the wire + protocol stream, and dispatches them to the translator libraries + appropriately. + + | + | PROTOCOL BYTE STREAM + _____v_____ + | | + | RENDERER | + |___________| + | | | + +-----------------+ | +-----------------+ + | | | + ____v____ ___v______ ____v_____ + | | | | | | HOST + |TRANSLATOR |TRANSLATOR| |TRANSLATOR| HOST + | EGL | | GLES 1.1 | | GLES 2.0 | TRANSLATOR + |_________| |__________| |__________| LIBRARIES + + + + - In reality, the protocol stream flows in both directions, even though most + of the commands result in data going from the guest to the host. A complete + picture of the emulation would thus be: + + + + + + _________ __________ __________ + | | | | | | + |EMULATION| |EMULATION | |EMULATION | GUEST + | EGL | | GLES 1.1 | | GLES 2.0 | SYSTEM + |_________| |__________| |__________| LIBRARIES + ^ ^ ^ + | | | + - - - | - - - - - - - - - | - - - - - - - - - | - - - - - + | | | + ____v____________________v____________________v____ GUEST + | | KERNEL + | QEMU PIPE | + |___________________________________________________| + ^ + | + - - - - - - - - - - - - - -|- - - - - - - - - - - - - - - - + | + | PROTOCOL BYTE STREAM + _____v_____ + | | + | EMULATOR | + |___________| + ^ + | UNMODIFIED PROTOCOL BYTE STREAM + _____v_____ + | | + | RENDERER | + |___________| + ^ ^ ^ + | | | + +-----------------+ | +-----------------+ + | | | + ____v____ ___v______ ____v_____ + | | | | | | + |TRANSLATOR |TRANSLATOR| |TRANSLATOR| HOST + | EGL | | GLES 1.1 | | GLES 2.0 | TRANSLATOR + |_________| |__________| |__________| LIBRARIES + ^ ^ ^ + | | | + - - - | - - - - - - - - - | - - - - - - - - - | - - - - - + | | | + ____v____ ____v_____ _____v____ HOST + | | | | | | SYSTEM + | Xgl | | GL 2.0 | | GL 2.0 | LIBRARIES + |_________| |__________| |__________| + + (NOTE: 'Xgl' is for Linux only, replace 'AGL' on OS X, and 'WGL' on Windows). + + +Note that, in the above graphics, only the host system libraries at the bottom +are _not_ provided by Android. + + +Design Requirements: +-------------------- + +The above design comes from several important requirements that were decided +early in the project: + +1 - The ability to run the renderer in a separate process from the emulator + itself is important. + + For various practical reasons, we plan to completely separate the core QEMU + emulation from the UI window by using two distinct processes. As such, the + renderer will be implemented as a library inside the UI program, but will + need to receive protocol bytes from the QEMU process. + + The communication channel will be either a fast Unix socket or a Win32 + named pipe between these two. A shared memory segment with appropriate + synchronization primitives might also be used if performance becomes + an issue. + + This explains why the emulator doesn't alter or even try to parse the + protocol byte stream. It only acts as a dumb proxy between the guest + system and the renderer. This also avoids adding lots of GLES-specific + code inside the QEMU code base which is terribly complex. + +2 - The ability to use vendor-specific desktop EGL/GLES libraries is + important. + + GPU vendors like NVidia, AMD or ARM all provide host versions of the + EGL/GLES libraries that emulate their respectivie embedded graphics + chipset. + + The renderer library can be configured to use these instead of the + translator libraries provided with this project. This can be useful to + more accurately emulate the behaviour of specific devices. + + Moreover, these vendor libraries typically expose vendor-specific + extensions that are not provided by the translator libraries. We cannot + expose them without modifying our code, but it's important to be able + to do so without too much pain. + + +Code organization: +------------------ + +All source code for the components above is spread over multiple directories +in the Android source trees: + + - The emulator sources are under $ANDROID/external/qemu, which we'll + call $QEMU in the rest of this document. + + - The guest and system libraries are under + $ANDROID/development/tools/emulator/opengl, which we'll call $EMUGL + + - The QEMU Pipe kernel driver is under $KERNEL/drivers/misc/qemupipe + +Where $ANDROID is the top of the open-source Android source tree, and +$KERNEL is the top of the qemu-specific kernel source tree (using one +of the android-goldfish-xxxx branches here). + +The emulator sources related to this projects are: + + $QEMU/hw/goldfish_pipe.c -> implement QEMU pipe virtual hardware + $QEMU/hw/opengles.c -> implement GLES initialization + $QEMU/hw/hw-pipe-net.c -> implements the communication channel + between the QEMU Pipe and the renderer library + +The other sources are: + + $EMUGL/system -> system libraries + $EMUGL/host -> host libraries (translator + renderer) + $EMUGL/shared -> shared libraries, used both in the guest and the host + $EMUGL/tests -> various test programs + + +Translator libraries: +--------------------- + +There are three translator host libraries provided by this project: + + libEGL_translator -> EGL 1.2 translation + libGLES_CM_translator -> GLES 1.1 translation + libGLES_V2_translator -> GLES 2.0 translation + +The full name of the library will depend on the host system. +For simplicity, only the library name suffix will change (i.e. the +'lib' prefix is not dropped on Windows), i.e.: + + libEGL_translator.so -> for Linux + libEGL_translator.dylib -> for OS X + libEGL_translator.dll -> for Windows + +The source code for these libraries is located under the following +path in the Android source tree: + + $EMUGL/host/libs/Translator/EGL + $EMUGL/host/libs/Translator/GLES_CM + $EMUGL/host/libs/Translator/GLES_V2 + +The translator libraries also use a common routines defined under: + + $EMUGL/host/libs/Translator/GLcommon + + +Wire Protocol Overiew: +---------------------- + +The "wire protocol" is implemented as follows: + + - EGL/GLES function calls are described through several "specification" + files, which describes the types, function signatures and various + attributes for each one of them. + + - These files are read by a tool called "emugen" which generates C + source files and headers based on the specification. These correspond + to both encoding, decoding and "wrappers" (more on this later). + + - System "encoder" static libraries are built using some of these generated + files. They contain code that can serialize EGL/GLES calls into simple + byte messages and send it through a generic "IOStream" object. + + - Host "decoder" static libraries are also built using some of these + generated files. Their code retrieves byte messages from an "IOStream" + object, and translates them into function callbacks. + +IOStream abstraction: +- - - - - - - - - - - + +The "IOStream" is a very simple abstract class used to send byte messages +both in the guest and host. It is defined through a shared header under +$EMUGL/host/include/libOpenglRender/IOStream.h + +Note that despite the path, this header is included by *both* host and guest +source code. The main idea around IOStream's design is that to send a message, +one does the following: + + 1/ call stream->allocBuffer(size), which returns the address of a + memory buffer of at least 'size' bytes. + + 2/ write the content of the serialized command (usually a header + some + payload) directly into the buffer + + 3/ call stream->commitBuffer() to send it. + +Alternatively, one can also pack several commands into a single buffer with +stream->alloc() and stream->flush(), as in: + + 1/ buf1 = stream->alloc(size1) + 2/ write first command bytes into buf1 + 3/ buf2 = stream->alloc(size2) + 4/ write second command bytes into buf2 + 5/ stream->flush() + +Finally, there are also explict read/write methods like stream->readFully() +or stream->writeFully() which can be used when you don't want an intermediate +buffer. This is used in certain cases by the implementation, e.g. to avoid +an intermediate memory copy when sending texture data from the guest to the +host. + +The host IOStream implementations are under $EMUGL/shared/OpenglCodecCommon/, +see in particular: + + $EMUGL/shared/OpenglCodecCommon/TcpStream.cpp -> using local TCP sockets + $EMUGL/shared/OpenglCodecCommon/UnixStream.cpp -> using Unix sockets + $EMUGL/shared/OpenglCodecCommon/Win32PipeStream.cpp -> using Win32 named pipes + +The guest IOStream implementation uses the TcpStream.cpp above, as well as +an alternative QEMU-specific source: + + $EMUGL/system/OpenglSystemCommon/QemuPipeStream.cpp -> uses QEMU pipe from the guest + +The QEMU Pipe implementation is _significantly_ faster (about 20x) due to +several reasons: + + - all succesful read() and write() operations through it are instantaneous + from the guest's point of view. + + - all buffer/memory copies are performed directly by the emulator, and thus + much faster than performing the same thing inside the kernel with emulated + ARM instructions. + + - it doesn't need to go through a kernel TCP/IP stack that will wrap the + data into TCP/IP/MAC packets, send them to an emulated ethernet device, + which is itself connected to an internal firewall implementation that + will unwrap the packets, re-assemble them, then send them through BSD + sockets to the host kernel. + +However, would it be necessary, you could write a guest IOStream implementation +that uses a different transport. If you do, please look at +$EMUGL/system/OpenglCodecCommon/HostConnection.cpp which contains the code +used to connect the guest to the host, on a per-thread basis. + + +Source code auto-generation: +- - - - - - - - - - - - - - + +The 'emugen' tool is located under $EMUGL/host/tools/emugen. There is a README +file that explains how it works. + +You can also look at the following specifications files: + +For GLES 1.1: + $EMUGL/system/GLESv1_enc/gl.types + $EMUGL/system/GLESv1_enc/gl.in + $EMUGL/system/GLESv1_enc/gl.attrib + $EMUGL/system/GLESv1_enc/gl.addon + +For GLES 2.0: + $EMUGL/system/GLESv2_enc/gl2.types + $EMUGL/system/GLESv2_enc/gl2.in + $EMUGL/system/GLESv2_enc/gl2.attrib + $EMUGL/system/GLESv2_enc/gl2.addon + +For EGL: + $EMUGL/system/renderControl_enc/renderControl.types + $EMUGL/system/renderControl_enc/renderControl.in + $EMUGL/system/renderControl_enc/renderControl.attrib + $EMUGL/system/renderControl_enc/renderControl.addon + +Note that the EGL specification files are under a directory named +"renderControl_enc" and have filenames that begin with "renderControl" + +This is mainly for historic reasons now, but is also related to the fact that +this part of the wire protocol contains support functions/calls/specifications +that are not part of the EGL specification itself, but add a few features +required to make everything works. For example, they have calls related to +the "gralloc" system library module used to manage graphics surfaces at a +lower level than EGL. + +Generally speaking, guest encoder sources are located under directories +named $EMUGL/system/<name>_enc/, while the corresponding host decoder +sources will be under $EMUGL/host/libs/<name>_dec/ + +However, all these sources use the same spec files located under the +encoding directories. The decoders may even need to include a few +non-auto-generated header files from the encoder directories. + + + +System libraries: +----------------- + +Meta EGL/GLES system libraries, and egl.cfg: +- - - - - - - - - - - - - - - - - - - - - - + +It is important to understand that the emulation-specific EGL/GLES libraries +are not directly linked by applications at runtime. Instead, the system +provides a set of "meta" EGL/GLES libraries that will load the appropriate +hardware-specific libraries on first use. + +More specifically, the system libEGL.so contains a "loader" which will try +to load: + + - hardware-specific EGL/GLES libraries + - the software-based rendering libraries (called "libagl") + +The system libEGL.so is also capable of merging the EGL configs of both the +hardware and software libraries transparently to the application. The system +libGLESv1_CM.so and libGLESv2.so, work with it to ensure that the thread's +current context will be linked to either the hardware or software libraries +depending on the config selected. + +For the record, the loader's source code in under +frameworks/base/opengl/libs/EGL/Loader.cpp. It depends on a file named +/system/lib/egl/egl.cfg which must contain two lines that look like: + + 0 1 <name> + 0 0 android + +The first number in each line is a display number, and must be 0 since the +system's EGL/GLES libraries don't support anything else. + +The second number must be 1 to indicate hardware libraries, and 0 to indicate +a software one. The line corresponding to the hardware library, if any, must +always appear before the one for the software library. + +The third field is a name corresponding to a shared library suffix. It really +means that the corresponding libraries will be named libEGL_<name>.so, +libGLESv1_CM_<name>.so and libGLESv2_<name>.so. Moreover these libraries must +be placed under /system/lib/egl/ + +The name "android" is reserved for the system software renderer. + +The egl.cfg that comes with this project uses the name "emulation" for the +hardware libraries. This means that it provides an egl.cfg file that contains +the following lines: + + 0 1 emulation + 0 0 android + +See $EMUGL/system/egl/egl.cfg and more generally the following build files: + + $EMUGL/system/egl/Android.mk + $EMUGL/system/GLESv1/Android.mk + $EMUGL/system/GLESv2/Android.mk + +to see how the libraries are named and placed under /system/lib/egl/ by the +build system. + + +Emulation libraries: +- - - - - - - - - - - + +The emulator-specific libraries are under the following: + + $EMUGL/system/egl/ + $EMUGL/system/GLESv1/ + $EMUGL/system/GLESv2/ + +The code for GLESv1 and GLESv2 is pretty small, since it mostly link against +the static encoding libraries. + +The code for EGL is a bit more complex, because it needs to deal with +extensions dynamically. I.e. if an extension is not available on the host +it shouldn't be exposed by the library at runtime. So the EGL code queries +the host for the list of available extensions in order to return them to +clients. Similarly, it must query the list of valid EGLConfigs for the +current host system. + + +"gralloc" module implementation: +- - - - - - - - - - - - - - - - - + +In addition to EGL/GLES libraries, the Android system requires a +hardware-specific library to manage graphics surfaces at a level lower than +EGL. This library must be what is called in Android land as a "HAL module". + +A "HAL module" must provide interfaces defined by Android's HAL +(Hardware Abstraction Library). These interface definitions can be found +under $ANDROID/hardware/libhardware/include/ + +Of all possible HAL modules, the "gralloc" one is used by the system's +SurfaceFlinger to allocate framebuffers and other graphics memory regions, +as well as eventually lock/unlock/swap them when needed. + +The code under $EMUGL/system/gralloc/ implements the module required by the +GLES emulation project. It's not very long, but there are a few things to +notice here: + +- first, it will probe the guest system to determine if the emulator that + is running the virtual device really supports GPU emulation. In certain + circumstances this may not be possible. + + If this is the case, then the module will redirect all calls to the + "default" gralloc module that is normally used by the system when + software-only rendering is enabled. + + The probing happens in the function "fallback_init" which gets called + when the module is first opened. This initializes the 'sFallback' variable + to a pointer to the default gralloc module when required. + +- second, this module is used by SurfaceFlinger to display "software surfaces", + i.e. those that are backed by system memory pixel buffers, and written to + directly through the Skia graphics library (i.e. the non-accelerated ones). + + the default module simply copies the pixel data from the surface to the + virtual framebuffer i/o memory, but this project's gralloc module sends it + to the renderer through the QEMU Pipe instead. + + It turns out that this results in _faster_ rendering/frame-rates overall, + because memory copies inside the guest are slow, while QEMU pipe transfers + are done directly in the emulator. + + +Host Renderer: +-------------- + +The host renderer library is located under $EMUGL/host/libs/libOpenglRender, +and it provides an interface described by the headers under +$EMUGL/host/include/libOpenglRender/render_api.h (e.g. for use by the emulator). + +In a nutshell, the rendering library is responsible for the following: + + - Providing a virtual off-screen video surface where everything will get + rendered at runtime. Its dimensions are fixed by the call to + initOpenglRender() that must happen just after the library is + initialized. + + - Provide a way to display the virtual video surface on a host application's + UI. This is done by calling createOpenGLSubWindow() which takes as argument + the window ID or handle of a parent window, some display dimensions and + a rotation angle. This allows the surface to be scaled/rotated when it is + displayed, even if the dimensions of the video surface do not change. + + - Provide a way to listen to incoming EGL/GLES commands from the guest. + This is done by providing a so-called "port number" to initOpenglRender(). + + By default, the port number corresponds to a local TCP port number that the + renderer will bind to and listen. Every new connection to this port will + correspond to the creation of a new guest host connection, each such + connection corresponding to a distinct thread in the guest system. + + For performance reasons, it is possible to listen to either Unix sockets + (on Linux and OS X), or to a Win32 named pipe (on Windows). To do so, one + had to call setStreamType() between library initialization + (i.e. initLibrary()) and construction (i.e. initOpenglRender()). + + Note that in these modes, the port number is still used to differentiate + between several emulator instances. These details are normally handled by + the emulator code so you shouldn't care too much. + +Note that an earlier version of the interface allowed a client of the renderer +library to provide its own IOStream implementation. However, this wasn't very +convenient for a number of reasons. This maybe something that could be done +again if it makes sense, but for now the performance numbers are pretty good. + + +Host emulator: +-------------- + +The code under $QEMU/android/opengles.c is in charge of dynamically loading +the rendering library and initializing / constructing it properly. + +QEMU pipe connections to the 'opengles' service are piped through the code +in $QEMU/android/hw-pipe-net.c. Look for the openglesPipe_init() function, +which is in charge of creating a connection to the renderer library +(either through a TCP socket, or a Unix pipe depending on configuration. +support for Win32 named pipes hasn't been implemented yet in the emulator) +whenever a guest process opens the "opengles" service through /dev/qemu_pipe. + +There is also some support code for the display of the GLES framebuffer +(through the renderer library's subwindow) under $QEMU/skin/window. + +Note that at the moment, scaling and rotation are supported. However, +brightness emulation (which used to modify the pixel values from the +hardware framebuffer before displaying them) doesn't work. + +Another issue is that it is not possible to display anything on top of the +GL subwindow at the moment. E.g. this will obscure the emulated trackball +image (that is normally toggled with Ctrl-T during emulation, or enabled +by pressing the Delete key). + |