AI 144596: am: CL 144595 Fix the AVD configuration code to support "sdcard.path" in config.ini to indicate an explicit SD Card image file (instead of using the one in the content directory)

  Note that this also fix a bug where the SD Card image was not properly locked in the previous implementation.
  Allow the http-proxy support code to actually manage to receive chunked encoding data, instead of complaining needlessly.
  Introduce a new CharBuffer object that is used indirectly by "-radio <hostdevice>" and "-gps <hostdevice>" options
  Add new documentation for QEMUD and CharDriverState objects
  Update the Audio documentation with ASCII graphics (because I'm an artist too)
  Original author: digit
  Merged from: //branches/cupcake/...

Automated import of CL 144596

3 files changed, 495 insertions, 18 deletions

diff --git a/docs/ANDROID-QEMUD.TXT b/docs/ANDROID-QEMUD.TXT
new file mode 100644
index 0000000..6415a47
--- /dev/null
+++ b/docs/ANDROID-QEMUD.TXT
@@ -0,0 +1,241 @@
+                    THE ANDROID "QEMUD" MULTIPLEXING DAEMON
+
+I. Overview:
+------------
+
+The Android system image includes a small daemon program named "qemud"
+which is started at boot time. Its purpose is to provide a multiplexing
+communication channel between the emulated system and the emulator program
+itself.
+
+Its goal is to allow certain parts of the system to talk directly to the
+emulator without requiring special kernel support; this simplifies a lot of
+things since it does *not* require:
+
+- writing/configuring a specific kernel driver
+- writing the corresponding hardware emulation code in hw/goldfish_xxxx.c
+- dealing with device allocation and permission issues in the emulated system
+
+The emulator provides 'services' to various parts of the emulated system.
+Each service is identified by a name and serves a specific purpose. For
+example:
+
+   "gsm"       Used to communicate with the emulated GSM modem with
+               AT commands.
+
+   "gps"       Used to receive NMEA sentences broadcasted from the
+               emulated GPS device.
+
+   "sensors"   Used to list the number of emulated sensors, as well as
+               enable/disable reception of specific sensor events.
+
+   "control"   Used to control misc. simple emulated hardware devices
+               (e.g. vibrator, leds, LCD backlight, etc...)
+
+
+II. Implementation:
+-------------------
+
+Since the "cupcake" platform, this works as follows:
+
+- A 'qemud client' is any part of the emulated system that wants to talk
+  to the emulator. It does so by:
+
+  - connecting to the /dev/socket/qemud Unix domain socket
+  - sending the service name through the socket
+  - receives two bytes of data, which will be "OK" in case of
+    success, or "KO" in case of failure.
+
+  After an OK, the same connection can be used to talk directly to the
+  corresponding service.
+
+
+- The /dev/socket/qemud Unix socket is created by init and owned by the
+  'qemud' daemon started at boot by /system/etc/init.goldfish.rc
+
+  The daemon also opens an emulated serial port (e.g. /dev/ttyS1) and
+  will pass all messages between clients and emulator services. Thus,
+  everything looks like the following:
+
+
+     emulator <==serial==> qemud <---> /dev/socket/qemud <-+--> client1
+                                                           |
+                                                           +--> client2
+
+  A very simple multiplexing protocol is used on the serial connection:
+
+           offset    size    description
+
+               0       2     2-char hex string giving the destination or
+                             source channel
+
+               2       4     4-char hex string giving the payload size
+
+               6       n     the message payload
+
+  Where each client gets a 'channel' number allocated by the daemon
+  at connection time.
+
+  Note that packets larger than 65535 bytes cannot be sent directly
+  through the qemud channel. This is intentional; for large data
+  communication, the client and service should use a fragmentation
+  convention that deals with this.
+
+  Zero-sized packets are silently discard by qemud and the emulator and
+  should normally not appear on the serial port.
+
+  Channel 0 is reserved for control messages between the daemon and the
+  emulator. These are the following:
+
+    - When a client connects to /dev/socket/qemud and sends a service
+      name to the daemon, the later sends to the emulator:
+
+        connect:<service>:<id>
+
+      where <service> is the service name, and <id> is a 4-hexchar string
+      giving the allocated channel index for the client.
+
+
+    - The emulator can respond in case of success with:
+
+        ok:connect:<id>
+
+      or, in case of failure, with:
+
+        ok:connect:<id>:<reason>
+
+      where <reason> is a liberal string giving the reason for failure.
+      It is never sent to clients (which will only receive a "KO") and
+      is used strictly for debugging purposes.
+
+    - After a succesful connect, all messages between the client and
+      the corresponding emulator service will be passed through the
+      corresponding numbered channel.
+
+      But if the client disconnects from the socket, the daemon will
+      send through channel 0 this message to the emulator:
+
+        disconnect:<id>
+
+    - If an emulator service decides, for some reason, to disconnect
+      a client, the emulator will send to the daemon (on channel 0):
+
+        disconnect:<id>
+
+      The daemon deals with this gracefully (e.g. it will wait that the
+      client has read all buffered data in the daemon before closing the
+      socket, to avoid packet loss).
+
+    - Any other command sent from the daemon to the emulator will result
+      in the following answer:
+
+        ko:bad command
+
+- Which exact serial port to open is determined by the emulator at startup
+  and is passed to the system as a kernel parameter, e.g.:
+
+        android.qemud=ttyS1
+
+
+- The code to support services and their clients in the emulator is located
+  in android/hw-qemud.c. This code is heavily commented.
+
+  The daemon's source is in $ROOT/development/emulator/qemud/qemud.c
+
+  The header in $ROOT/hardware/libhardware/include/hardware/qemud.h
+  can be used by clients to ease connecting and talking to QEMUD-based
+  services.
+
+  This is used by $ROOT/developement/emulator/sensors/sensors_qemu.c which
+  implements emulator-specific sensor support in the system by talking to
+  the "sensors" service provided by the emulator (if available).
+
+  Code in $ROOT/hardware/libhardware_legacy also uses QEMUD-based services.
+
+
+- Certain services also implement a simple framing protocol when exchanging
+  messages with their clients. The framing happens *after* serial port
+  multiplexing and looks like:
+
+           offset    size    description
+
+               0       4     4-char hex string giving the payload size
+
+               4       n     the message payload
+
+  This is needed because the framing protocol used on the serial port is
+  not preserved when talking to clients through /dev/socket/qemud.
+
+  Certain services do not need it at all (GSM, GPS) so it is optional and
+  must be used depending on which service you talk to by clients.
+
+
+III. Legacy 'qemud':
+--------------------
+
+The system images provided by the 1.0 and 1.1 releases of the Android SDK
+implement an older variant of the qemud daemon that uses a slightly
+different protocol to communicate with the emulator.
+
+This is documented here since this explains some subtleties in the
+implementation code of android/hw-qemud.c
+
+The old scheme also used a serial port to allow the daemon and the emulator
+to communicate. Besides, the same multiplexing protocol was used to pass
+messages through multiple channels. However, several other differences
+exist, best illustrated by the following graphics:
+
+    emulator <==serial==> qemud <-+--> /dev/socket/qemud_gsm <--> GSM client
+                                  |
+                                  +--> /dev/socket/qemud_gps <--> GPS client
+                                  |
+                                  +--> /dev/socket/qemud_control <--> client(s)
+
+Now, for the details:
+
+ - instead of a single /dev/socket/qemud, init created several Unix domain
+   sockets, one per service:
+
+        /dev/socket/qemud_gsm
+        /dev/socket/qemud_gps
+        /dev/socket/qemud_control
+
+   note that there is no "sensors" socket in 1.0 and 1.1
+
+ - the daemon created a de-facto numbered channel for each one of these
+   services, even if no client did connect to it (only one client could
+   connect to a given service at a time).
+
+ - at startup, the emulator does query the channel numbers of all services
+   it implements, e.g. it would send *to* the daemon on channel 0:
+
+        connect:<service>
+
+   where <service> can be one of "gsm", "gps" or "control"
+
+   (Note that on the current implementation, the daemon is sending connection
+   messages to the emulator instead).
+
+ - the daemon would respond with either:
+
+        ok:connect:<service>:<hxid>
+
+   where <service> would be the service name, and <hxid> a 4-hexchar channel
+   number (NOTE: 4 chars, not 2). Or with:
+
+        ko:connect:bad name
+
+
+This old scheme was simpler to implement in both the daemon and the emulator
+but lacked a lot of flexibility:
+
+  - adding a new service required to modify /system/etc/init.goldfish.rc
+    as well as the daemon source file (which contained a hard-coded list
+    of sockets to listen to for client connections).
+
+  - only one client could be connected to a given service at a time,
+    except for the GPS special case which was a unidirectionnal broadcast
+    by convention.
+
+The current implementation moves any service-specific code to the emulator,
+only uses a single socket and allows concurrent clients for a all services.
diff --git a/docs/AUDIO.TXT b/docs/AUDIO.TXT
index 1a25b77..71ec288 100644
--- a/docs/AUDIO.TXT
+++ b/docs/AUDIO.TXT
@@ -34,7 +34,8 @@ SOUND PLAYBACK DETAILS:
 Each HWVoiceOut has the following too:
 
   - A fixed-size circular buffer of stereo samples (for stereo).
-    whose format is either floats or int64_t per sample (depending on build configuration).
+    whose format is either floats or int64_t per sample (depending on build
+    configuration).
 
   - A 'samples' field giving the (constant) number of sample pairs in the stereo buffer.
 
@@ -45,17 +46,32 @@ Each HWVoiceOut has the following too:
   - A 'rpos' offset into the circular buffer which tells where to read the next samples
     from the stereo buffer for the next conversion through 'clip'.
 
+
+            |<----------------- samples ----------------------->|
+
+            |                                                   |
+
+            |       rpos                                        |
+                    |
+            |_______v___________________________________________|
+            |       |                                           |
+            |       |                                           |
+            |_______|___________________________________________|
+
+
   - A 'run_out' method that is called each time to tell the output backend to
     send samples from the stereo buffer to the host sound card/server. This method
     shall also modify 'rpos' and returns the number of samples 'played'. A more detailed
     description of this process appears below.
 
   - A 'write' method callback used to write a buffer of emulated sound samples from
-    a SWVoiceOut into the stereo buffer. *All* backends simply call the generic
-    function audio_pcm_sw_write() to implement this. It's difficult to see why
-    it's needed at all ?
+    a SWVoiceOut into the stereo buffer. Currently all backends simply call the generic
+    function audio_pcm_sw_write() to implement this.
+
+    According to malc, the audio sub-system's original author, this is to allow
+    a backend to use a platform-specific function to do the same thing if available.
 
-    (Similarly, all backends have a 'read' methods which simply calls 'audio_pcm_sw_read')
+    (Similarly, all input backends have a 'read' methods which simply calls 'audio_pcm_sw_read')
 
 Each SWVoiceOut has the following:
 
@@ -67,6 +83,27 @@ Each SWVoiceOut has the following:
     HWVoiceOut's 'rpos' offset). NOTE: this is a count of samples in the HWVoiceOut
     stereo buffer, not emulated hardware sound samples, which can have different
     properties (frequency, size, endianess).
+                                         ______________
+                                        |              |
+                                        |  SWVoiceOut2 |
+                                        |______________|
+                  ______________           |
+                 |              |          |
+                 |  SWVoiceOut1 |          |     thsm<N> := total_hw_samples_mixed
+                 |______________|          |                for SWVoiceOut<N>
+                           |               |
+                           |               |
+                    |<-----|------------thsm2-->|
+                    |      |                    |
+                    |<---thsm1-------->|        |
+             _______|__________________v________|_______________ 
+            |       |111111111111111111|        v               |
+            |       |222222222222222222222222222|               |
+            |_______|___________________________________________|
+                    ^
+                    |         HWVoiceOut stereo buffer
+                    rpos
+
 
   - a 'ratio' value, which is the ratio of the target HWVoiceOut's frequency by
     the SWVoiceOut's frequency, multiplied by (1 << 32), as a 64-bit integer.
@@ -86,24 +123,19 @@ Each SWVoiceOut has the following:
 Here's a small graphics that explains it better:
 
    SWVoiceOut:  emulated hardware sound buffers:
-
-          |
-          |   (mixed through AUD_write() from user-provided callback)
           |
+          |   (mixed through AUD_write() called from user-provided
+          |    callback which is itself called on each audio timer
+          |    tick).
           v
-
    HWVoiceOut: stereo sample circular buffer
-
           |
-          |   (through HWVoiceOut's 'clip' function, invoked from the
-          |    'run_out' method)
+          |   (sent through HWVoiceOut's 'clip' function, which is
+          |    invoked from the 'run_out' method, also called on each
+          |    audio timer tick)
           v
-
    backend-specific sound buffers
 
-THERE IS NO COMMON TIMEBASE BETWEEN ALL LAYERS. DON'T EXPECT ANY HIGH-ACCURACY /
-LOW-LATENCY IN THIS IMPLEMENTATION.
-
 
 The function audio_timer() in audio/audio.c is called periodically and it is used as
 a pulse to perform sound buffer transfers and mixing. More specifically for audio
@@ -123,7 +155,7 @@ output voices:
 
 It's important to note that the SWVoiceOut callback:
 
-- takes a 'free' parameter which is the number of emulated sound samples that can
+- takes a 'free' parameter which is the number of stereo sound samples that can
   be sent to the hardware stereo buffer (before rate adjustment, i.e. not the number
   of sound samples in the SWVoiceOut emulated hardware sound buffer).
 
@@ -157,4 +189,7 @@ So, in the end, we have the pseudo-code:
 SOUND RECORDING DETAILS:
 ========================
 
-Things are similar but in reverse order.
+Things are similar but in reverse order. I.e. the HWVoiceIn acquires sound samples
+in its stereo sound buffer, and the SWVoiceIn objects must consume them as soon as
+they can.
+
diff --git a/docs/CHAR-DEVICES.TXT b/docs/CHAR-DEVICES.TXT
new file mode 100644
index 0000000..9805a4a
--- /dev/null
+++ b/docs/CHAR-DEVICES.TXT
@@ -0,0 +1,201 @@
+QEMU CHARACTER "DEVICES" MANAGEMENT
+
+I. CharDriverState objects:
+---------------------------
+
+One of the strangest abstraction in QEMU is the "CharDriverState"
+(abbreviated here as "CS").
+
+The CS is essentially an object used to model a character stream that
+can be connected to things like a host serial port, a host network socket,
+an emulated device, etc...
+
+What's really unusual is its interface though, which comes from the fact
+that QEMU implements a big event loop with no blocking i/o allowed. You
+can see "qemu-char.h" for the full interface, but here we will only describe
+a few important functions:
+
+  - qemu_chr_write() is used to try to write data into a CS object. Note that
+    success is not guaranteed: the function returns the number of bytes that
+    were really written (which can be 0) and the caller must deal with it.
+    This is very similar to writing to a non-blocking BSD socket on Unix.
+
+       int  qemu_chr_read( CharDriverState*  cs,
+                           const uint8_t*    data,
+                           int               datalen );
+
+    This function may return -1 in case of error, but this depends entirely
+    on the underlying implementation (some of them will just return 0 instead).
+    In practice, this means it's not possible to reliably differentiate between
+    a "connection reset by peer" and an "operation in progress" :-(
+
+    There is no way to know in advance how many bytes a given CharDriverState
+    can accept, nor to be notified when its underlying implementation is ready
+    to accept data again.
+
+
+  - qemu_chr_add_handler() is used to add "read" and "event" handlers
+    to a CS object. We will ignore "events" here and focus on the
+    "read" part.
+
+    Thing is, you cannot directly read from a CS object. Instead, you provide
+    two functions that will be called whenever the object has something for
+    you:
+
+        - a 'can_read' function that shall return the number of bytes
+          that you are ready to accept from the CharDriverState. Its
+          interface is:
+
+             typedef int  IOCanRWHandler (void*  opaque);
+
+        - a 'read' function that will send you bytes from the CharDriverState
+
+             typedef void IOReadHandler  (void*           opaque,
+                                          const uint8_t*  data,
+                                          int             datalen);
+
+          normally, the value of 'datalen' cannot be larger than the result
+          of a previous 'can_read' call.
+
+    For both callbacks, 'opaque' is a value that you pass to the function
+    qemu_chr_add_handler() which signature is:
+
+         void qemu_chr_add_handlers(CharDriverState *s,
+                                    IOCanRWHandler  *fd_can_read,
+                                    IOReadHandler   *fd_read,
+                                    IOEventHandler  *fd_event,
+                                    void            *opaque);
+
+  - qemu_chr_open() is used to create a new CharDriverState object from a
+    descriptive string, its interface is:
+
+         CharDriverState*  qemu_chr_open(const char*  filename);
+
+    there are various formats for acceptable 'filenames', and they correspond
+    to the parameters of the '-serial' QEMU option described here:
+
+       http://www.nongnu.org/qemu/qemu-doc.html#SEC10
+
+    For example:
+
+       "/dev/<file>" (Linux and OS X only):
+            connect to a host character device file (e.g. /dev/ttyS0)
+
+       "file:<filename>":
+            Write output to a given file (write only)
+
+       "stdio":
+            Standard input/output
+
+       "udp:[<remote_host>]:<remote_port>[@[<src_ip>]:<src_port>]":
+            Connect to a UDP socket for both read/write.
+
+       "tcp:[<host>]:<port>[,server][,nowait][,nodelay]"
+            Connect to a TCP socket either as a client or a server.
+
+            The 'nowait' option is used to avoid waiting for a client
+            connection.
+
+            The 'nodelay' is used to disable the TCP Nagle algorithm to
+            improve throughput.
+
+    For Android, a few special names have been added to the internal
+    implementation and redirect to program functions:
+
+       "android-kmsg":
+            A CharDriverState that is used to receive kernel log messages
+            from the emulated /dev/ttyS0 serial port.
+
+       "android-qemud":
+            A CharDriverState that is used to exchange messages between the
+            emulator program and the "qemud" multiplexing daemon that runs in
+            the emulated system.
+
+            The "qemud" daemon is used to allow one or more clients in the
+            system to connect to various services running in the emulator
+            program. This is mainly used to bypass the kernel in order to
+            implement certain features with ease.
+
+            (see docs/ANDROID-QEMUD.TXT for details)
+
+       "android-gsm":
+            A CharDriverState that is used to connect the emulated system to
+            a host modem device with the -radio <device> option. Otherwise,
+            the system uses qemud to connect to the emulator's internal modem
+            emulation.
+
+        "android-gps":
+            A CharDriverState that is used to connect the emulated system to a
+            host GPS device with the -gps <device> option. Otherwise the
+            system uses qemud to connect to the emulator's internal GPS
+            emulation.
+
+
+II. CharDriverState users:
+--------------------------
+
+As described above, a CharDriverState "user" is a piece of code that can write
+to a CharDriverState (by calling qemu_chr_write() explicitely) and can also
+read from it after registering can_read/read handlers for it through
+qemu_chr_add_handlers().
+
+Typical examples are the following:
+
+  - The hardware serial port emulation (e.g. hw/goldfish_tty.c) will read data
+    from the kernel then send it to a CS. It also uses a small buffer that is
+    used to read data from the CS and send it back to the kernel.
+
+  - The Android emulated modem also uses a CS to talk with its client,
+    which will in most cases be an emulated serial port.
+
+
+III. CharBuffer objects:
+------------------------
+
+The Android emulator provides an object called a CharBuffer which acts as
+a CharDriverState object that implements a *write* buffer to send data to a
+given CS object, called the endpoint. You can create one with:
+
+    #include "charpipe.h"
+    CharDriverState*  qemu_chr_open_buffer( CharDriverState*  endpoint );
+
+This function returns a new CS object that will buffer in the heap any data
+that is sent to it, but cannot be sent to the endpoint yet. On each event loop
+iteration, the CharBuffer will try to send data to the endpoint until it
+doesn't have any data left.
+
+This can be useful to simplify certain CS users who don't want to maintain
+their own emit buffer. Note that writing to a CharBuffer always succeeds.
+
+Note also that calling qemu_chr_add_handler() on the CharBuffer will do the
+same on the endpoint. Any endpoint-initiated calls to can_read()/read()
+callbacks are passed directly to your handler functions.
+
+
+IV. CharPipe objects:
+---------------------
+
+The Android emulator also provides a convenient abstraction called a "charpipe"
+used to connect two CharDriverState users together. For example, this is used
+to connect a serial port emulation (in hw/goldfish_tty.c) to the internal
+GSM modem emulation (see telephony/modem_driver.c).
+
+Essentially, a "charpipe" is a bi-directionnal communication pipe whose two
+endpoints are both CS objects. You call "qemu_chr_open_pipe()" to create the
+pipe, and this function will return the two endpoints to you:
+
+    #include "charpipe.h"
+    int  qemu_chr_open_pipe(CharDriverState* *pfirst,
+                            CharDriverState* *psecond);
+
+When you write to one end of the pipe (with qemu_chr_write()), the charpipe will
+try to write as much data as possible to the other end. Any remaining data is stored
+in a heap-allocated buffer.
+
+The charpipe will try to re-send the buffered data on the next event loop
+iteration by calling the can_read/read functions of the corresponding user,
+if there is one.
+
+Note that there is no limit on the amount of data buffered in a charpipe,
+and writing to it is never blocking. This simplifies CharDriverState
+users who don't need to worry about buffering issues.