docs/html/ndk/guides/audio/opensl-prog-notes.jd - LeafOS-Project/android_frameworks_base - Gitiles

 page.title=OpenSL ES Programming Notes
 @jd:body

 <div id="qv-wrapper">
     <div id="qv">
       <h2>On this page</h2>

       <ol>
         <li><a href="#init">Objects and Interface Initialization</a></li>
         <li><a href="#prefetch">Audio Player Prefetch</a></li>
         <li><a href="#destroy">Destroy</a></li>
         <li><a href="#panning">Stereo Panning</a></li>
         <li><a href="#callbacks">Callbacks and Threads</a></li>
         <li><a href="#perform">Performance</a></li>
         <li><a href="#sandp">Security and Permissions</a></li>
       </ol>
     </div>
 </div>

 <p>
 The notes in this section supplement the
 <a class="external-link" href="https://www.khronos.org/registry/sles/">OpenSL ES 1.0.1
 specification</a>.
 </p>

 <h2 id="init">Objects and Interface Initialization</h2>

 <p>
 Two aspects of the OpenSL ES programming model that may be unfamiliar to new developers are the
 distinction between objects and interfaces, and the initialization sequence.
 </p>

 <p>
 Briefly, an OpenSL ES object is similar to the object concept in
  programming languages such as Java
 and C++, except an OpenSL ES object is only visible via its associated interfaces.
  This includes
 the initial interface for all objects, called {@code SLObjectItf}.
  There is no handle for an object
 itself, only a handle to the {@code SLObjectItf} interface of the object.
 </p>

 <p>
 An OpenSL ES object is first <em>created</em>, which returns an {@code SLObjectItf}, then
 <em>realized</em>. This is similar to the common programming pattern of first constructing an
 object (which should never fail other than for lack of memory or invalid parameters), and then
 completing initialization (which may fail due to lack of resources). The realize step gives the
 implementation a logical place to allocate additional resources if needed.
 </p>

 <p>
 As part of the API to create an object, an application specifies an array of desired interfaces
 that it plans to acquire later. Note that this array does not automatically
  acquire the interfaces;
 it merely indicates a future intention to acquire them. Interfaces are distinguished as
 <em>implicit</em> or <em>explicit</em>. An explicit interface must be listed in the array if it
 will be acquired later. An implicit interface need not be listed in the
  object create array, but
 there is no harm in listing it there. OpenSL ES has one more kind of interface called
 <em>dynamic</em>, which does not need to be specified in the object
  create array and can be added
 later after the object is created. The Android implementation provides
  a convenience feature to
 avoid this complexity, which is described in
  <a href="{@docRoot}ndk/guides/audio/opensl-for-android.html#dynamic-interfaces">Dynamic interfaces at object creation</a>.
 </p>

 <p>
 After the object is created and realized, the application should acquire interfaces for each
 feature it needs, using {@code GetInterface} on the initial {@code SLObjectItf}.
 </p>

 <p>
 Finally, the object is available for use via its interfaces, though note that
  some objects require
 further setup. In particular, an audio player with URI data source needs a bit
  more preparation in
 order to detect connection errors. See the
  <a href="#prefetch">Audio player prefetch</a> section for details.
 </p>

 <p>
 After your application is done with the object, you should explicitly destroy it; see the
 <a href="#destroy">Destroy</a> section below.
 </p>

 <h2 id="prefetch">Audio Player Prefetch</h2>

 <p>
 For an audio player with URI data source, {@code Object::Realize} allocates
  resources but does not
 connect to the data source (<em>prepare</em>) or begin pre-fetching data. These occur once the
 player state is set to either {@code SL_PLAYSTATE_PAUSED} or {@code SL_PLAYSTATE_PLAYING}.
 </p>

 <p>
 Some information may still be unknown until relatively late in this sequence. In
 particular, initially {@code Player::GetDuration} returns {@code SL_TIME_UNKNOWN} and
 {@code MuteSolo::GetChannelCount} either returns successfully with channel count zero or the
 error result {@code SL_RESULT_PRECONDITIONS_VIOLATED}. These APIs return the proper values
 once they are known.
 </p>

 <p>
 Other properties that are initially unknown include the sample rate and
  actual media content type
 based on examining the content's header (as opposed to the
  application-specified MIME type and
 container type). These are also determined later during
  prepare/prefetch, but there are no APIs to
 retrieve them.
 </p>

 <p>
 The prefetch status interface is useful for detecting when all information
  is available, or your
 application can poll periodically. Note that some information, such as the
  duration of a streaming
 MP3, may <em>never</em> be known.
 </p>

 <p>
 The prefetch status interface is also useful for detecting errors. Register a callback
  and enable
 at least the {@code SL_PREFETCHEVENT_FILLLEVELCHANGE} and {@code SL_PREFETCHEVENT_STATUSCHANGE}
 events. If both of these events are delivered simultaneously, and
 {@code PrefetchStatus::GetFillLevel} reports a zero level, and
 {@code PrefetchStatus::GetPrefetchStatus} reports {@code SL_PREFETCHSTATUS_UNDERFLOW},
  then this
 indicates a non-recoverable error in the data source. This includes the inability to
  connect to the
 data source because the local filename does not exist or the network URI is invalid.
 </p>

 <p>
 The next version of OpenSL ES is expected to add more explicit support for
  handling errors in the
 data source. However, for future binary compatibility, we intend to continue
  to support the current
 method for reporting a non-recoverable error.
 </p>

 <p>
 In summary, a recommended code sequence is:
 </p>

 <ol>
   <li>{@code Engine::CreateAudioPlayer}</li>
   <li>{@code Object:Realize}</li>
   <li>{@code Object::GetInterface} for {@code SL_IID_PREFETCHSTATUS}</li>
   <li>{@code PrefetchStatus::SetCallbackEventsMask}</li>
   <li>{@code PrefetchStatus::SetFillUpdatePeriod}</li>
   <li>{@code PrefetchStatus::RegisterCallback}</li>
   <li>{@code Object::GetInterface} for {@code SL_IID_PLAY}</li>
   <li>{@code Play::SetPlayState} to {@code SL_PLAYSTATE_PAUSED}, or
   {@code SL_PLAYSTATE_PLAYING}</li>
 </ol>

 <p class="note"><strong>Note: </strong>
 Preparation and prefetching occur here; during this time your callback is called with
 periodic status updates.
 </p>

 <h2 id="destroy">Destroy</h2>

 <p>
 Be sure to destroy all objects when exiting from your application.
  Objects should be destroyed in
 reverse order of their creation, as it is not safe to destroy an object that has any dependent
 objects. For example, destroy in this order: audio players and recorders, output mix, and then
 finally the engine.
 </p>

 <p>
 OpenSL ES does not support automatic garbage collection or
 <a class="external-link" href="http://en.wikipedia.org/wiki/Reference_counting">reference
 counting</a> of interfaces. After you call {@code Object::Destroy}, all extant
  interfaces that are
 derived from the associated object become undefined.
 </p>

 <p>
 The Android OpenSL ES implementation does not detect the incorrect use of such interfaces.
 Continuing to use such interfaces after the object is destroyed can cause your application to
 crash or behave in unpredictable ways.
 </p>

 <p>
 We recommend that you explicitly set both the primary object interface and all associated
 interfaces to NULL as part of your object destruction sequence, which prevents the accidental
 misuse of a stale interface handle.
 </p>

 <h2 id="panning">Stereo Panning</h2>

 <p>
 When {@code Volume::EnableStereoPosition} is used to enable stereo panning of a mono source,
  there is a 3-dB reduction in total
 <a class="external-link" href="http://en.wikipedia.org/wiki/Sound_power_level">sound power
 level</a>. This is needed to permit the total sound power level to remain constant as
  the source is
 panned from one channel to the other. Therefore, only enable stereo positioning if you need
 it. See the Wikipedia article on
 <a class="external-link" href="http://en.wikipedia.org/wiki/Panning_(audio)">audio panning</a>
  for more information.
 </p>

 <h2 id="callbacks">Callbacks and Threads</h2>

 <p>
 Callback handlers are generally called synchronously with respect to the event. That is, at the
 moment and location that the event is detected by the implementation. This point is
 asynchronous with respect to the application, so you should use a non-blocking synchronization
 mechanism to control access to any variables shared between the application and the callback
 handler. In the example code, such as for buffer queues, we have either omitted this
 synchronization or used blocking synchronization in the interest of simplicity. However, proper
 non-blocking synchronization is critical for any production code.
 </p>

 <p>
 Callback handlers are called from internal non-application threads that are not attached to the
 Android runtime, so they are ineligible to use JNI. Because these internal threads are
 critical to
 the integrity of the OpenSL ES implementation, a callback handler should also not block
  or perform
 excessive work.
 </p>

 <p>
 If your callback handler needs to use JNI or execute work that is not proportional to the
 callback, the handler should instead post an event for another thread to process. Examples of
 acceptable callback workload include rendering and enqueuing the next output buffer
 (for an AudioPlayer), processing the just-filled input buffer and enqueueing the next
  empty buffer
 (for an AudioRecorder), or simple APIs such as most of the <em>Get</em> family. See the
 <a href="#perform">Performance</a> section below regarding the workload.
 </p>

 <p>
 Note that the converse is safe: an Android application thread that has entered JNI
  is allowed to
 directly call OpenSL ES APIs, including those that block. However, blocking calls are not
 recommended from the main thread, as they may result in
  <em>Application Not Responding</em> (ANR).
 </p>

 <p>
 The determination regarding the thread that calls a callback handler is largely left up to the
 implementation. The reason for this flexibility is to permit future optimizations,
  especially on
 multi-core devices.
 </p>

 <p>
 The thread on which the callback handler runs is not guaranteed to have the same
  identity across
 different calls. Therefore, do not rely on the {@code pthread_t returned by pthread_self()}
  or the
 {@code pid_t returned by gettid()} to be consistent across calls. For the same reason,
  do not use
 the thread local storage (TLS) APIs such as {@code pthread_setspecific()} and
 {@code pthread_getspecific()} from a callback.
 </p>

 <p>
 The implementation guarantees that concurrent callbacks of the same kind, for the
  same object, does
 not occur. However, concurrent callbacks of different kinds for the same object are possible on
 different threads.
 </p>

 <h2 id="perform">Performance</h2>

 <p>
 As OpenSL ES is a native C API, non-runtime application threads that call OpenSL ES have no
 runtime-related overhead such as garbage collection pauses. With one exception described below,
 there is no additional performance benefit to the use of OpenSL ES other than this.
  In particular,
 the use of OpenSL ES does not guarantee enhancements such as lower audio latency and higher
 scheduling priority over that which the platform generally provides. On the other hand, as the
 Android platform and specific device implementations continue to evolve, an OpenSL ES application
 can expect to benefit from any future system performance improvements.
 </p>

 <p>
 One such evolution is support for reduced
 <a href="{@docRoot}ndk/guides/audio/output-latency.html">audio output latency</a>.
 The underpinnings for reduced
 output latency were first included in Android 4.1 (API level 16), and then
 continued progress occurred in Android 4.2 (API level 17). These improvements are available via
 OpenSL ES for device implementations that
  claim feature {@code android.hardware.audio.low_latency}.
 If the device doesn't claim this feature but supports Android 2.3 (API level 9)
 or later, then you can still use the OpenSL ES APIs but the output latency may be higher.
  The lower
 output latency path is used only if the application requests a buffer size and sample rate
  that are
 compatible with the device's native output configuration. These parameters are
  device-specific and
 should be obtained as described below.
 </p>

 <p>
 Beginning with Android 4.2 (API level 17), an application can query for the
 platform native or optimal output sample rate and buffer size for the device's primary output
 stream. When combined with the feature test just mentioned, an app can now configure itself
 appropriately for lower latency output on devices that claim support.
 </p>

 <p>
 For Android 4.2 (API level 17) and earlier, a buffer count of two or more is
 required for lower latency. Beginning with Android 4.3 (API level 18), a buffer
 count of one is sufficient for lower latency.
 </p>

 <p>
 All OpenSL ES interfaces for output effects preclude the lower latency path.
 </p>

 <p>
 The recommended sequence is as follows:
 </p>

 <ol>
   <li>Check for API level 9 or higher to confirm the use of OpenSL ES.</li>
   <li>Check for the {@code android.hardware.audio.low_latency} feature using code such as this:
     <pre>import android.content.pm.PackageManager;
 ...
 PackageManager pm = getContext().getPackageManager();
 boolean claimsFeature = pm.hasSystemFeature(PackageManager.FEATURE_AUDIO_LOW_LATENCY);
     </pre></li>
   <li>Check for API level 17 or higher to confirm the use of
   {@code android.media.AudioManager.getProperty()}.</li>
   <li>Get the native or optimal output sample rate and buffer size for this device's
   primary output
   stream using code such as this:
     <pre>import android.media.AudioManager;
 ...
 AudioManager am = (AudioManager) getSystemService(Context.AUDIO_SERVICE);
 String sampleRate = am.getProperty(AudioManager.PROPERTY_OUTPUT_SAMPLE_RATE));
 String framesPerBuffer = am.getProperty(AudioManager.PROPERTY_OUTPUT_FRAMES_PER_BUFFER));
     </pre>
   Note that {@code sampleRate} and {@code framesPerBuffer} are <em>strings</em>. First check for
   null and then convert to int using {@code Integer.parseInt()}.</li>
     <li>Now use OpenSL ES to create an AudioPlayer with PCM buffer queue data locator.</li>
 </ol>

 <p class="note"><strong>Note: </strong>
 You can use the
 <a class="external-link"
  href="https://play.google.com/store/apps/details?id=com.levien.audiobuffersize">
  Audio Buffer Size</a>
 test app to determine the native buffer size and sample rate for OpenSL ES audio
 applications on your audio device. You can also visit GitHub to view <a class="external-link"
 href="https://github.com/gkasten/high-performance-audio/tree/master/audio-buffer-size">
 audio-buffer-size</a> samples.

 <p>
 The number of lower latency audio players is limited. If your application requires more
 than a few
 audio sources, consider mixing your audio at the application level. Be sure to destroy your audio
 players when your activity is paused, as they are a global resource shared with other apps.
 </p>

 <p>
 To avoid audible glitches, the buffer queue callback handler must execute within a small and
 predictable time window. This typically implies no unbounded blocking on mutexes, conditions,
 or I/O operations. Instead consider <em>try locks</em>, locks and waits with timeouts, and
 <a class="external-link"
  href="https://source.android.com/devices/audio/avoiding_pi.html#nonBlockingAlgorithms">
  non-blocking algorithms</a>.
 </p>

 <p>
 The computation required to render the next buffer (for AudioPlayer) or consume the previous
 buffer (for AudioRecord) should take approximately the same amount of time for each callback.
 Avoid algorithms that execute in a non-deterministic amount of time or are <em>bursty</em> in
 their computations. A callback computation is bursty if the CPU time spent in any given callback
 is significantly larger than the average. In summary, the ideal is for the CPU execution time of
 the handler to have variance near zero, and for the handler to not block for unbounded times.
 </p>

 <p>
 Lower latency audio is possible for these outputs only:
 </p>

 <ul>
   <li>On-device speakers.</li>
   <li>Wired headphones.</li>
   <li>Wired headsets.</li>
   <li>Line out.</li>
   <li><a class="external-link" href="https://source.android.com/devices/audio/usb.html">
   USB digital
   audio</a>.</li>
 </ul>

 <p>
 On some devices, speaker latency is higher than other paths due to digital signal processing for
 speaker correction and protection.
 </p>

 <p>
 As of API level 21,
 <a href="{@docRoot}ndk/guides/audio/input-latency.html">lower latency audio input</a>
  is supported
 on select devices. To take advantage of
 this feature, first confirm that lower latency output is available as described above. The
 capability for lower latency output is a prerequisite for the lower latency input feature. Then,
 create an AudioRecorder with the same sample rate and buffer size as would be used for output.
 OpenSL ES interfaces for input effects preclude the lower latency path. The record preset
 {@code SL_ANDROID_RECORDING_PRESET_VOICE_RECOGNITION} must be used for lower latency; this preset
 disables device-specific digital signal processing that may add latency to the input path. For
 more information on record presets, see the <a href="#configuration-interface">Android
 configuration interface</a> section above.
 </p>

 <p>
 For simultaneous input and output, separate buffer queue completion handlers are used for each
 side. There is no guarantee of the relative order of these callbacks, or the synchronization of
 the audio clocks, even when both sides use the same sample rate. Your application
  should buffer the
 data with proper buffer synchronization.
 </p>

 <p>
 One consequence of potentially independent audio clocks is the need for asynchronous sample rate
 conversion. A simple (though not ideal for audio quality) technique for asynchronous sample rate
 conversion is to duplicate or drop samples as needed near a zero-crossing point.
  More sophisticated
 conversions are possible.
 </p>

 <h2 id="sandp">Security and Permissions</h2>

 <p>
 As far as who can do what, security in Android is done at the process level. Java programming
 language code cannot do anything more than native code, nor can native code do anything more than
 Java programming language code. The only differences between them are the available APIs.
 </p>

 <p>
 Applications using OpenSL ES must request the permissions that they would need for similar
 non-native APIs. For example, if your application records audio, then it needs the
 {@code android.permission.RECORD_AUDIO} permission. Applications that use audio effects need
 {@code android.permission.MODIFY_AUDIO_SETTINGS}. Applications that play network URI resources
 need {@code android.permission.NETWORK}. See
 <a href="https://developer.android.com/training/permissions/index.html">Working with System
 Permissions</a> for more information.
 </p>

 <p>
 Depending on the platform version and implementation, media content parsers and
  software codecs may
 run within the context of the Android application that calls OpenSL ES (hardware codecs are
 abstracted but are device-dependent). Malformed content designed to exploit parser and codec
 vulnerabilities is a known attack vector. We recommend that you play media only from trustworthy
 sources or that you partition your application such that code that handles media from
 untrustworthy sources runs in a relatively <em>sandboxed</em> environment. For example, you could
 process media from untrustworthy sources in a separate process. Though both processes would still
 run under the same UID, this separation does make an attack more difficult.
 </p>
	page.title=OpenSL ES Programming Notes
	@jd:body

	<div id="qv-wrapper">
	<div id="qv">
	<h2>On this page</h2>

	<ol>
	<li><a href="#init">Objects and Interface Initialization</a></li>
	<li><a href="#prefetch">Audio Player Prefetch</a></li>
	<li><a href="#destroy">Destroy</a></li>
	<li><a href="#panning">Stereo Panning</a></li>
	<li><a href="#callbacks">Callbacks and Threads</a></li>
	<li><a href="#perform">Performance</a></li>
	<li><a href="#sandp">Security and Permissions</a></li>
	</ol>
	</div>
	</div>

	<p>
	The notes in this section supplement the
	<a class="external-link" href="https://www.khronos.org/registry/sles/">OpenSL ES 1.0.1
	specification</a>.
	</p>

	<h2 id="init">Objects and Interface Initialization</h2>

	<p>
	Two aspects of the OpenSL ES programming model that may be unfamiliar to new developers are the
	distinction between objects and interfaces, and the initialization sequence.
	</p>

	<p>
	Briefly, an OpenSL ES object is similar to the object concept in
	programming languages such as Java
	and C++, except an OpenSL ES object is only visible via its associated interfaces.
	This includes
	the initial interface for all objects, called {@code SLObjectItf}.
	There is no handle for an object
	itself, only a handle to the {@code SLObjectItf} interface of the object.
	</p>

	<p>
	An OpenSL ES object is first <em>created</em>, which returns an {@code SLObjectItf}, then
	<em>realized</em>. This is similar to the common programming pattern of first constructing an
	object (which should never fail other than for lack of memory or invalid parameters), and then
	completing initialization (which may fail due to lack of resources). The realize step gives the
	implementation a logical place to allocate additional resources if needed.
	</p>

	<p>
	As part of the API to create an object, an application specifies an array of desired interfaces
	that it plans to acquire later. Note that this array does not automatically
	acquire the interfaces;
	it merely indicates a future intention to acquire them. Interfaces are distinguished as
	<em>implicit</em> or <em>explicit</em>. An explicit interface must be listed in the array if it
	will be acquired later. An implicit interface need not be listed in the
	object create array, but
	there is no harm in listing it there. OpenSL ES has one more kind of interface called
	<em>dynamic</em>, which does not need to be specified in the object
	create array and can be added
	later after the object is created. The Android implementation provides
	a convenience feature to
	avoid this complexity, which is described in
	<a href="{@docRoot}ndk/guides/audio/opensl-for-android.html#dynamic-interfaces">Dynamic interfaces at object creation</a>.
	</p>

	<p>
	After the object is created and realized, the application should acquire interfaces for each
	feature it needs, using {@code GetInterface} on the initial {@code SLObjectItf}.
	</p>

	<p>
	Finally, the object is available for use via its interfaces, though note that
	some objects require
	further setup. In particular, an audio player with URI data source needs a bit
	more preparation in
	order to detect connection errors. See the
	<a href="#prefetch">Audio player prefetch</a> section for details.
	</p>

	<p>
	After your application is done with the object, you should explicitly destroy it; see the
	<a href="#destroy">Destroy</a> section below.
	</p>

	<h2 id="prefetch">Audio Player Prefetch</h2>

	<p>
	For an audio player with URI data source, {@code Object::Realize} allocates
	resources but does not
	connect to the data source (<em>prepare</em>) or begin pre-fetching data. These occur once the
	player state is set to either {@code SL_PLAYSTATE_PAUSED} or {@code SL_PLAYSTATE_PLAYING}.
	</p>

	<p>
	Some information may still be unknown until relatively late in this sequence. In
	particular, initially {@code Player::GetDuration} returns {@code SL_TIME_UNKNOWN} and
	{@code MuteSolo::GetChannelCount} either returns successfully with channel count zero or the
	error result {@code SL_RESULT_PRECONDITIONS_VIOLATED}. These APIs return the proper values
	once they are known.
	</p>

	<p>
	Other properties that are initially unknown include the sample rate and
	actual media content type
	based on examining the content's header (as opposed to the
	application-specified MIME type and
	container type). These are also determined later during
	prepare/prefetch, but there are no APIs to
	retrieve them.
	</p>

	<p>
	The prefetch status interface is useful for detecting when all information
	is available, or your
	application can poll periodically. Note that some information, such as the
	duration of a streaming
	MP3, may <em>never</em> be known.
	</p>

	<p>
	The prefetch status interface is also useful for detecting errors. Register a callback
	and enable
	at least the {@code SL_PREFETCHEVENT_FILLLEVELCHANGE} and {@code SL_PREFETCHEVENT_STATUSCHANGE}
	events. If both of these events are delivered simultaneously, and
	{@code PrefetchStatus::GetFillLevel} reports a zero level, and
	{@code PrefetchStatus::GetPrefetchStatus} reports {@code SL_PREFETCHSTATUS_UNDERFLOW},
	then this
	indicates a non-recoverable error in the data source. This includes the inability to
	connect to the
	data source because the local filename does not exist or the network URI is invalid.
	</p>

	<p>
	The next version of OpenSL ES is expected to add more explicit support for
	handling errors in the
	data source. However, for future binary compatibility, we intend to continue
	to support the current
	method for reporting a non-recoverable error.
	</p>

	<p>
	In summary, a recommended code sequence is:
	</p>

	<ol>
	<li>{@code Engine::CreateAudioPlayer}</li>
	<li>{@code Object:Realize}</li>
	<li>{@code Object::GetInterface} for {@code SL_IID_PREFETCHSTATUS}</li>
	<li>{@code PrefetchStatus::SetCallbackEventsMask}</li>
	<li>{@code PrefetchStatus::SetFillUpdatePeriod}</li>
	<li>{@code PrefetchStatus::RegisterCallback}</li>
	<li>{@code Object::GetInterface} for {@code SL_IID_PLAY}</li>
	<li>{@code Play::SetPlayState} to {@code SL_PLAYSTATE_PAUSED}, or
	{@code SL_PLAYSTATE_PLAYING}</li>
	</ol>

	<p class="note"><strong>Note: </strong>
	Preparation and prefetching occur here; during this time your callback is called with
	periodic status updates.
	</p>

	<h2 id="destroy">Destroy</h2>

	<p>
	Be sure to destroy all objects when exiting from your application.
	Objects should be destroyed in
	reverse order of their creation, as it is not safe to destroy an object that has any dependent
	objects. For example, destroy in this order: audio players and recorders, output mix, and then
	finally the engine.
	</p>

	<p>
	OpenSL ES does not support automatic garbage collection or
	<a class="external-link" href="http://en.wikipedia.org/wiki/Reference_counting">reference
	counting</a> of interfaces. After you call {@code Object::Destroy}, all extant
	interfaces that are
	derived from the associated object become undefined.
	</p>

	<p>
	The Android OpenSL ES implementation does not detect the incorrect use of such interfaces.
	Continuing to use such interfaces after the object is destroyed can cause your application to
	crash or behave in unpredictable ways.
	</p>

	<p>
	We recommend that you explicitly set both the primary object interface and all associated
	interfaces to NULL as part of your object destruction sequence, which prevents the accidental
	misuse of a stale interface handle.
	</p>

	<h2 id="panning">Stereo Panning</h2>

	<p>
	When {@code Volume::EnableStereoPosition} is used to enable stereo panning of a mono source,
	there is a 3-dB reduction in total
	<a class="external-link" href="http://en.wikipedia.org/wiki/Sound_power_level">sound power
	level</a>. This is needed to permit the total sound power level to remain constant as
	the source is
	panned from one channel to the other. Therefore, only enable stereo positioning if you need
	it. See the Wikipedia article on
	<a class="external-link" href="http://en.wikipedia.org/wiki/Panning_(audio)">audio panning</a>
	for more information.
	</p>

	<h2 id="callbacks">Callbacks and Threads</h2>

	<p>
	Callback handlers are generally called synchronously with respect to the event. That is, at the
	moment and location that the event is detected by the implementation. This point is
	asynchronous with respect to the application, so you should use a non-blocking synchronization
	mechanism to control access to any variables shared between the application and the callback
	handler. In the example code, such as for buffer queues, we have either omitted this
	synchronization or used blocking synchronization in the interest of simplicity. However, proper
	non-blocking synchronization is critical for any production code.
	</p>

	<p>
	Callback handlers are called from internal non-application threads that are not attached to the
	Android runtime, so they are ineligible to use JNI. Because these internal threads are
	critical to
	the integrity of the OpenSL ES implementation, a callback handler should also not block
	or perform
	excessive work.
	</p>

	<p>
	If your callback handler needs to use JNI or execute work that is not proportional to the
	callback, the handler should instead post an event for another thread to process. Examples of
	acceptable callback workload include rendering and enqueuing the next output buffer
	(for an AudioPlayer), processing the just-filled input buffer and enqueueing the next
	empty buffer
	(for an AudioRecorder), or simple APIs such as most of the <em>Get</em> family. See the
	<a href="#perform">Performance</a> section below regarding the workload.
	</p>

	<p>
	Note that the converse is safe: an Android application thread that has entered JNI
	is allowed to
	directly call OpenSL ES APIs, including those that block. However, blocking calls are not
	recommended from the main thread, as they may result in
	<em>Application Not Responding</em> (ANR).
	</p>

	<p>
	The determination regarding the thread that calls a callback handler is largely left up to the
	implementation. The reason for this flexibility is to permit future optimizations,
	especially on
	multi-core devices.
	</p>

	<p>
	The thread on which the callback handler runs is not guaranteed to have the same
	identity across
	different calls. Therefore, do not rely on the {@code pthread_t returned by pthread_self()}
	or the
	{@code pid_t returned by gettid()} to be consistent across calls. For the same reason,
	do not use
	the thread local storage (TLS) APIs such as {@code pthread_setspecific()} and
	{@code pthread_getspecific()} from a callback.
	</p>

	<p>
	The implementation guarantees that concurrent callbacks of the same kind, for the
	same object, does
	not occur. However, concurrent callbacks of different kinds for the same object are possible on
	different threads.
	</p>

	<h2 id="perform">Performance</h2>

	<p>
	As OpenSL ES is a native C API, non-runtime application threads that call OpenSL ES have no
	runtime-related overhead such as garbage collection pauses. With one exception described below,
	there is no additional performance benefit to the use of OpenSL ES other than this.
	In particular,
	the use of OpenSL ES does not guarantee enhancements such as lower audio latency and higher
	scheduling priority over that which the platform generally provides. On the other hand, as the
	Android platform and specific device implementations continue to evolve, an OpenSL ES application
	can expect to benefit from any future system performance improvements.
	</p>

	<p>
	One such evolution is support for reduced
	<a href="{@docRoot}ndk/guides/audio/output-latency.html">audio output latency</a>.
	The underpinnings for reduced
	output latency were first included in Android 4.1 (API level 16), and then
	continued progress occurred in Android 4.2 (API level 17). These improvements are available via
	OpenSL ES for device implementations that
	claim feature {@code android.hardware.audio.low_latency}.
	If the device doesn't claim this feature but supports Android 2.3 (API level 9)
	or later, then you can still use the OpenSL ES APIs but the output latency may be higher.
	The lower
	output latency path is used only if the application requests a buffer size and sample rate
	that are
	compatible with the device's native output configuration. These parameters are
	device-specific and
	should be obtained as described below.
	</p>

	<p>
	Beginning with Android 4.2 (API level 17), an application can query for the
	platform native or optimal output sample rate and buffer size for the device's primary output
	stream. When combined with the feature test just mentioned, an app can now configure itself
	appropriately for lower latency output on devices that claim support.
	</p>

	<p>
	For Android 4.2 (API level 17) and earlier, a buffer count of two or more is
	required for lower latency. Beginning with Android 4.3 (API level 18), a buffer
	count of one is sufficient for lower latency.
	</p>

	<p>
	All OpenSL ES interfaces for output effects preclude the lower latency path.
	</p>

	<p>
	The recommended sequence is as follows:
	</p>

	<ol>
	<li>Check for API level 9 or higher to confirm the use of OpenSL ES.</li>
	<li>Check for the {@code android.hardware.audio.low_latency} feature using code such as this:
	<pre>import android.content.pm.PackageManager;
	...
	PackageManager pm = getContext().getPackageManager();
	boolean claimsFeature = pm.hasSystemFeature(PackageManager.FEATURE_AUDIO_LOW_LATENCY);
	</pre></li>
	<li>Check for API level 17 or higher to confirm the use of
	{@code android.media.AudioManager.getProperty()}.</li>
	<li>Get the native or optimal output sample rate and buffer size for this device's
	primary output
	stream using code such as this:
	<pre>import android.media.AudioManager;
	...
	AudioManager am = (AudioManager) getSystemService(Context.AUDIO_SERVICE);
	String sampleRate = am.getProperty(AudioManager.PROPERTY_OUTPUT_SAMPLE_RATE));
	String framesPerBuffer = am.getProperty(AudioManager.PROPERTY_OUTPUT_FRAMES_PER_BUFFER));
	</pre>
	Note that {@code sampleRate} and {@code framesPerBuffer} are <em>strings</em>. First check for
	null and then convert to int using {@code Integer.parseInt()}.</li>
	<li>Now use OpenSL ES to create an AudioPlayer with PCM buffer queue data locator.</li>
	</ol>

	<p class="note"><strong>Note: </strong>
	You can use the
	<a class="external-link"
	href="https://play.google.com/store/apps/details?id=com.levien.audiobuffersize">
	Audio Buffer Size</a>
	test app to determine the native buffer size and sample rate for OpenSL ES audio
	applications on your audio device. You can also visit GitHub to view <a class="external-link"
	href="https://github.com/gkasten/high-performance-audio/tree/master/audio-buffer-size">
	audio-buffer-size</a> samples.

	<p>
	The number of lower latency audio players is limited. If your application requires more
	than a few
	audio sources, consider mixing your audio at the application level. Be sure to destroy your audio
	players when your activity is paused, as they are a global resource shared with other apps.
	</p>

	<p>
	To avoid audible glitches, the buffer queue callback handler must execute within a small and
	predictable time window. This typically implies no unbounded blocking on mutexes, conditions,
	or I/O operations. Instead consider <em>try locks</em>, locks and waits with timeouts, and
	<a class="external-link"
	href="https://source.android.com/devices/audio/avoiding_pi.html#nonBlockingAlgorithms">
	non-blocking algorithms</a>.
	</p>

	<p>
	The computation required to render the next buffer (for AudioPlayer) or consume the previous
	buffer (for AudioRecord) should take approximately the same amount of time for each callback.
	Avoid algorithms that execute in a non-deterministic amount of time or are <em>bursty</em> in
	their computations. A callback computation is bursty if the CPU time spent in any given callback
	is significantly larger than the average. In summary, the ideal is for the CPU execution time of
	the handler to have variance near zero, and for the handler to not block for unbounded times.
	</p>

	<p>
	Lower latency audio is possible for these outputs only:
	</p>

	<ul>
	<li>On-device speakers.</li>
	<li>Wired headphones.</li>
	<li>Wired headsets.</li>
	<li>Line out.</li>
	<li><a class="external-link" href="https://source.android.com/devices/audio/usb.html">
	USB digital
	audio</a>.</li>
	</ul>

	<p>
	On some devices, speaker latency is higher than other paths due to digital signal processing for
	speaker correction and protection.
	</p>

	<p>
	As of API level 21,
	<a href="{@docRoot}ndk/guides/audio/input-latency.html">lower latency audio input</a>
	is supported
	on select devices. To take advantage of
	this feature, first confirm that lower latency output is available as described above. The
	capability for lower latency output is a prerequisite for the lower latency input feature. Then,
	create an AudioRecorder with the same sample rate and buffer size as would be used for output.
	OpenSL ES interfaces for input effects preclude the lower latency path. The record preset
	{@code SL_ANDROID_RECORDING_PRESET_VOICE_RECOGNITION} must be used for lower latency; this preset
	disables device-specific digital signal processing that may add latency to the input path. For
	more information on record presets, see the <a href="#configuration-interface">Android
	configuration interface</a> section above.
	</p>

	<p>
	For simultaneous input and output, separate buffer queue completion handlers are used for each
	side. There is no guarantee of the relative order of these callbacks, or the synchronization of
	the audio clocks, even when both sides use the same sample rate. Your application
	should buffer the
	data with proper buffer synchronization.
	</p>

	<p>
	One consequence of potentially independent audio clocks is the need for asynchronous sample rate
	conversion. A simple (though not ideal for audio quality) technique for asynchronous sample rate
	conversion is to duplicate or drop samples as needed near a zero-crossing point.
	More sophisticated
	conversions are possible.
	</p>

	<h2 id="sandp">Security and Permissions</h2>

	<p>
	As far as who can do what, security in Android is done at the process level. Java programming
	language code cannot do anything more than native code, nor can native code do anything more than
	Java programming language code. The only differences between them are the available APIs.
	</p>

	<p>
	Applications using OpenSL ES must request the permissions that they would need for similar
	non-native APIs. For example, if your application records audio, then it needs the
	{@code android.permission.RECORD_AUDIO} permission. Applications that use audio effects need
	{@code android.permission.MODIFY_AUDIO_SETTINGS}. Applications that play network URI resources
	need {@code android.permission.NETWORK}. See
	<a href="https://developer.android.com/training/permissions/index.html">Working with System
	Permissions</a> for more information.
	</p>

	<p>
	Depending on the platform version and implementation, media content parsers and
	software codecs may
	run within the context of the Android application that calls OpenSL ES (hardware codecs are
	abstracted but are device-dependent). Malformed content designed to exploit parser and codec
	vulnerabilities is a known attack vector. We recommend that you play media only from trustworthy
	sources or that you partition your application such that code that handles media from
	untrustworthy sources runs in a relatively <em>sandboxed</em> environment. For example, you could
	process media from untrustworthy sources in a separate process. Though both processes would still
	run under the same UID, this separation does make an attack more difficult.
	</p>