ImportantDue to NaCl deprecation by the Chromium project, Tizen TV will continue its support for NaCl only until 2021-year products. Meanwhile, Tizen TV will start focusing on high-performance, cross-browser WebAssembly from 2020-year products.
This topic describes the "Audio in C++" sample application implementation.
This tutorial describes how to implement a simple audio playback application that can load and play multiple audio files. The user can load WAV sounds and start, pause, and stop their playback.
For information on how to access the sample application cheat sheet and run the application, see Sample-based Tutorials.
In the sample application source code, pay attention to the following member variables in the AudioInstance class:
audio_ is a pp::Audio object.
header_ is a pointer to a WAV file header.
file_data_bytes_ is the WAV file data.
sound_instances_ is a vector of sound instances read from the WAV file.
active_sounds_ is an array of the active sound instances.
file_names_ is a vector of WAV file names.
Initializing the Instance and Context
To implement audio playback, you must initialize the instance and the audio context:
In the class constructor, initialize the required members, callback factory, and the Logger class:
Define the audio buffer size by retrieving the recommended sample frame count for the given audio parameters, using the RecommendSampleFrameCount() function.
Initialize the audio playback configuration object with the retrieved sample frame count.
Initialize the pp::Audio instance with the AudioConfig object and the AudioCallback() function pointer. The AudioCallback() function implements the main audio playback logic: it is called repeatedly during playback, and mixes the audio data from multiple files.
bool AudioInstance::Init(uint32_t /*argc*/, const char* /*argn*/[],
const char* /*argv*/[]) {
// Retrieve the recommended sample frame count
sample_frame_count_ =
pp::AudioConfig::RecommendSampleFrameCount(this,
PP_AUDIOSAMPLERATE_44100,
SAMPLE_FRAME_COUNT);
// Create an audio resource configuration with a 44.1 kHz sample rate
// and the recommended sample frame count
pp::AudioConfig audio_config = pp::AudioConfig(this,
PP_AUDIOSAMPLERATE_44100,
sample_frame_count_);
audio_ = pp::Audio(this, // Pointer to pp::Instance
audio_config,
AudioCallback,
this); // argument of type void* for the AudioCallback call
return true;
}
Implementing Playback Controls
When a playback control button is clicked, the JavaScript application component uses the PostMessage() method to notify the Native Client (NaCl) plugin.
In the NaCl plugin, parse the received message using the HandleMessage() function and pass the data to the appropriate playback control function:
void AudioInstance::HandleMessage(const pp::Var& var_message) {
if (var_message.is_string()) {
const std::string message = var_message.AsString();
if (StartsWith(message, kPlayCommand)) {
// Start playback
Play(GetIdFromMessage(message, kPlayCommand));
} else if (StartsWith(message, kPauseCommand)) {
// Pause playback
Pause(GetIdFromMessage(message, kPauseCommand));
} else if (StartsWith(message, kStopCommand)) {
// Stop playback
Stop(GetIdFromMessage(message, kStopCommand));
} else if (StartsWith(message, kLoadCommand)) {
// Receive the file URL and load it
PrepareReadingFile(message.substr(strlen(kLoadCommand)));
}
}
}
To implement the playback controls:
To load and read an audio file:
Load the WAV file to a data buffer using the PrepareReadingFile() function.
In the ReadWAVE() function, interpret the WAV data:
Copy the input string to a char vector, and cast the pointer for the data vector to the WAVEFileHeader structure pointer. This structure checks whether the header is correctly formatted.
If the header check succeeds, cast the address for the beginning of the rest of the data array to a smart pointer.
Create a SoundInstance object to contain the audio data.
Push the SoundInstance object to the sound_instances_ collection, from where it can be retrieved for playback.
void AudioInstance::ReadWAVE(const std::string& data) {
Logger::Log("Interpreting WAVE data of file %s",
file_names_[file_number_].c_str());
// Clean and create a buffer for interpreting file data
file_data_bytes_.reset(new char[data.size() + 1]);
// Copy file data to the buffer
std::copy(data.begin(), data.end(), file_data_bytes_.get());
file_data_bytes_[data.size()] = '\0';
// Read the header data
header_ = reinterpret_cast<WAVEFileHeader*>(file_data_bytes_.get());
// Check for malformed header
std::string header_error = CheckForWAVEHeaderErrors(*header_);
if (!header_error.empty()) {
Logger::Error("Unsupported file %s - bad header:\n%s",
file_names_[file_number_].c_str(), header_error.c_str());
PostMessage(CreateCommandMessage(kErrorMessage, file_number_));
return;
}
std::unique_ptr<uint16_t> sample_data(reinterpret_cast<uint16_t*>(
file_data_bytes_.get() + sizeof(WAVEFileHeader)));
// Create a sound instance, fill its fields, and add it
// to the sound instances vector
std::shared_ptr<SoundInstance> instance(new SoundInstance(file_number_,
header_->num_channels, (data.size() - sizeof(WAVEFileHeader)) /
sizeof(uint16_t), &sample_data));
sound_instances_.insert(std::pair<int, std::shared_ptr<SoundInstance> >(
instance->id_, instance));
// Notify the JavaScript component
PostMessage(CreateCommandMessage(kFileLoadedMessage, file_number_));
Logger::Log("File %s data loaded",
file_names_[file_number_].c_str());
}
To start playback:
Check whether any sounds are currently active and playing.
If no sounds are active, start playback by using the StartPlayback() function to initiate the audio callback loop.
Flag the sound as active and increment the active sounds counter.
Notify the JavaScript component that playback has started.
void AudioInstance::Play(int number) {
// Start playing if nothing is active
if (active_sounds_number_ == 0) {
audio_.StartPlayback();
Logger::Log("Playing started");
}
assert((number - 1 >= 0) || (number - 1 < kNumberOfInputSounds));
active_sounds_[number - 1] = true;
++active_sounds_number_;
// Notify the JavaScript component
PostMessage(CreateCommandMessage(kPlayMessage, number));
}
To pause playback:
Flag the sound as inactive and decrement the active sounds counter.
If there are no active sounds, stop the audio callback loop using the StopPlayback() function.
Notify the JavaScript component that playback has paused.
void AudioInstance::Pause(int number) {
// Set the sound as inactive
--active_sounds_number_;
assert((number - 1 >= 0) || (number - 1 < kNumberOfInputSounds));
active_sounds_[number - 1] = false;
// Stop playing if nothing is active
if (active_sounds_number_ == 0) {
audio_.StopPlayback();
Logger::Log("Playing stopped");
}
// Notify the JavaScript component
PostMessage(CreateCommandMessage(kPauseMessage, number));
}
To stop playback:
Flag the sound as inactive and decrement the active sounds counter.
If there are no active sounds, stop the audio callback loop using the StopPlayback() function.
Reset the sample_data_offset_ variable, so the file plays from the beginning the next time playback starts.
Notify the JavaScript component that playback has stopped.
void AudioInstance::Stop(int number) {
// Set the sound as inactive
assert((number - 1 >= 0) || (number - 1 < kNumberOfInputSounds));
if (active_sounds_[number - 1]) {
--active_sounds_number_;
active_sounds_[number - 1] = false;
}
// Stop playing if nothing is active
if (active_sounds_number_ == 0) {
audio_.StopPlayback();
Logger::Log("Playing stopped");
}
// Reset the playback offset
sound_instances_[number]->sample_data_offset_ = 0;
// Notify the JavaScript component
PostMessage(CreateCommandMessage(kStopMessage, number));
}
Implementing the Audio Callback Loop
The audio callback loop fills the audio buffer with sample data and mixes data from multiple sources in real time. During playback, the AudioCallback() function is called regularly. Each iteration fills the buffer with the next chunk of data.
Cast the pointer to the AudioInstance object and buffer, and fill the buffer with zeroes.
To maintain the same overall volume level regardless of the number of sounds playing simultaneously, calculate a volume modifier. The modifer is defined as the inverse square root of the active sound count.
Fill the buffer with data. For each SoundInstance object in the sound_instances_ map:
Check whether the end of the sound data has been reached.
Write the data chunk located at the SoundInstance object data offset to the buffer using the SafeAdd() function. This constrains the buffer values to between a minimum and maximum value for each sample.
Increment the data offset in the SoundInstance object. The data offset allows the buffer to resume filling from the last position the next time the AudioCallback() function is called.
If the end of any SoundInstance data is reached, notify the JavaScript component.
void AudioInstance::AudioCallback(void* samples, uint32_t buffer_size,
void* data) {
// Instance pointer is passed when creating Audio resource
AudioInstance* audio_instance = reinterpret_cast<AudioInstance*>(data);
// Fill the audio buffer
int16_t* buff = reinterpret_cast<int16_t*>(samples);
memset(buff, 0, buffer_size);
assert(audio_instance->active_sounds_number_ > 0);
// Compute the volume of the sound instances
const double volume = 1.0 / sqrt(audio_instance->active_sounds_number_);
// Prevent writing outside the buffer
assert(buffer_size >= (sizeof(*buff) * MAX_CHANNELS_NUMBER *
audio_instance->sample_frame_count_));
for (SoundInstances::iterator it =
audio_instance->sound_instances_.begin(); it !=
audio_instance->sound_instances_.end(); ++it) {
std::shared_ptr<SoundInstance> instance(it->second);
// Main loop for writing samples to audio buffer
for (size_t i = 0; i < audio_instance->sample_frame_count_; ++i) {
// If there are samples to play
if (audio_instance->active_sounds_[instance->id_ - 1] &&
instance->sample_data_offset_ < instance->sample_data_size_) {
audio_instance->SafeAdd(&buff[2 * i], volume *
(int16_t)instance->sample_data_[instance->sample_data_offset_]);
// For mono sound, each sample is passed to both channels, but for
// stereo sound, samples are written successively to all channels
if (instance->channels_ == 2) {
++instance->sample_data_offset_;
}
audio_instance->SafeAdd(&buff[2 * i + 1], volume *
(int16_t)instance->sample_data_[instance->sample_data_offset_]);
++instance->sample_data_offset_;
}
}
// When the end of a sound is reached, stop playing it
if (instance->sample_data_offset_ == instance->sample_data_size_) {
// Pepper API calls are normally avoided in audio callbacks,
// as they can cause audio dropouts when the audio callback thread
// is swapped out
// Audio dropout is not a concern here because the audio has ended
audio_instance->PostMessage(audio_instance->CreateCommandMessage(
kReturnStopMessage, instance->id_));
}
}
}
Manage Your Cookies
We use cookies to improve your experience on our website and to show you relevant
advertising. Manage you settings for our cookies below.
Essential Cookies
These cookies are essential as they enable you to move around the website. This
category cannot be disabled.
Company
Domain
Samsung Electronics
.samsungdeveloperconference.com
Analytical/Performance Cookies
These cookies collect information about how you use our website. for example which
pages you visit most often. All information these cookies collect is used to improve
how the website works.
Company
Domain
LinkedIn
.linkedin.com
Meta (formerly Facebook)
.samsungdeveloperconference.com
Google Inc.
.samsungdeveloperconference.com
Functionality Cookies
These cookies allow our website to remember choices you make (such as your user name, language or the region your are in) and
tailor the website to provide enhanced features and content for you.
Company
Domain
LinkedIn
.ads.linkedin.com, .linkedin.com
Advertising Cookies
These cookies gather information about your browser habits. They remember that
you've visited our website and share this information with other organizations such
as advertisers.
Company
Domain
LinkedIn
.linkedin.com
Meta (formerly Facebook)
.samsungdeveloperconference.com
Google Inc.
.samsungdeveloperconference.com
Preferences Submitted
You have successfully updated your cookie preferences.