API Usage

ImportantDue to NaCl deprecation by the Chromium project, Tizen TV will continue its support for NaCl only until 2021-year products. Meanwhile, Tizen TV will start focusing on high-performance, cross-browser WebAssembly from 2020-year products.

This topic describes how to implement a NaCl multimedia player for Samsung Smart TV.

Related Info

NaCl Player API

The NaCl (Native Client) Player API allows you to implement various multimedia playback modes in your application, depending on the source of the multimedia data. You can play content from a multimedia resource URL, or from elementary stream packets demuxed by the application. Both modes can handle DRM-protected content, as well as subtitles.

Prerequisites

To enable additional functionality for NaCl Player media playback:

To access content on the Internet, the application has to request permission by adding the following privilege to the "config.xml" file:
```
<tizen:privilege name='http://tizen.org/privilege/internet'></tizen:privilege> 
```
To play PlayReady-protected content, the application has to request permission by adding the following privilege to the "config.xml" file:
```
<tizen:privilege name='http://developer.samsung.com/privilege/drmplay'></tizen:privilege>
```

Preparing Managing Class

All examples below will be written as parts of implementation of some managing class, which is declared to have members:

// An instance of player
Samsung::NaClPlayer::MediaPlayer player_;

// An instance of class derived from Samsung::NaClPlayer::XxxListener,
// where Xxx is MediaEvents, Buffering, DRM or Subtitle
MyXxxListener xxx_listener_;

// Use either this or ESDataSource
Samsung::NaClPlayer::URLDataSource url_data_source_;
// Use either this or URLDataSource
Samsung::NaClPlayer::ESDataSource es_data_source_;

// Following members are needed only with ESDataSource:
// An instance of audio elementary stream
Samsung::NaClPlayer::AudioElementaryStream audio_stream_;
// An instance of video elementary stream
Samsung::NaClPlayer::VideoElementaryStream video_stream_;
// An instance of class derived from Samsung::NaClPlayer::ElementaryStreamListener
MyESListener audio_listener_;
// An instance of class derived from Samsung::NaClPlayer::ElementaryStreamListener
MyESListener video_listener_;

Initializing the Media Player

To implement playback using the NaCl Player API, you must initialize the following classes:

The MediaPlayer class is a representation of the platform player, which allows you to control it.
Creating the MediaPlayer object is parameter-free:
```
// Create the player object 
player_ = Samsung::NaClPlayer::MediaPlayer(); 
```
The MediaDataSource class, specialized as either a URLDataSource class or an ESDataSource class, is an abstract representation of a data source. Each specialization has a different initialization procedure. For more information, see Configuring Media Data Sources.

The initialized MediaDataSource object must be attached to the MediaPlayer object:

// Application-defined function to initialize the MediaDataSource-derived object
InitializeDataSource(es_data_source_);
// or InitializeDataSource(url_data_source_);

// Attach the initialized data source
player_.AttachDataSource(es_data_source_);
// or player_.AttachDataSource(url_data_source_);

ImportantPrefer using API methods that use raw pointers and don't take ownership of passed objects. There is also deprecated API that uses shared_pointers for backward compatibility, but it introduces risk of creating cyclic dependencies and is highly discouraged.

Registering Listeners

To receive notifications about NaCl Player events, you must implement listeners. The NaCl Player listener classes do not provide event handling by default; event handlers must be implemented by the application.

To receive notifications from the NaCl Player:

Override the appropriate listener class, and handle the virtual function associated with the event.

To receive playback notifications from the NaCl Player, override the MediaEventsListener class:

class MyMediaEventsListener : public Samsung::NaClPlayer::MediaEventsListener {
  public:
    // During playback, provides the current playback time
    void OnTimeUpdate(Samsung::NaClPlayer::TimeTicks time) override {
      // Handle the event...
    }

    // Triggers when playback has finished
    void OnEnded() override {
      // Handle the event...
    }

    // Triggers when a player error occurs
    void OnError(Samsung::NaClPlayer::MediaPlayerError) override {
      // Handle the event...
    }
};

To receive buffering notifications from the NaCl Player, override the BufferingListener class:

class MyBufferingListener : public Samsung::NaClPlayer::BufferingListener {
  public:
    // Triggers when the NaCl player starts buffering data
    void OnBufferingStart() override {
      // Handle the event...
    }

    // Provides the buffering progress
    void OnBufferingProgress(uint32_t percent) override {
      // Handle the event...
    }

    // Triggers when the NaCl Player finishes buffering data
    void OnBufferingComplete() override {
      // Handle the event...
    }
};

The meaning of the buffering events depends on the data source.

Event	Elementary stream source	URL source
`OnBufferingStart()`	This event is fired when a data source is attached to the MediaPlayer object. The NaCl Player data buffer is ready to be filled with elementary stream packets.	This event is fired when a data source is attached to the `MediaPlayer` object and during seek. The NaCl Player has started buffering.
`OnBufferingProgress()`	-	Sends information about the buffering completion percentage.
`OnBufferingComplete()`	The NaCl Player data buffer has received enough packets to start playback.	The NaCl Player has finished buffering. Playback can be started, or resumed after seeking.

Table 1. Buffering events

To receive DRM-related notifications from the NaCl Player, override the DRMListener class:

class MyDRMListener : public Samsung::NaClPlayer::DRMListener {
  public:
    DRMListener(const Samsung::NaClPlayer::MediaPlayer* player) 
      : player_(player) {
    }
  // Notifies when the DRM initialization data is loaded
  void OnInitdataLoaded(
      Samsung::NaClPlayer::DRMType drm_type, uint32_t init_data_size,
      const void* init_data) override {
    }

    // Notifies that a license is required to play protected content
    // The request parameter contains challenge data for obtaining the license
    void OnLicenseRequest(uint32_t request_size, const void* request) override {
      // Connect to a license server and download the license
      std::string license;
      // Fill the license...
      
      // Install the obtained license
      player_->SetDRMSpecificData(
          DRMType_/* type */, DRMOperation_InstallLicense,
          license.size(), license.data());
    }
  private:
    Samsung::NaClPlayer::MediaPlayer* player_;
};

When protected content needs to be decrypted, and there is no installed license that can decrypt it, the OnLicenseRequest() event is triggered. Generate the license request and install the license with the MediaPlayer::SetDRMSpecificData() function. If multiple tracks are protected by the same license, install it only once.

To receive subtitle change notifications from the NaCl Player, override the SubtitleListener class. The listener only notifies about subtitle changes in the currently-selected text track.

class SubtitleListener : public Samsung::NaClPlayer::SubtitleListener {
  public:
    // Provide the subtitle text and duration
    void OnShowSubtitle(
      Samsung::NaClPlayer::TimeDelta duration, const char* text) override {
        // Show the subtitle...
      }
  };

NoteYou must implement subtitle display in the application separately.

Register each listener in the MediaPlayer object, using the corresponding SetXxxListener() function:
```
// Create a listener and register it in the player 
xxx_listener_ = MyXxxListener(); // or MyXxxListener(&player) if needed 
player_.SetXxxListener(&xxx_listener_); 
```
NoteYou can only register 1 listener of each type. Registering a new listener of the same type automatically deregisters the previous one.
To unsubscribe from events, deregister the applicable listener:
```
player_.SetXxxListener(nullptr); 
```

Configuring Media Data Sources

Media data sources are configured in different ways, depending on whether the source is provided as a URL or an elementary stream.

URL Data Sources

The URLDataSource class allows you to play media from a specific URL source.

To create a URLDataSource object, call its constructor and attach it to the MediaPlayer object:

// Create a media data source for the URL and attach it to the player
url_data_source_ = Samsung::NaClPlayer::URLDataSource(media_content_url);
player_.AttachDataSource(url_data_source_);

Elementary Stream Data Sources

The ESDataSource class allows you to play media using demuxed elementary stream packets.

To create and configure a ESDataSource object:

Instantiate the ESDataSource class:

// Create the media data source
es_data_source_ = Samsung::NaClPlayer::ESDataSource();

Implement event handling using the ElementaryStreamListener class:

class MyESListener : public Samsung::NaClPlayer::ElementaryStreamListener {
 public:
  // Triggers when NaCl Player needs ES packets
  void OnNeedData(int32_t bytes) {
    // Send one or more packets to the NaCl Player
  }

  // Triggers when the NaCl Player ES packet buffer is full
  void OnEnoughData() {
    // Stop sending packets
  }

  // Triggers when the NaCl Player performs seek operations
  void OnSeekData(Samsung::NaClPlayer::TimeTicks new_position) {
    // Adjust the time position from which packets are sent
  }
};

Add elementary streams to the ESDataSource object and associate them with the listener:

// Add VideoElementaryStream
video_listener_ = MyESListener();
video_stream_ = Samsung::NaClPlayer::VideoElementaryStream();
es_data_source_.AddStream(video_stream_, &video_listener_);

// Add AudioElementaryStream
audio_listener_ = MyESListener();
audio_stream_ = Samsung::NaClPlayer::AudioElementaryStream();
es_data_source_->AddStream(audio_stream_, &audio_listener_);

Configure each elementary stream:

To configure an AudioElementaryStream object:

Define the parameters in the AudioConfig struct:

AudioConfig audio_config;
// Fill audio_config with the appropriate information

audio_stream_.SetAudioCodecType(audio_config.codec_type);
audio_stream_.SetAudioCodecProfile(audio_config.codec_profile);
audio_stream_.SetSampleFormat(audio_config.sample_format);
audio_stream_.SetChannelLayout(audio_config.channel_layout);
audio_stream_.SetBitsPerChannel(audio_config.bits_per_channel);
audio_stream_.SetSamplesPerSecond(audio_config.samples_per_second);
audio_stream_.SetCodecExtraData(audio_config.extra_data.size(),
                                 &audio_config.extra_data.front());

Confirm and save the configuration using the ElementaryStream::InitializeDone() function:
```
audio_stream_.InitializeDone();
```

To configure a VideoElementaryStream object:

Define the parameters in the VideoConfig struct:

VideoConfig video_config;
// Fill video_config with the appropriate information

video_stream_.SetVideoCodecType(video_config.codec_type);
video_stream_.SetVideoCodecProfile(video_config.codec_profile);
video_stream_.SetVideoFrameFormat(video_config.frame_format);
video_stream_.SetVideoFrameSize(video_config.size);
video_stream_.SetFrameRate(video_config.frame_rate);
video_stream_.SetCodecExtraData(video_config.extra_data.size(),
                                 &video_config.extra_data.front());

Confirm and save the configuration using the ElementaryStream::InitializeDone() function:
```
video_stream_.InitializeDone();
```

When the elementary streams have been configured, attach the ESDataSource object to the MediaPlayer object:
```
player_.AttachDataSource(es_data_source_);
```
The application can receive elementary stream events and start appending elementary stream packets.
Start appending packets to the buffer (as part of main playback loop if you implement push mode or as a response to ElementaryStreamListener::OnNeedData() event if you implement pull mode):
```
audio_stream_.AppendPacket(es_pkt);
video_stream_.AppendPacket(es_pkt);
```
When the MediaPlayer object sends the BufferingListener::OnBufferingComplete() event, the application can begin playback.

Playing DRM-protected Content

The NaCl Player can handle multimedia content protected by DRM technology. The logic for handling DRM-protected content varies depending on the data source.

DRM for URL Data Sources

In the URL data source scenario, the DRM is configured automatically by the platform, based on the DRM configuration stored in a file at a specific URL (for example, a DASH manifest file).

DRM for Elementary Stream Data Sources

In the elementary stream source scenario, the DRM engine must be configured by the application during stream configuration. When the DRM license is requested, the license information must be retrieved and installed.

In this paragraph we will go into details of implementing MyDRMListener, which was already mentioned earlier.

NoteAn application can also use a pre-installed license, instead of waiting for a license request. In this situation, the DRM initialization data simply needs to be set during stream configuration.

Define the DRM listener.
The DRMListener class listens for the OnLicenseRequest() event and reacts to it by installing the license required to play the DRM-protected content:

class MyDRMListener : public Samsung::NaClPlayer::DRMListener {
  public:
    MyDRMListener(const Samsung::NaClPlayer::MediaPlayer* player)
      : player_(player) {
  }

    // Notify that DRM protection data has been loaded
    void OnInitdataLoaded(
      Samsung::NaClPlayer::DRMType drm_type, uint32_t init_data_size,
      const void* init_data) override;

    // Signal that a license is required to play the current content
    void OnLicenseRequest(uint32_t request_size, const void* request) override;

  private:
    // OnLicenseRequest calls this function when it downloads a license 
    void InstallLicense(std:: string license);
    Samsung::NaClPlayer::MediaPlayer* player_;
};

// Create the listener and register it in the player
drm_listener_ = MyDRMListener(&player_);
player_.SetDRMListener(&drm_listener_);

For more information, see Registering Listeners.

Configure the elementary streams.
The DRM information for each elementary stream must be configured at the same time as the stream's other parameters, using the ElementaryStream::SetDRMInitData() function:

// ...
// Add a video elementary stream as in an earlier example
es_data_source_.AddStream(video_stream_, &video_listener_);
// Configure the video stream as we did before
// and add DRM init data before calling ElementaryStream::InitializeDone()
video_stream_.SetDRMInitData(encryption_type, init_data_size, init_data_ptr); 
video_stream_.InitializeDone();

// Repeat the above for an audio elementary stream...
// ...

Request and install the DRM license.
If the license required to play the current DRM-protected content is not available, the OnLicenseRequest() event is triggered. The DRMListener() event handler must download the license information from the license server and pass it to the NaCl Player.

// Download a license asynchronously and call the license_downloaded_cb() functor
void DownloadLicense(
  std::string license_server_url, uint32_t request_size, const void* request,
  std::function<std::string> license_downloaded_cb);

void MyDRMListener::OnLicenseRequest(
  uint32_t request_size, const void* request) {
  // Download a valid license from the license server
  // For application responsiveness, download the license asynchronously
  DownloadLicense("license server URL", request_size, request, std::bind(
    &MyDRMListener::InstallLicense, this, std::placeholders::_1));
}

void MyDRMListener::InstallLicense(std::string license) {
  int32_t result = player_.SetDRMSpecificData(
    Samsung::NaClPlayer::DRMType_/* type */,
    Samsung::NaClPlayer::DRMOperation_InstallLicense,
    license.size(), license.data());
  if (result < Samsung::NaClPlayer::ErrorCodes::Success) {
    // Handle the error...
  }
}

NoteThe location of the license server URL depends on the usage scenario. For example, a DASH stream with CENC protection stores the license server URL in a manifest file.

Interact with the DRM system using the MediaPlayer::SetDRMSpecificData()function.

NoteNot all DRM systems support all available NaCl Player DRM operations.

Supported Player Modes

Tizen Version	default	D2TV
2.4	?	?
3.0	?	?
4.0	?	?
5.0	?	?