This topic covers using advanced Tizen WASM Player features such as multitasking and DRM, as well as detecting which WASM Player features are available on a device.
Detecting Available WASM Player Features
Features supported by the WASM Player and the support for the WASM Player itself can vary depending on the platform version. Availability of the features (and presence of the ElementaryMediaStreamSource class implementation) can be determined at runtime by checking the EmssVersionInfo structure entries:
// Query platform for WASM Player features
auto features = samsung::wasm::EmssVersionInfo::Create();
if (features.has_emss || features.has_legacy_emss) {
// Above condition evaluates to true if ElementaryMediaStreamSource data
// source is available on the platform
std::cout << "Platform supports WASM Player!" << std::endl;
}
Additionally, if the WASM Player is not available on the platform but a source object is created, its ElementaryMediaStreamSource::IsValid() method always returns false.
Getting Detailed WASM Player Version Information
Use the EmssVersionInfo structure to check features supported by WASM Player on the current device. However, this method does not provide the WASM Player version information directly. Detailed version information can be checked with Tizen TV WASM versioning APIs:
#include <algorithm>
#include <samsung/wasm/tizen_tv_api_info.h>
// ...
using TizenTVApiInfo = samsung::wasm::TizenTVApiInfo;
auto apis = samsung::wasm::GetAvailableApis();
auto api_iterator =
std::find_if(apis.begin(), apis.end(), [](const TizenTVApiInfo& api) {
return api.name == "ElementaryMediaStreamSource";
});
if (api_iterator != apis.end()) {
// WASM Player is available on the current device! api_iterator entries can
// be checked for detailed version information.
}
The TizenTVApiInfo structure has the following entries:
name, as seen above. The WASM Player is identified by its main class name: ElementaryMediaStreamSource.
version, which is a string in format: API_LEVEL.DETAILED_VERSION (such as 1.0).
api_levels, a std::vector<uint32_t> of all API levels supported on the current device.
The API level identifies a single WASM Player API revision, which maps to EmssVersionInfo entries. Detailed version is an internal implementation revision ID.
Features and Compatibility per Tizen Version
WASM Player features available on the current device can be conveniently checked by looking up the EmssVersionInfo structure entries:
EmssVersionInfo Entry
Description
API Level
has_ultra_low_latency
WASM Player LatencyMode::kUltraLow mode is available on this device
5
has_low_latency_video_texture
WASM Player LatencyMode::kLow in conjunction with RenderingMode::kVideoTexture is available on this device
4
has_decoding_mode
DecodingMode::kSoftware and DecodingMode::kHardwareWithFallback are available on this device
WASM Player is available on the device in the legacy version; please see below for notes on compatibility.
0
Table 1: EmssVersionInfo Structure
Presence of ElementaryMediaStreamSource can be determined by checking both has_emss and has_legacy_emss flags. Only one of these two flags can be true at a time. Depending on the playback scenario, the application can be obliged to distinguish between these two WASM Player versions, since has_legacy_emss being true comes with certain limitations:
Seek operations for has_legacy_emss require extra care, see the following guidelines:
Seek bursts (multiple consecutive seeks occurring in close succession) must be avoided. Essentially, no new seek can start while HTMLMediaElement.seeking is true.
The application must not call any ElementaryMediaTrack::AppendPacket()s after HTMLMediaElement::SetCurrentTime() is called. No packets can be sent to the WASM Player until ElementaryMediaTrackListener::OnTrackOpen() event is fired after the Seek.
HTMLMediaElement loop is executed before the playback position reaches duration.
The above differences affect only the normal latency mode. Low latency modes work the same on both versions.
Multitasking
Multitasking happens when the application is either hidden from the screen or moved back to being shown on the device's screen (such as when the user switches the active application). The application needs to always properly handle Multitasking.
When the application is suspended (in other words, when visibilitychange reports document.hidden changing to true):
If ElementaryMediaStreamSource is ReadyState::kOpen:
ElementaryMediaStreamSource state is changed to ReadyState::kOpenPending and ElementaryMediaStreamSourceListener::OnSourceOpenPending() is fired.
ElementaryMediaTracks are closed and ElementaryMediaTrackListener::OnTrackClosed() with CloseReason::kSourceSuspended is fired for each track.
If the HTMLMediaElement::IsPaused() attribute is false, the HTMLMediaElement is paused and HTMLMediaElement::GetCurrentTime() is stored internally by WASM Player.
(This step is omitted in low latency modes)
When the application is resumed (in other words, when visibilitychange reports document.hidden changing to false):
If ElementaryMediaStreamSource was ReadyState::kOpen prior to suspend, an automatic Seek is performed to the stored HTMLMediaElement::GetCurrentTime() value.
As a result, ElementaryMediaStreamSource is set to ReadyState::kOpen and each ElementaryMediaTrack is opened.
(In low latency modes, the Seek operation is omitted; instead the source and tracks are opened immediately.)
If HTMLMediaElement::IsPaused() was false before playback being suspended, the HTMLMediaElement is unpaused.
Threading
Media applications tend to make use of threads, to both speed up multimedia processing and free the main thread that is responsible for handling the UI. While JavaScript lacks convenient thread support, WebAssembly offers full thread support, so using threads with the WASM Player is encouraged.
The main application thread is a JS main thread that runs a JS event loop. This is the thread that runs all JS events (this includes UI/user interaction and the application's JS code not running in WebWorkers) and that can also execute WASM code. However, the application must not execute long running operations in WASM module on the main thread.
The application must offload work to a number of worker threads to increase performance and ensure UI responsiveness. For a multimedia application, worker threads can be used to, for example:
Download data (For example, when sockets are used)
Process content (Such as. demux containers)
Send and process Elementary Media Packets
Thread Affinity of Listeners
All events emitted by listeners related to the WASM Player (ElementaryMediaStreamSourceListener, ElementaryMediaTrackListener, and HTMLMediaElementListener) are delivered through the main JS thread.
Please note:
Event handlers must not include any long-running operations; if one is necessary, dispatch it to a worker thread.
Excessive locking of the main JS thread is not advised, and must be avoided whenever possible.
Session ID
A session ID is an ElementaryMediaTrack parameter associated with all ElementaryMediaPacket objects sent to a track between OnTrackOpen and OnTrackClose.
A session starts when the track opens (OnTrackOpen) and lasts until it closes (OnTrackClose). All packets sent between those two events belong to a single session. When appending either a packet or an end of track to ElementaryMediaTrack, the application must mark them with the current session_id value.
When session ID changes, an ElementaryMediaTrackListener::OnSessionIdChanged() event is fired:
class MyTrackListener : public samsung::wasm::ElementaryMediaTrackListener {
// ...
void OnSessionIdChanged(uint32_t session_id) override {
// Update stored session_id: propagate it to components managing
// ElementaryMediaTracks
}
// ...
};
NoteElementaryMediaPackets marked with non-current session IDs are dropped by the platform. This causes AppendPacket() to return an OperationResult::kAppendIgnored error. However, this is not a fatal condition and does not impair playback in a properly written application.
Using DRM
NoteThis section does not apply in low latency modes, as they do not support DRM.
Then WASM Player supports DRM-protected content playback. Supported formats are the same as those specified in the EME section in Media Specifications.
To set up encrypted media playback in the application:
Prepare a DRMConfig structure:
samsung::wasm::DRMConfig drm_config;
drm_config.cdm = /* samsung::wasm::ContentDecryptionModule value */;
drm_config.encryption_mode = /* samsung::wasm::EncryptionMode value */;
drm_config.license_server /* URL to a license server */;
drm_config.init_data = /* DRM system-specific initialization data */;
// If audio track is encrypted:
drm_config.audio_mime_type = /* MIME type of encrypted audio or empty string if track is clear */;
drm_config.audio_robustness = /* samsung::wasm::Robustness parameter for audio track */;
// If video track is encrypted:
drm_config.video_mime_type = /* MIME type of encrypted video or empty string if track is clear */;
drm_config.video_robustness = /* samsung::wasm::Robustness parameter for video track */;
Create a MediaKey using the DRMConfig created in the previous step and assign it to the corresponding ElementaryMediaTracks:
using OperationResult = samsung::wasm::OperationResult;
using MediaKey = samsung::wasm::MediaKey;
// ...
// Created MediaKey must be stored by the application:
MediaKey decryption_key;
// ...
auto result = samsung::wasm::MediaKey::SetupEncryption(drm_config, [](OperationResult result, MediaKey media_key) {
if (result != OperationResult::kSuccess) {
// Handle error
return;
}
// Store the key. Make sure it outlives tracks it is assigned to below.
decryption_key = std::move(media_key);
// If audio is encrypted...
audio_track->SetMediaKey(&decryption_key)
// If video is encrypted...
video_track->SetMediaKey(&decryption_key)
})
NoteWhen creating a MediaKey instance, implementation automatically contacts the license server specified in the drm_config.license_server parameter.
When multimedia content is DRM-protected, the application needs to follow normal usage guidelines for Elementary Media Stream source except for one difference: encrypted Elementary Media Packets are sent to ElementaryMediaTracks with an AppendEncryptedPacketAsync method:
samsung::wasm::EncryptedElementaryMediaPacket encrypted_packet;
// ...
// Set all clear packet information as usual
// ...
encrypted_packet.subsamples = /* Subsample information: a vector of samsung::wasm::EncryptedSubsampleDescription */;
encrypted_packet.key_id = /* Id of a key that should be used to decrypt this packet */;
encrypted_packet.initialization_vector = /* IV required to decrypt this packet */;
encrypted_packet.encryption_mode = /* samsung::wasm::EncryptionMode for this packet */;
auto result = track->AppendEncryptedPacketAsync(encrypted_packet);
if (!result) {
// Handle error
}
NoteAppendEncryptedPacketAsync() validates the packet and returns a result synchronously. However, packet decryption is executed asynchronously. If a decryption error occurs, it is signaled with an ElementaryMediaTrackListener::OnAppendError() event. For details, see Handling async append errors.
To receive the decryption result synchronously, it is possible to use an AppendEncryptedPacket() operation instead. However, decrypting packets synchronously is not recommended: decryption is potentially a long running operation and blocking the append thread for extended periods of time can reduce video playback performance.
Appends for the same track can be a mix of synchronous and asynchronous calls.
Manage Your Cookies
We use cookies to improve your experience on our website and to show you relevant
advertising. Manage you settings for our cookies below.
Essential Cookies
These cookies are essential as they enable you to move around the website. This
category cannot be disabled.
Company
Domain
Samsung Electronics
.samsungdeveloperconference.com
Analytical/Performance Cookies
These cookies collect information about how you use our website. for example which
pages you visit most often. All information these cookies collect is used to improve
how the website works.
Company
Domain
LinkedIn
.linkedin.com
Meta (formerly Facebook)
.samsungdeveloperconference.com
Google Inc.
.samsungdeveloperconference.com
Functionality Cookies
These cookies allow our website to remember choices you make (such as your user name, language or the region your are in) and
tailor the website to provide enhanced features and content for you.
Company
Domain
LinkedIn
.ads.linkedin.com, .linkedin.com
Advertising Cookies
These cookies gather information about your browser habits. They remember that
you've visited our website and share this information with other organizations such
as advertisers.
Company
Domain
LinkedIn
.linkedin.com
Meta (formerly Facebook)
.samsungdeveloperconference.com
Google Inc.
.samsungdeveloperconference.com
Preferences Submitted
You have successfully updated your cookie preferences.