Use AR Emoji on Games and 3D Apps


Objective

Learn how to use AR Emoji as character of games or 3D applications using Galaxy AR Emoji SDK for Unity.

Partnership Request

To use the Galaxy AR Emoji SDK for Unity, you must become an official Samsung partner. Once done, you can fully utilize this Code Lab. You can learn more about the partnership process by visiting Galaxy AR Emoji SDK page, here in Samsung Developers.

Overview

Galaxy AR Emoji SDK for Unity

With your Samsung Galaxy device, you can use the AR Emoji feature to create an animated 3D avatar from your selfie photo or from a selection of a few suggested characters. The avatar can display a range of emotions and body movements. The Galaxy AR Emoji SDK for Unity enables you to use this said AR Emoji avatar in your application, such as using it to represent a game character.

Tutorial Process

In this Code Lab, you will get to know about Unity packages that can help you to use AR Emoji avatars as character models in your Unity application like games with simple code updates.

You will first learn about the basic pipeline for creation of app with AR Emoji character with AR Emoji SDK by loading a character model and rendering on the mobile app. Afterwards, you will learn a way of applying custom animation to the AR Emoji character.

You can make your own app with a simple sequence of operation and by following the steps in this Code Lab. Moreover, you can learn about the character file import process and get to know more how you can maximize Galaxy AR Emoji SDK.

Set up your environment

You will need the following:

  • Unity for Windows (LTS Release 2020.3.Xf1 or latest compatible version)
  • Editor for editing Unity script (like Visual Studio)
  • Select different values to test how your game will look on monitors with different aspect ratios. This is set to free aspect by default. (Recommended: 1024x2048)
  • Samsung Galaxy device supporting AR Emoji

Create your own AR Emoji

First, create your own AR Emoji with the following steps:

  1. Run Camera application.

  2. Click More > AR Zone > AR Emoji Camera and the + button.

  3. Take your selfie and choose your gender.

  4. Edit your AR Emoji.

    a. Customize the looks, clothes, or accessories of your AR Emoji.

    b. Press Next, once done.

  5. Check your avatar in both Mask mode and in AR Emoji Studio.

Create a Unity project and configure the settings

  1. Run Unity and create an empty 3D sample project.

  2. Go to Build Settings.

    a. File > Build Settings

    b. Platform > Android

    c. Click Switch Platform

    Specific package name and keystore are necessary for testing. In the next steps, it is explained in detail.

  3. Go to Edit > Player Settings > Player > Other Settings > Identification >
    Override Default Package Name     and enable it. Set the Package Name to samsung.com.fileprovidersample.

  4. Go to Edit > Project Setting > Player > Publishing Settings.

    a. Enable Custom Keystore.

    b. Add keystore file (fileprovidersample.jks) with browser (included in Docs folder of gettingAREmojiV3.3.unitypackage).

    c. Enter the Keystore Password: 123456

    d. Set the Project Key setting:

    • Alias: keyalias
    • Password: 123456

Import packages

  1. Run gettingAREmojiV3.3.unitypackage, then click Import.

  2. Check the imported sample app.

    a. Go to Assets > Scenes > GettingAREmojiSample Scene.

    b. Click Canvas.

  3. Import the next package, glTFImporterV3.3.unitypackage.

Integrate the glTF Importer to the sample app

  1. Go to the sample scene.

    a. Click Assets > Scenes > GettingEmojiSample.unity.

    b. Click Canvas.

  2. Copy GLTFComponent prefab into your app to use GLTF Component property.

    a. Click Assets > Prefabs.

    b. Drag GLTFComponent prefab to your scene.

Build and run

  1. Go to File > Build Settings > Build and Run.

  2. Launch the sample app in your device.

    a. Click getAvatarList button and check the AR Emoji list stored on the device.

    b. Select one of them.

    c. Click getAvatarThumbnailUri button.

    d. Confirm if the AR Emoji thumbnail is showing.

    e. Click getAvatarZipUri Button.

    f. Check if it shows the AR Emoji avatar.

Learn the basics of animation in Unity

Animation system overview

Unity has a rich and sophisticated animation system (sometimes referred to as Mecanim). While Mecanim is recommended for use in most situations, Unity has retained its legacy animation system, which existed before Unity 4. When working with an older content created before Unity 4, you may need to use the legacy animation system.

Animator Controller

To apply the animation to the model, you must have a basic knowledge of the animation controller. Developers can easily control the animation with it.
An Animator Controller allows you to arrange and maintain a set of animation clips and associated Animation Transitions for a character or object. In most cases it is normal to have multiple animations and switch between them when certain game conditions occur. For example, you could switch from a walk animation clip to a jump animation clip whenever the space bar is pressed. However, even if you only have a single animation clip, you still need to place it into an Animator Controller to use it on a GameObject.

The Animator Controller has references to the animation clips used within it, and manages the various animation clips and the transitions between them using a State Machine, which could be thought of as a flow-chart of animation clips and transitions, or a simple program written in a visual programming language within Unity.
Here, the model is a loaded runtime, and then an animation controller component is added to the model. Therefore, you will learn how to make an animation clip and apply a simple animation through a State Machine.

Animation State Machine

Unity’s Animation State Machines provide a way to see the overview all of the animation clips related to a particular character and allow various events in the game, such as user inputs, to trigger different animations.

State Machine Transitions exist to help you simplify complex State Machines. This allows you to have a higher level of abstraction over the state machine logic. Each view in the animator window has an Entry and Exit node. The animator window specifies the State Machine Transitions. The Entry node is used when transitioning into a state machine. It will be evaluated and will branch to the destination state according to the conditions set. In this way, the entry node can control which state the state machine begins in, by evaluating the state of your parameters when the state machine begins. Because the state machine always has a default state, there will always be a default transition branching from the entry node to the default state.

Humanoid Animations

Unity’s Animation system also has numerous special features for handling humanoid characters which give you the ability to retarget humanoid animation from any source to your own character model, as well as adjusting muscle definitions. These special features are enabled by Unity’s Avatar system, where humanoid characters are mapped to a common internal format. One of the most powerful features of Mecanim is retargeting of humanoid animations. This means that with relative ease, you can apply the same set of animations to various character models. Retargeting is only possible for humanoid models, where an Avatar has been configured because this gives a correspondence between the models’ bone structure.

Generate animation clips

Animation clips are one of the core elements to Unity’s animation system. Unity supports importing animation from external sources, and offers the ability to create animation clips from scratch within the editor using the Animation window.

Animation data collection

This Code Lab provides a tool that works with the model. Before then, you need animation data. It can be made directly with a professional tool like Maya and 3Ds Max or it can be obtained for free from Mixamo.

Make the character walk

  1. Get the animation data here from Mixamo.

    a. Select the Walking animation data.

    b. Click the Download button.

    c. The Download Settings appears and the animation data can be obtained by setting the following:

  2. Transfer the file to Unity. This allows you to set the following settings in the Inspector window.

    After changing the animation type to Humanoid, click the Apply button. It is automatically retargeted to the Unity Avatar system.

  3. The animation controller is basically set in the model.

    a. In Resources > Animations > Avatar > Controller, double click the Player.controller,

    then you can see the Animator window.

    b. The animation set by default is Gangnam Style. It’s recommended to change this to IK Idle.

    c. Next, apply the animation by dragging the walking animation to the button.

  4. Make the transition from Idle to Walk.

    a. Right-click the Idle node.

    b. Connect lines from Idle node to the Walk node.

  5. Select the connecting line for transition (Idle → Walk).

    a. Uncheck Has Exit Time.

    b. Set the parameter to Bool. Animation Parameters are variables that are defined within an Animator Controller that can be accessed and assigned values from scripts. This is how a script can control or affect the flow of the state machine.

    c. Add the condition of transition (Idle → Walk).

  6. Select the connecting line for transition (Walk → Idle).

    Repeat in the same way with previous steps. However, the condition of walk should be set to false.

  7. Look for the Button Behaviour (Script).

  8. In ButtonBehaviour.cs, write a script that modifies parameters based on user input. Add the following at the bottom of the existing code:

    public Animator anim;
public float speed = 1.0f;

// Update is called once per frame
void Update(){
     if (Input.GetKey(KeyCode.Space)){
         if (anim == null){
            if (GameObject.FindGameObjectWithTag("GLTFComponent")){
                Debug.Log("Find GLTFComponent");
                GameObject o = GameObject.FindGameObjectWithTag("GLTFComponent");
                gLTFComponent = o.GetComponent<UnityGLTF.GLTFComponent>();
            }
            anim = gLTFComponent.GetComponent<Animator>();
         }
     }

     if (Input.GetKey(KeyCode.RightArrow)){
         gLTFComponent.transform.rotation = Quaternion.Euler(0, 90, 0);
         gLTFComponent.transform.position += gLTFComponent.transform.forward * Time.deltaTime * speed;
         anim.SetBool("walk", true);
     }

     else if (Input.GetKey(KeyCode.LeftArrow)){
         gLTFComponent.transform.rotation = Quaternion.Euler(0, -90, 0);
         gLTFComponent.transform.position += gLTFComponent.transform.forward * Time.deltaTime * speed;
         anim.SetBool("walk", true);
     }

     else if (Input.GetKey(KeyCode.DownArrow)){
         gLTFComponent.transform.rotation = Quaternion.Euler(0, 180, 0);
         gLTFComponent.transform.position += gLTFComponent.transform.forward * Time.deltaTime * speed;
         anim.SetBool("walk", true);
     }

     else if (Input.GetKey(KeyCode.UpArrow)){
         gLTFComponent.transform.rotation = Quaternion.Euler(0, 0, 0);
         gLTFComponent.transform.position += gLTFComponent.transform.forward * Time.deltaTime * speed;
         anim.SetBool("walk", true);
     }

     else{
         anim.SetBool("walk", false);
     }
}

  9. Play and review the walking animation in the Unity Studio.

    a. Press the spacebar on your keyboard and now you can operate the animation you set.

    b. Try pressing the arrow keys of the keyboard, to make your AR Emoji character move using the Walking Animation.

You're done!

Congratulations! You have successfully achieved the goal of this Code Lab. Now, you can create an AR Emoji avatar and use it as a character in games or apps, all by yourself!

Learn more by going to Galaxy AR Emoji.