Foldable Device Features with Unity and Unreal Engine 4
Lochlann Henry Ramsay-Edwards
Galaxy GameDev Engineer
Intro
Foldable technology for mobile is a ground-breaking experience not only for users, but also for developers. The presence of many form factors, like immersive display, app continuity, Flex mode, and UX optimization, challenge developers to think outside of the box to adapt to this technology. There are already a large number of blogs talking about general application considerations, but what about gaming on a foldable device? In this blog, we look at the challenges and new opportunities foldable devices present for mobile game development. This blog focuses on Unity and Unreal Engine as they are the most common game engines for mobile development. However, a lot of the information applies to other engines as well.
App Continuity and Multi-Window Mode
Firstly, let's establish that the large internal display is called the Main Display whilst the smaller external display is called the Cover Display.
App Continuity and Multi-Window Mode are two key features of foldable smartphones. With App Continuity, you can seamlessly transition between the Cover Display to the Main Display without needing to reopen the app. With multi-window, you can run multiple apps at once and multitask like never before.
App Continuity
Moving an app between the two displays affects the size, density, and aspect ratio of the display it can use. App Continuity is enabled and disabled from the Android Manifest of the app: the <activity> element has a new attribute called resizeableActivity.
If not set, this attribute defaults to true, thus enabling App Continuity and assuming the app fully supports both App Continuity and Multi-Window Mode. If set to true then the application automatically attempts to resize when moving between displays, as shown in the video below.
If set to false then the system still allows you to transition from the Cover Display to the Main Display but does not attempt to resize the application. Instead, a new button appears, allowing the user to restart the application at a time of their choosing, as shown below.
When the app transitions between the two displays, the activity is destroyed and recreated. As such, the app data needs to be stored and then used to restore the previous state. From testing, both Unity and Unreal Engine appear to handle this process already. However, if you are developing on a custom engine, it is worth confirming that your engine/app can handle this.
Continue on Cover Screen
By default, foldable devices simply lock the device when closing the Main Display. However, it is possible for users to disable this functionality for certain apps.
In the device's Display Settings, there is an option called "Continue on Cover Screen" that has to be enabled by the user. Entering this menu displays a list of all applications on the device and enables users to set specific applications to not lock the device when closing the Main Display.
If an application has resizeableActivity set to false then the option to enable this functionality is greyed out and is not available. If your application supports resizing and App Continuity then you should make sure to test that this works as users may wish to enable this functionality on their device.
Multi-Window Mode
Multi-Window Mode can allow up to three apps to run at the same time in a split-screen arrangement (see left image) or for two apps to run in pop-up mode which keeps one app full-screen but creates another app in a smaller pop-up window (see right image).
Just like App Continuity, this is enabled using the attribute resizeableActivity in the Android Manifest. If resizeableActivity is not set then it defaults to true. So again, it is recommended that you set this yourself to ensure that your app works as intended.
App Continuity without Multi-Window Mode
If you would like your app to support App Continuity but don't want it to be used by the Multi-Window Mode, then this is possible.
First, you should set the resizableActivity attribute to false to disable both App Continuity and Multi-Window Mode. Next use the following <meta-data> element in your <activity> element:
<meta-data android:name="android.supports_size_changes" android:value="true" />
This re-enables the App Continuity feature without enabling Multi-Window Mode.
Game Engine Features to Use
If using existing game engines then there are several existing features that are useful if developing for foldable devices. This section provides a high-level look at these features. For more in-depth information, I recommend visiting the engine's documentation.
Unity
Unity is a very good engine for developing on foldable devices, requiring little to no setup to enable features for foldable devices.
The engine automatically handles resizing of the application without any modifications required. The resizeableActivity attribute is controlled by an option in the Player settings: Resizable Window. When enabled, resizeableActivity appears to be set to true; when disabled, resizeableActivity appears to be set to false. Note this option is not available in older versions of Unity. If this option is not present then you have to set the resizeableActivity manually.
The engine also provides a very robust UI scaling system that helps to reduce the amount of work required to create a UI that works across both the Cover Display and Main Display. The Canvas Scaler and Anchor system work very well with foldable devices and work to resize and position elements consistently across the two displays, thus preventing developers from having to create two UI designs (one for Cover Display and one for Main Display).
Unreal Engine 4
Unreal Engine 4 is another good engine for developing on foldable devices, although it requires a little more work to get it set up for foldable devices. First of all, Unreal Engine 4 by default disables the resize event on Android devices, However, it is possible to re-enable it in device profiles using the console variable:
r.EnableNativeResizeEvent = 1
Unreal Engine 4 by default also has a max aspect ratio of 2.1. This is, however, too small for the Cover Display of some foldable devices which can result in black bars on either side of the image. Fortunately, Unreal Engine makes this value easily changeable in the project settings:
Platform -> Android -> Max Aspect Ratio
From my testing, 2.8 is a good value. However, I recommend users experiment to find the best value.
Unreal Engine's UMG (Unreal Motion Graphics UI Designer) has a comprehensive set of tools for creating responsive UI designs. This system automatically scales UI elements to fit the screen size, and the anchor system is also very useful for ensuring the UI elements are positioned correctly between the two displays.
Samsung Remote Test Lab
No matter what engine you use, the best way to ensure your app works well on a foldable device is to test it on a foldable device. Samsung Remote Test Lab has a range of foldable devices available for developers to test their applications on if getting hold of a physical device is too difficult or expensive. All you need to do is create a Samsung account and you can start using these devices for testing.
Android Jetpack WindowManager
Despite being very good engines for foldable game development, neither Unity nor Unreal currently provide information about the current state of the foldable device (that is, is it open/closed/halfway?, where is the fold positioned?, what is the fold's orientation? and so forth). However, Android has recently created a new library as a part of their Android Jetpack Libraries that enables developers to access this information and make use of it in their apps.
Android Jetpack in their own words is "a suite of libraries to help developers follow best practices, reduce boilerplate code, and write code that works consistently across Android versions and devices so that developers can focus on the code they care about."
WindowManager is a new library from the Android Jetpack suite, intended to help application developers support new device form factors and multi-window environments. The library had its 1.0.0 release in January 2022, and targeted foldable devices but—according to the documentation—future versions will extend to more display types and window features.
More Technical Resources
This blogpost has an accompanying technical blogpost and Code Lab tutorial, demonstrating how to use the Android Jetpack WindowManager with both Unity and Unreal to take advantage of the Flex Mode feature of Samsung's foldable devices.
I recommend starting with the Jetpack WindowManager blogpost to learn how to set up the WindowManager in Java:
Then, follow it up with the Code Labs to learn how to make use of the WindowManager to implement a simple flex mode setup in either Unreal or Unity:
-
https://developer.samsung.com/codelab/gamedev/flex-mode-unreal.html
-
https://developer.samsung.com/codelab/gamedev/flex-mode-unity.html
Also, visit our game developer community to join the discussion around mobile gaming and foldable devices.
Click here to find out more about other design considerations when designing apps for foldable devices and large screens.
Final Thoughts
Foldable devices provide a richer experience than regular phones. Hopefully, this blogpost and accompanying tutorial have provided you with the necessary information to begin taking advantage of these new form factors and start providing users a richer foldable experience.
Additional resources on the Samsung Developers site
The Samsung Developers site has many resources for developers looking to build for and integrate with Samsung devices and services. Stay in touch with the latest news by creating a free account and subscribing to our monthly newsletter. Visit the Galaxy Store Games page for information on bringing your game to Galaxy Store and visit the Marketing Resources page for information on promoting and distributing your Android apps. Finally, our Developer Forum is an excellent way to stay up-to-date on all things related to the Galaxy ecosystem.
