Unity First steps with NoesisGUI and Unity

First contact with the integration of NoesisGUI within Unity is described in this tutorial. It is recommended that you read the rest of tutorials to familiarize yourself with the basic concepts of XAML and Noesis architecture. This document is structured as follows:

  • How to install or upgrade NoesisGUI in Unity
  • Configuring NoesisGUI
  • XAML assets
  • Rendering
  • Loading XAML by code
  • Hit Testing
  • Software Keyboard
  • Extra instructions for building on iOS, Android and UWP.



Although not strictly necessary we recommend removing the folder /Assets/NoesisGUI before installing a new version of NoesisGUI.

The first thing is installing the provided NoesisGUI-X.X.X.XXXX.unitypackage into your Unity project. You can do this by clicking on the Unity menu option:

Assets -> Import Package -> Custom Package...

and selecting the file NoesisGUI-X.X.X.XXXX.unitypackage that you have downloaded from our web page. A faster alternative is just dragging the unitypackage into the project window.


Once the installation finishes a welcome window should appear giving you information about the new installed version of NoesisGUI.


Due to a restriction in Unity regarding native plugins you may get a warning requesting a restart of Unity.


Configuring NoesisGUI

Clicking on Unity menu option Tools -> NoesisGUI -> Settings will open an inspector window that displays global options for NoesisGUI. If you want more detail about a specific option just hover the mouse over it to get a useful tooltip description.


XAML assets

When you drag and drop a XAML file into Unity Asset folder it will automatically create a NoesisXaml asset. If the asset already exists it will update it. As Unity does not currently support importing custom file extensions you will end up having two files inside your project, the .xaml and the .asset. The .xaml file is the one you edit externally and the .asset if the resource that Unity understands. If you don't need to edit the .xaml you can remove it from your project because it is not necessary for the runtime.


We recommend placing the Blend project (.csproj) next to your /Assets folder and edit .xamls directly from there. Each time you make changes, Unity will update corresponding resources.

NoesisXaml resources provide useful thumbnails and previews you can use to distinguish from the original .xaml in your project.



The MonoBehaviour in charge of rendering XAMLs is NoesisView. You need to attach that component to your GameObject. If you attach to a Camera object then the user interface will be rendered as a layer on top of the camera.

You can easily setup a user interface rendering in camera by doing drag and drop of a XAML asset into the Scene View. You can also drag and drop XAMLs into the Hierarchy Window.


To display and interacts with the user interface you must enter Play mode.


Sometimes you don't want the user interface to be directly rendered to the screen or camera. NoesisGUI supports rendering XAML to textures that are mapped to the mesh of an object. For that purpose you must create a Render Texture asset.



For masking, NoesisGUI needs an active stencil buffer. This can be activated in Unity by selecting a 24-bits Depth Buffer.

This texture must be used by one of the materials of the Mesh, a Plane in this example.


And finally, the NoesisView component must be attached to the object. Once this have been done, the XAML will be rendered onto the texture the next time you hit Play.


If you want to interact with the GUI rendered in the texture, your GameObject must have a MeshCollider component so texture coordinates can be obtained when doing the hit testing projection.

Loading XAML by code

Similar to standard Unity assets, if you want to load XAML by code you need to have a reference to the corresponding NoesisXaml asset in a MonoBehaviour component or just store it in a Resources/ folder and load it by using Resources.Load().

NoesisXaml xaml = (NoesisXaml)UnityEngine.Resources.Load(path, typeof(NoesisXaml));
Button button = (Button)xaml.Load();

Problem with this approach is that it is not compatible with WPF. In case you are interested in having code that is compatible with both Unity and WPF we provide a way to preload XAMLs by adding them to a list available in the Settings panel.


Preloaded XAMLs can be loaded using string paths, for example.

Button button = (Button)Noesis.GUI.LoadXaml("Assets/UI/Button.xaml");


Even if you move the .asset you must always use the original path that was used when the XAML was imported

Hit Testing

In some situations it is necessary to check if mouse is over or was clicked over any element of the UI. This can be done with the HitTest method available through the VisualTreeHelper class.

Here is an example where a Collider mouse event was received and we want to find out if we should discard it because any UI element could have been clicked before:

using UnityEngine;
using Noesis;

public class HitTestSample : MonoBehaviour
    Visual _root;

    void Start()
        var gui = GetComponent<NoesisView>();
        _root = (Visual)gui.Content;

    void OnMouseDown()
        UnityEngine.Vector3 mousePos = Input.mousePosition;
        Point point = new Point(mousePos.x, Screen.height - mousePos.y);
        HitTestResult hit = VisualTreeHelper.HitTest(this._root, point);
        if (hit.VisualHit == null)
            // No UI element was hit

            // ...your code here

Software Keyboard

When using a mobile/touch platform a software keyboard is used to generate the input keys that are sent to the application. NoesisGUI uses the TouchScreenKeyboard provided by Unity to open the software keyboard whenever a TextBox or PasswordBox gets the application focus. The default software keyboard manager is exposed by the class UnitySoftwareKeyboard:

public class UnitySoftwareKeyboard: SoftwareKeyboard
    /// <summary>
    /// Override if you want to open the software keyboard for a TextBox with your own options.
    /// </summary>
    protected virtual UnityEngine.TouchScreenKeyboard ShowOnTextBox(TextBox textBox);

    /// <summary>
    /// Override if you want to open the software keyboard for a PasswordBox with your own options.
    /// </summary>
    protected virtual UnityEngine.TouchScreenKeyboard ShowOnPasswordBox(PasswordBox passwordBox);

The default behavior can be overridden by inheriting from UnitySoftwareKeyboard:

public class AppKeyboardManager : Noesis.UnitySoftwareKeyboard
    protected override TouchScreenKeyboard ShowOnTextBox(Noesis.TextBox textBox)
        if (textBox.GetName() == "emailField")
            return TouchScreenKeyboard.Open(textBox.GetText(), TouchScreenKeyboardType.EmailAddress);
            return base.ShowOnTextBox(textBox);
public class AppLogic : MonoBehaviour
    // Use this for initialization
    void Start ()
        Noesis.GUI.SoftwareKeyboard = new AppKeyboardManager();

Building on iOS

For now, Bitcode is not supported. It must be disabled in the project settings.


Building on Android

The process of building for Android is very simple, only a few settings should be taken into account. The steps to follow are:

  1. Select File -> Build Settings...
  1. Select the Scenes to be deployed
  2. Select platform Android
  3. Adjust the desired Player Settings (Resolution, Icons, Splash Image...). Make sure that Minimum API Level is at least Android 2.3.3 'Gingerbread'. NoesisGUI is not compatible with previous Android versions.
  4. Click the Build button
  5. Select the location where you want to save the .apk file
  6. Install the .apk into your Android device and Run it

You can also generate an Eclipse project to extend your application. In that case the steps are the following:

  1. Select File -> Build Settings...
  2. Select the Scenes to be deployed
  3. Select platform Android
  4. Adjust the desired Player Settings (Resolution, Icons, Splash Image...). Make sure that Minimum API Level is at least Android 2.3.3 'Gingerbread'. NoesisGUI is not compatible with previous Android versions.
  5. Click the Export button
  6. Select the location where you want to store the Eclipse project
  7. Optimize deployed data

By default all the directory /Assets/StreamingAssets of the Unity project is deployed to the assets/ folder of the Eclipse project. If you want to optimize the data that is copied to the device it is recommended to leave only the ANDROID folder in ($ECLIPSE_PROJECT_DIR)/assets/NoesisGUI/Data/ folder.

  1. Build project with Eclipse to generate the .apk file
  2. Install the .apk into your Android device and Run it


If your application needs to use .OBB extension files you should probably have checked the option Split Application Binary. Then it is convinient to set Player Settings option Write Access to External (SDCard). We found several clients having problems running the application just after being installed, because Unity can't access the extension file and XAML resources won't be found.


Building on Universal Windows Platform

Only Universal Windows Platform for Windows 10 is supported.

  1. Make sure the SDK is set to "Universal 10".
  1. Press "Build" button.
  2. Open the generated Visual Studio project.
  3. Build the solution for the desired platform, ARM, x86 or x86_64.
  4. Deploy and Execute.
© 2017 Noesis Technologies