NoesisGUI

First steps with NoesisGUI and Unreal

unreal.png

This document will guide you through the process of setting up the NoesisGUI Unreal Engine plugin. Noesis entirely replaces Unreal UI offering a much more convenient and efficient approach. You will learn how to start editing the samples included in NoesisGUI plugin and how to create new games from scratch. NoesisGUI is deeply integrated with Unreal; XAMLs appear as native resources that directly use assets imported by Unreal like Textures, FontFaces and Sounds. It is strongly recommended that you read the rest of tutorials to familiarize yourself with the basic concepts of XAML and Noesis architecture. Some knowledge of Unreal Engine is also assumed, but this document attempts to be as thorough as possible.

With the NoesisGUI plugin for Unreal Engine you have all the power of the native NoesisGUI SDK at your disposal. There are a few particularities about the use that will be explained later, but the bottom line is that everything that can be done with the native SDK will work in Unreal Engine.

We also provide an easy way to allow users to leverage the power of NoesisGUI using Unreal Engine's Blueprints. A mechanism is provided that allows Bindings to Blueprint properties, without having to write a single line of C++ code.

With this approach NoesisGUI will be more accesible for everyone, but at the same time will allow more advanced users to use the native SDK directly, with the complete set of features available to them as in any other platform.

spacer.png

What's new in NoesisGUI 3.1

  • Support for custom Brushes using Unreal Materials. See our Materials sample.
  • Support for custom Effects using Unreal Materials. See our VideoEffect example.
  • Added new Blending Modes. See our VideoEffect example.
  • Native support of MediaSources for playing Video using MediaElement. See our VideoEffect example.
  • Sprites are now supported in DataContexts and accessible to ImageSource properties in XAML using data-binding. See our Inventory sample.
  • Added support for binding Map properties, and improved native DataContext integration with support for getter/setter properties and command properties.
  • All the improvements included in 3.1 Core are also available for Unreal.

Installation

You can download the latest version of the plugin directly from our website. The zip file contains an Unreal project with some samples and the plugin ready to be used.

UnrealTutorialImg1.png

NoesisGUI works with versions of Unreal Engine installed through the Epic Games Launcher as well as those built by yourself from the source code in GitHub. And it can be installed as either an Engine Plugin or a Game Plugin.

Engine Plugin

If you want to use it as an Engine plugin please extract the contents of Plugins/NoesisGUI folder from the downloaded file into [UE4Root]/Engine/Plugins/NoesisGUI. [UE4Root] is the directory where you installed the engine from the Epic Games Launcher, or the directory where you cloned the UnrealEngine GitHub repository.

Game Plugin

If you want to use it as a Game plugin, so other game projects don't use it, please extract the contents of Plugins/NoesisGUI folder into [ProjectRoot]/Plugins/NoesisGUI, being [ProjectRoot] the directory where you have your .uproject file.

Examples

Within the zip file we provide a project with a few working samples. You can find a level asset for each one in the 'Content' folder root. We also provide Microsoft Blend projects for each sample inside 'Assets' folder.

Along with the samples you will find a Theme folder (xaml and font sources inside Assets folder) which defines default styles for all UI controls and are a good start to create your own game styles.

A lot more examples can be found at our GitHub repository.

UnrealTutorialImg2.png

Configuring NoesisGUI

At this point the plugin should be ready to use, so we can proceed to enable it and configure some settings for your project.

Run the Unreal Engine editor. You may get a notification on the bottom right corner alerting you that new plugins have been installed. If you don't, open the Plugins dialog from the Edit menu, and select UI from the category list on the left. You can enable the plugin from the right hand panel.

UnrealTutorialImg3.png

You can also do this manually by modifying your Project's .uproject file to add the plugin dependency, like this:

"Plugins": [
   {
      "Name": "NoesisGUI",
      "Enabled": true
   }
]

Noesis Settings

With the plugin enabled, you can configure some NoesisGUI global settings for your project. You can find them by opening the NoesisGUI top menu and selecting Settings. These settings have sensible default values, so if you're unsure about them you can just leave them as they are. Modifying some of these settings will require you to restart the editor for the changes to take effect.

UnrealTutorialImg4.png UnrealTutorialImg5.png

Project Setup

The process to set up your project to work with NoesisGUI depends on whether it is a Blueprint-only project or a C++ project.

  • For a Blueprint-only project, enabling the plugin as described above is all you need to do.

  • For a C++ project, there are two additional steps. First, you must add the module dependencies to your project. To do that, simply add the following line to you module's ModuleRules derived class (in your module's .Build.cs file):

    PrivateDependencyModuleNames.AddRange(new string[] { "Noesis", "NoesisRuntime" });
    

    Next, you have to #include the necessary headers for your project. To simplify this we've created a single header file you can include, preferably in your game project's PCH file. Just add the following line:

    #include "NoesisRuntime.h"
    

Blend

Unity3DTutorialImg3_.png

Microsoft Blend for Visual Studio (formerly Microsoft Expression Blend) is an interactive WYSIWYG front-end for designing XAML-based interfaces. In this section we will explain how you can use it so changes performed in Blend appear immediately in Unreal Editor, even in Play mode. This workflow allows quick iterations while designing user interfaces.

Note

For getting started with Blend we recommend reading our Blend Native Tutorial

To start using Blend in your Unreal project follow these steps:

  1. Open Blend and create a WPF application. We recommend placing the Blend project in a folder outside Content directory, for example in <Project>/Assets/.
UnrealTutorialImg23.png
  1. Install our Noesis.GUI.Extension package by right clicking on the project name and selecting "Manage NuGet Packages...".
UnrealTutorialImg24.png
  1. Open App.xaml and add our theme (or your own theme dictionary if you already have one):
<Application
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  StartupUri="GUI/MainWindow.xaml">
  <Application.Resources>
    <ResourceDictionary Source="pack://application:,,,/Noesis.GUI.Extensions;component/Theme/NoesisTheme.DarkBlue.xaml" />
  </Application.Resources>
</Application>
  1. Create your XAML files following the same folder structure you are going to use in your Unreal Content folder.
UnrealTutorialImg25.png

Now you can configure your Unreal project to map the assets from the Blend project, as explained in next section.

Assets

At this point you're ready to start using NoesisGUI in your project. The first step is importing your assets. These will consist of XAML files, fonts and images. The easiest way to work with NoesisGUI assets in Unreal Engine is to put them in your game's Content folder. They will be automatically imported into Unreal Engine, and any modifications you make will be instantly re-imported. After import, your XAML files will appear in Unreal Engine's Content Browser as NoesisXaml assets, while your images will appear as Texture2Ds and fonts as FontFaces.

UnrealTutorialImg6.png

When importing your XAML files, these may contain references to other XAML files, as well as images and fonts, as stated above. These references may be in the form of relative or absolut URIs. When working with absolute URIs in Unreal Engine, these are considered as relative to the Content folder of your game project. Please refer to the document URIs in NoesisGUI for more information.

If you keep your XAML project separate from your game's assets, you can rely on Unreal Engine's Auto Reimport feature to map source asset directories to folders in the Content Browser.

UnrealTutorialImg7.png

Note

Our samples use this approach to keep Blend project and source files separated from Unreal assets in Content folder. This way changes made in XAML files inside Blend are automatically reflected in Unreal even in Play mode, improving your workflow.

User Controls

Before explaining how to create your own user controls in Unreal it is recommended to read first the UserControl tutorial documentation.

As you should know now, a UserControl is composed of a xaml that defines the UI contents of the control, and the class which defines the properties, events and logic of the control. When your user control is created it should load the associated xaml by calling LoadComponent with the path to the xaml asset inside the Content folder.

class NumericUpDown : public Noesis::UserControl
{
public:
   NumericUpDown()
   {
      Noesis::GUI::LoadComponent(this, "UserControls/NumericUpDown.xaml");
   }

   NS_IMPLEMENT_INLINE_REFLECTION_(NumericUpDown, "UserControls.NumericUpDown")
};

Next step is to register this class into the component factory, so it could be created when xaml parser finds it in the xaml. This should be done during the module startup.

class UserControlModule : public FDefaultGameModuleImpl
{
   virtual void StartupModule() override
   {
      Noesis::RegisterComponent<UserControls::NumericUpDown>();
   }

   virtual void ShutdownModule() override
   {
      Noesis::UnregisterComponent<UserControls::NumericUpDown>();
   }
};

IMPLEMENT_MODULE(UserControlModule, UserControlModule);

Then you can reference this user control in any other xaml using a prefix that maps the namespace specified in the reflection:

<Grid
   xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
   xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
   xmlns:local="clr-namespace:UserControls">
   <local:NumericUpDown />
</Grid>

Blueprints

There is another aspect where asset paths are important: when you reference a Blueprint type in your XAML.

<Grid
   xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
   xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
   xmlns:local="clr-namespace:TicTacToe">
   <Grid.Style>
      <Style TargetType="Grid">
         <Setter Property="Background" Value="Blue"/>
         <Style.Triggers>
            <DataTrigger Binding="{Binding State}" Value="{x:Static local:State.Player1}">
               <Setter Property="Background" Value="Red"/>
            </DataTrigger>
         </Style.Triggers>
      </Style>
   </Grid.Style>
</Grid>

This xaml references the type TicTacToe.State, which can be an enumeration defined in Blueprint instead of native code. For Blueprint types we use the path to the asset as its namespace, so State Blueprint class should be created as /Game/TicTacToe/State in Unreal in order to be found by Noesis when xaml gets parsed.

Materials

NoesisGUI supports the use of custom shaders in two scenarios, Effects which apply a post-processing shader to any element of the UI tree, and Brushes which define how elements can be filled or stroked. In Unreal you can directly use Materials for this purpose. As mentioned before for Blueprint types, the path to the Material asset will be used to expose the type to Noesis and how you should reference it in the XAML file.

In case you want to create an Effect you have to define the Material Domain as Post Process. Material parameters will automatically be exposed as properties that can be set or even animated from the XAML.

UnrealTutorialImg21.png
<Grid
   xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
   xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
   xmlns:local="clr-namespace:CustomEffects">
   <Grid>
      <Grid.Effect>
         <local:ChromaticAberrationEffect Amount="10">
      <Grid.Effect>
      <Image Source="Background.jpg"/>
      <TextBlock Text="Effects Sample" HorizontalAlignment="Center" VerticalAlignment="Center"/>
   </Grid>
</Grid>

In order to use a Material as a Brush you have to define the Domain as User Interface. Same as before, the material parameters can be modified in the XAML file.

UnrealTutorialImg22.png
<Grid
   xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
   xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
   xmlns:noesis="clr-namespace:NoesisGUIExtensions;assembly=Noesis.GUI.Extensions"
   xmlns:local="clr-namespace:CustomBrushes">
   <Rectangle Width="400" Height="200">
      <Rectangle.Fill>
         <ImageBrush ImageSource="Background.jpg">
            <noesis:Brush.Shader>
               <local:NoiseBrush Color="Gold"/>
            </noesis:Brush.Shader>
         </ImageBrush>
      </Rectangle.Fill>
   </Rectangle>
</Grid>

Sprites

NoesisGUI supports Sprite images exposed in Blueprint view models, which can be consumed by any ImageSource property in XAML using data-binding.

<Image Source="{Binding Item.Icon}"/>
UnrealTutorialImg26.png

Views

Now that you have your assets imported into the game, you have to create a View for it. The easiest way is by right-clicking on any xaml asset and select Add to Viewport. This will create and configure a View automatically, and add it to current Level map so you can rapidly see the UI rendered in the viewport when playing.

UnrealTutorialImg8.png

A NoesisView is also an Unreal Engine UMG Widget. This means that you can use it wherever a native UMG Widget is used, so if you're familiar with Unreal Engine's UMG system it will be very simple to get up and running with NoesisGUI. Please, refer to the document Creating Widgets for information about how to manually add a NoesisView to your viewport and how to handle input. Because a NoesisView is also an UMG Widget it means you can also create 3D UI elements by using a Widget Component. Please, refer to the document Widget Components for more information.

Double-clicking the View asset will open the familiar Blueprint Class editor. Press the Class Settings button on the top toolbar to edit the View properties.

UnrealTutorialImg9.png

The most important of all is the XAML property. Here you'll reference the NoesisXaml for which you want to create the View. Enable PPAA allows you to enable per-primitive antialiasing in case you're not using multisampling on your main render target. Tessellation Quality lets you select the degree of subdivision for your path geometry. Remember to compile and save your NoesisView before using it.

UnrealTutorialImg10.png

The following Level Blueprint will render the View in the screen.

UnrealTutorialImg11.png

Property Binding

One of the most powerful features of NoesisGUI is the support for a Model-View-ViewModel pattern through Data Binding. This is made even more powerful in Unreal Engine by combining it with the graphical scripting capabilities provided by Blueprints.

When you create a NoesisView for a NoesisXaml and you create a Widget from it, the instantiated object itself is set as the Data Context. You can also override this behaviour by calling the function Set Data Context on the NoesisView itself and specifying a different Unreal Engine Object.

Whether you decide to use the NoesisView or any other Object as the Data Context, you can define properties that can be bound to your XAML, as described in the aforementioned document. You can define a property in two ways:

  • As a variable. The name of the property for binding purposes will be the name of the variable. In C++, properties should be defined with the BlueprintReadWrite attribute to be visible by the binding system:

    UPROPERTY(BlueprintReadWrite)
    int32 NumBots;
    
UnrealTutorialImg12.png
  • As a Get function that has no input parameters and a single output parameter, with alternatively a Set function with no output parameters and a single input parameter of the same type as the output parameter of the Get function. The name of the property for binding purposes will be the name of the function without the Get/Set prefix. In C++, getter and setter functions should be defined with the BlueprintCallable attribute to be visible by the binding system:

    UFUNCTION(BlueprintCallable)
    void SetNumBots(int32 value);
    
    UFUNCTION(BlueprintCallable)
    int32 GetNumBots() const;
    
UnrealTutorialImg13.png

Here's a table with the types supported for Data Binding and the corresponding native NoesisGUI SDK types. Whenever there was a native Unreal Engine type that could work we've decided to use that instead of creating a new type.

Unreal Engine Type NoesisGUI Type
Boolean bool
Integer int32_t
Float float
String Noesis::String
Text Noesis::String
Color Noesis::Color
Vector2D Noesis::Point
Box2D Noesis::Rect
NoesisSize Noesis::Size
NoesisThickness Noesis::Thickness
NoesisCornerRadius Noesis::CornerRadius
Timespan Noesis::TimeSpan
NoesisDuration Noesis::Duration
NoesisKeyTime Noesis::KeyTime
Texture2D Noesis::ImageSource
TextureRenderTarget2D Noesis::ImageSource
PaperSprite Noesis::ImageSource

The plugin also supports custom Blueprint Enums, Structures and Classes. Blueprint Structures and Classes work similarly and they both expose their members as sub-properties. The difference is that Structures are treated as atomic objects, and the performance characteristics are different. We recommend you use Blueprint Classes for long lived objects, and Blueprint Structures for small objects with shorter lifespans that don't require a full Blueprint Class.

We also support Arrays of the aforementioned types, which are exposed as a List to the Binding system.

UPROPERTY(BlueprintReadWrite)
TArray<Color> Colors;

And Maps with string keys to the supported types, which are exposed as a Dictionary to the Binding system.

UPROPERTY(BlueprintReadWrite)
TMap<FString, Color> TeamColors;

Additionally we support binding of Blueprint functions to NoesisGUI Commands. A function that either takes no parameters, or takes a parameter of a compatible Unreal Engine type from the table above, and has no output will be turned into a functor implementing the Noesis::ICommand interface. If you also add a function with the same name but prefixed with CanExecute that takes either no parameters, or a parameter of the same type as the matching Command function, and returns a Boolean, it will be used to decide whether the command can be executed.

UFUNCTION(BlueprintCallable)
void JoinGame();

UFUNCTION(BlueprintCallable)
bool CanExecuteJoinGame();
UnrealTutorialImg14.png

Property change notifications

Performance is a key concern of NoesisGUI. For this reason, instead of polling all the data sources for all bound properties in every update, it uses a reactive model in which the user notifies NoesisGUI that a property has changed and the system updates the minimum possible set of its internal structures affected by that change.

In the NoesisGUI Native SDK this is achieved by having your classes implement the Noesis::INotifyPropertyChanged or Noesis::INotifyCollectionChanged interfaces. The typical implementation exposes Set functions that compare the new value with the old and notify any potential listeners if they are different.

To simplify this in Unreal Engine, we've provided custom nodes that you can use in your Blueprint code that replace the standard Set Property and array operations with custom versions that, additionally notify listeners if necessary. Here's a table with the native Unreal Engine nodes and the corresponding NoesisGUI ones:

Unreal Engine Node NoesisGUI Node
Set Set w/ NotifyChanged
Add (Array) Add w/ NotifyArrayChanged
Add Unique (Array) Add Unique w/ NotifyArrayChanged
Shuffle (Array) Shuffle w/ NotifyArrayChanged
Append Array (Array) Append Array w/ NotifyArrayChanged
Insert (Array) Insert w/ NotifyArrayChanged
Remove Index (Array) Remove Index w/ NotifyArrayChanged
Remove Item (Array) Remove Item w/ NotifyArrayChanged
Clear (Array) Clear w/ NotifyArrayChanged
Resize (Array) Resize w/ NotifyArrayChanged
Set Array Elem (Array) Set Array Elem w/ NotifyArrayChanged
UnrealTutorialImg15.png

Using this function is the most convenient way to notify NoesisGUI of changes in your data. But sometimes it is necessary to manually notify that a property has changed. This is the case, for example, when a property is implemented as a pair of Get/Set functions, that are not bound to a variable and therefore don't allow you to use these nodes. For these situations you can manually call the functions NotifyChanged and NotifyArrayChanged, passing the name of the property.

UnrealTutorialImg16.png

Localization

Noesis allows you to use Unreal native localization system based on FText and StringTables directly in XAML files. There are two approaches depending if you need static or dynamic localized texts.

Markup Extensions

When defining static texts in XAML that need to be localized you can use the following markup extensions:

  • LocTable: Equivalent to using LOCTABLE in C++ code, provides access to a localized text with the given Key in a StringTable. The table Id can be specified as an attached property in any parent container and it will be inherited down the tree, or you can set it locally in the markup extension itself. When using this markup extension we recommend to include the Source text so Microsoft Blend can show it in the designer view.

    <Grid
      xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
      xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
      xmlns:noesis="clr-namespace:NoesisGUIExtensions;assembly=Noesis.GUI.Extensions">
      <StackPanel noesis:LocTable.Id="GameTableId">
        <TextBlock Text="{noesis:LocTable 'Hello World!', Key=HelloWorld}"/>
        <TextBlock Text="{noesis:LocTable 'This uses a different table', Key=SampleText, Id=OtherTableId}"/>
      </StackPanel>
    </Grid>
    
  • LocText: Equivalent to using LOCTEXT in C++ code, generates a new FText in a namespace with the given Key. The Namespace can also be specified as an attached property in any parent container and it will be inherited down the tree, or you can set it locally in the markup extension itself. The generated text will be discoverable by Unreal when gathering localized texts in the project if you include XAML assets in the search.

    <Grid
      xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
      xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
      xmlns:noesis="clr-namespace:NoesisGUIExtensions;assembly=Noesis.GUI.Extensions">
      <StackPanel noesis:LocText.Namespace="GameNamespace">
        <TextBlock Text="{noesis:LocText 'Hello World!', Key=HelloWorld}"/>
        <TextBlock Text="{noesis:LocText 'This uses a different namespace', Key=SampleText, Namespace=OtherNamespace}"/>
      </StackPanel>
    </Grid>
    

Bindings

If you need to show localized text in XAML that is dynamic or is formatted from some parameters you can expose FText properties in your DataContext and Noesis will keep track of them to automatically notify when language is changed in Unreal.

UnrealTutorialImg20.png
<Grid
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
  <TextBlock Text="{Binding WorldsText}"/>
</Grid>

Performance stats

NoesisGUI main tasks are profiled using Unreal stats system.

The following command shows information about the time spent processing input events, updating the views, and generating the render commands. It also shows the total memory allocated through Noesis:

stat Noesis
UnrealTutorialImg17.png

Inside Unreal's GPU category you can find the time spent rendering Noesis UI:

stat GPU
UnrealTutorialImg18.png

Logging and debugging

If following the previous steps you still can't see your NoesisGUI interface, the first step would be to take a look at the Output Log window. You can select the verbosity of the output from the plugin settings, as described above. The default setting is Quiet, and won't output anything. Normal will allow you to discover common errors, and Binding also provides extra information about the Binding system which is separated form the rest as it can be quite detailed.

Additionally logging messages in NoesisGUI have several levels of verbosity. Here is a table with the levels defined by NoesisGUI and how they map to Unreal Engine:

NoesisGUI Verbosity Level Unreal Engine Verbosity
Trace VeryVerbose
Debug Verbose
Info Log
Warning Warning
Error Error

You can filter the NoesisGUI log messages by selecting the LogNoesis category from the Categories drop down list.

UnrealTutorialImg19.png
© 2017 Noesis Technologies