building a fast user interface
TRANSCRIPT
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
BUILDING A FAST USER INTERFACE• Tim Cooper• Rune Johansen
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
THINGS YOU WILL KNOW• Author high performance UI’s• Use the new Unity Frame debugger• Understand what ‘batching’ is and why you want it• The performance cost of a CanvasRenderer• How to write optimised custom UI components
COPYRIGHT 2014 @ UNITY TECHNOLOGIESCOPYRIGHT 2015 @ UNITY TECHNOLOGIES
WHO ARE WE?
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
TIM COOPER
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
RUNE JOHANSEN
COPYRIGHT 2014 @ UNITY TECHNOLOGIESCOPYRIGHT 2015 @ UNITY TECHNOLOGIES
BATCHING
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
WHAT IS BATCHING?• Decreases draw calls
• Combine input elements into one output element• Reduces the load on the render thread
No Batching
Batching
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
HOW DOES BATCHING GET CALCULATED?• Based on a number of internal rules
• Material changes• Element overlaps• Shared material properties
• You need to configure your project to take advantage of this
• You need to understand your content ;)
**BEFORE WE BEGIN**
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
UNITY 5 FRAME DEBUGGER• Step through individual draw calls• Easiest way to see what is getting batched
• Works in editor!
• How do I use it?
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
DEMO
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
BACK TO BATCHING• Batch calculations happen for each canvas
• This includes nested canvases!
• What are the rules?
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
RULE #1A• If elements use the same material and texture
• They can batch together
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
RULE #1B• If elements use the same material and texture and
overlap
• They can still batch!
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
RULE #2• Batches will be broken when a material / texture
change is required
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
RULE #2A• Simple case:
• Material / Texture change
• Elements can not batch• Separate draw calls are required
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
RULE #2B• Less simple case
• Overlapping elements
• Elements need to be rendered in order• If there are different materials / textures between
two non overlapping elements…• Batching will break
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
BATCH SORTING• Internal to the UI System
• Uses rect overlaps to calculate this.• Rearranges draw order
• Least material changes• Least texture changes
• Keeps output appearing the same
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
SPECIAL RULE (5.2)• Only elements that are coplanar with the Canvas will
sort for batching• If the UI is in perspective mode, or elements are
rotated things can render incorrectly.
Modified internal code to show issue
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
SPECIAL RULE (5.2)• If we ‘break’ the sorting
• The issue will be fixed
• Depending on content this may or may not cost more draw calls.
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
SPECIAL RULE (5.2)• What makes an element break coplanerness?
• Elements having a different z-value from the canvas• Elements rotated out of canvas alignment• Input mesh with vertices where the z value is not
aligned with the canvas
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
RULE SUMMARY• Changing materials breaks a batch• Changing textures breaks a batch• Elements offset from the canvas breaks a batch
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
APPLYING THE RULES• Minimise material changes in your UI
• Use materials for effects, or special things• Use a single material for most of the UI
• 5.2• Text and normal UI elements can use the same
material
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
APPLYING THE RULES• Minimise texture changes in your UI
• Use sprite atlases.
• Note - Text is not put in a sprite atlas• Can lead to increased draw calls.
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
APPLYING THE RULES• Keep elements in coplanar with the Canvas
• You can use nested canvas• It’s okay to move some elements off the canvas
• Buttons that pop out ect• Make it the exception
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
RENDERING
•Fast Batches•Slow Batches
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
SLOW BATCHES• Whenever a material is changed
• The shader must be changed• Slow process• Internal rendering state needs to be rebuilt• “SetPass” in the stats window
• Minimise material changes
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
FAST BATCHES• Whenever a texture is changed
• The material can stay the same• Internal rendering state stays the same
• The shader uniforms are updated• Just another draw call
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
IN PRACTICE• Example
COPYRIGHT 2014 @ UNITY TECHNOLOGIESCOPYRIGHT 2015 @ UNITY TECHNOLOGIES
CANVAS RENDERER
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
BUILDING BLOCKS• All renderable UI elements have a CanvasRenderer• The UI components configure a CanvasRenderer
• Texture• Mesh• Colour
• What happens when you change a CanvasRenderer?• What happens if you reparent or reorder CanvasRenders?
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
MODIFICATION• Causes Rebatching!
• Need to recalculate internal state• Could change batching / other elements• Not slow, but also not fast
• Do it as little as possible!
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
REORDERING• Rearrange hierarchy / Change sibling order
• We need to recalculate the depths of all elements• Manually iterate all CanvasRenders on a Canvas
• Then do a Rebatch
• Avoid this whenever you can!
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
PIXEL PERFECT• Snaps RectTransform edges to vertex boundaries• Happens whenever a Rect is moved
• Including children• Means that vertices need to be regenerated
• SLOW!
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
PIXEL PERFECT• For speed
• Disable pixel perfect on Canvases where elements move
• You can disable at the start of movement, reenable after
• Pixel perfect only works on Screen Space canvas!
COPYRIGHT 2014 @ UNITY TECHNOLOGIESCOPYRIGHT 2015 @ UNITY TECHNOLOGIES
OPTIMIZING CUSTOM UI COMPONENTS
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
CUSTOM UI ELEMENTSYou can make your own UI components• Custom graphics (controls Canvas Renderer)• Custom layout (controls Rect Transforms)
Optimization means:• Layout and graphics should only be rebuilt when
needed
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
public interface ICanvasElement{void Rebuild(CanvasUpdate executing);…
}
• Custom UI elements can be Canvas Elements• Implement ICanvasElement
CANVAS ELEMENT
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
• CanvasUpdate• Prelayout• Layout• PostLayout• PreRender• LatePreRender
public interface ICanvasElement{void Rebuild(CanvasUpdate executing);…
}
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
• PreLayout• Update data needed for layout calculations
• Layout• Used by auto-layout system
• PostLayout• Update data dependent on layout
• PreRender• Update vertices and materials
• LatePreRender• Update vertices dependent on generated text character data
REBUILD CALLBACKS
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
LayoutCanvasUpdateRegistry.RegisterCanvasElementForLayoutRebuild
• PreLayout• Layout• PostLayout
GraphicCanvasUpdateRegistry.RegisterCanvasElementForGraphicRebuild
• PreRender• LatePreRender
TRIGGERING REBUILDS
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
• Rebuilds are delayed, not immediate• Multiple things may trigger the same rebuild
• but it only happens once
TRIGGERING REBUILDS
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
Canvas.willRenderCanvases >delegate invoked (5.2+)
The CanvasUpdater is invoked as part of willRenderCanvases
Each rebuild can be triggered by multiple things in the same frame, but the callback only happens once.
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
• We want rebuilds to happen when needed• Otherwise we get buggy behavior
• We only want the rebuilds when needed• Otherwise performance suffers
WHAT SHOULD TRIGGER WHICH REBUILDS?
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
PROPERTY CHANGES IN YOUR COMPONENTIf a property change may require changes to vertices or materials (anything on CanvasRenderer)• RegisterCanvasElementForGraphicRebuild
Examples• Color change• Sprite change• Essentially anything that affects appearance
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
PROPERTY CHANGES IN YOUR COMPONENTIf a property change may require changes to position or size (anything on RectTransform)
• Change RectTransform immediately (simple things - slider)
• LayoutRebuilder.MarkLayoutForRebuild (auto-layout)
• CanvasUpdateRegistry.RegisterCanvasElementForLayoutRebuildRegisterCanvasElementForLayoutRebuild is only rarely used
Normally higher level LayoutRebuilder is used instead.
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
CHANGES TO RECTTRANSFORM OR HIERARCHYOutside changes may require a rebuild
• OnRectTransformDimensionsChange
• OnTransformParentChanged
• OnTransformChildrenChanged (for layout groups)
• OnCanvasGroupChanged
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
SHORTCUTSInherit from existing built-in classes that do much of the work
• Graphic• Selectable• LayoutGroup
COPYRIGHT 2014 @ UNITY TECHNOLOGIESCOPYRIGHT 2015 @ UNITY TECHNOLOGIES
OPTIMIZING CUSTOM LAYOUT COMPONENTS
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
AUTO-LAYOUTAuto-Layout system has• Layout Elements (Image, Text, LayoutElement)• Layout Controllers
• Layout Groups (HorizontalLayoutGroup)• Layout Self Controllers (ContentSizeFitter)
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
LAYOUT ELEMENTSLayout Elements provide layout input:
• Minimum size• Preferred size• Flexible size
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
LAYOUT ELEMENTSpublic interface ILayoutElement{
void CalculateLayoutInputHorizontal();void CalculateLayoutInputVertical();
float minWidth { get; }float preferredWidth { get; }float flexibleWidth { get; } float minHeight { get; }float preferredHeight { get; }float flexibleHeight { get; }
int layoutPriority { get; }}
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
LAYOUT CONTROLLERSLayout Controllers use the layout input and control
• RectTransform sizes• RectTransform positions
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
LAYOUT CONTROLLERSpublic interface ILayoutController{
void SetLayoutHorizontal();void SetLayoutVertical();
}
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
AUTO-LAYOUT CALLBACK ORDERLayout input data can depend on children• Calculate children first, parents afterwards!
Layout control of RectTransform can affect children• Calculate parents first, children afterwards!
This avoids calculations becoming obsolete because of child or parent changes in the same frame
• No need to re-calculate
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
COPYRIGHT 2015 @ UNITY TECHNOLOGIES
LAYOUT ELEMENT OPTIMIZATIONOptimize the layout calculations by storing the calculated layout input for when it’s used
• CalculateLayoutInputHorizontal• Calculate and store minimum, preferred, flexible
• minWidth, preferredWidth, flexibleWidth properties• Simply return stored result