The Adventure Pals - Porting to console via Unity - Part 1
 
tap-article.png
 

The Adventure Pals was released almost 10 months ago now - in April 2018. In this series of technical articles Matt will discuss how the existing game written in Haxe/OpenFL, was ported to Playstation 4, Xbox One and Switch using the somewhat unusual approach of going via Unity. Matt will cover the general approach and dive into some of the technical detail and interesting challenges that popped up along the way including the bizarre “giant tree mode”!

While the game took Massive Monster several years to develop from start to finish, the Unity and console conversion work started in January 2017 where it ran alongside the final game development and was finished in April 2018 when the game launched.

There were over 192 commits spanning 150+ man days of effort over this period and representing a significant contribution to the project - not simply a “porting” task. All releases of the game use our Unity conversion and optimisation with further work applied to target Microsoft Xbox One, Nintendo Switch and Sony PS4 and to pass certification.

So let’s get started at the beginning…

Sometime in mid 2016, Jay from Massive Monster got back in contact with me to revive a discussion we’d had a year or so before about using Unity to bring an existing title to console. The concept centered around the fact that Haxe could be cross compiled into C# and I’d had some experience cross compiling games many years back during the dawn of the mobile games industry.

Haxe, OpenFL and Lime

Haxe refers to itself as “The Cross platform Toolkit”. This includes - the Haxe programming language, cross compiler, standard library and tools to allow developers to write applications or games in a nice, modern language which is then cross-compiled to each platform’s language de-riguer; allowing excellent performance with no overhead from a virtual machine.

But Haxe by itself doesn’t include everything needed to write a game, you also need rendering, I/O etc. This is where OpenFL and Lime come in.

 
Screen Shot 2018-11-10 at 19.56.40.png
 

OpenFL is a framework for Haxe which mirrors the Adobe Flash API. Therefore Haxe/OpenFL is a very popular combination for developers migrating away from Flash, as Massive Monster were with The Adventure Pals. It’s also a great cross platform framework for building new games. One of the most notable titles in the recent times being Papers Please by Lukas Pope.

OpenFL actually uses a library called Lime to abstract the platform specific functionality. Rendering, timers, I/O etc.

Given this stack was designed to be portable in the first place, we started off attempting to port this to Unity as a target. After a few weeks it became clear this was a pretty big task. Most games tend to use a very specific subset of any platform and to get even basic things working we were having to port large chunks of related but unused functionality just to get simple things running. Eventually that approach would probably reach a critical mass - but we didn’t have that long. So a different approach was needed.

The Right Abstractions

Many years previous to all this. I had worked at leading UK independent mobile games studio IOMO. As well as writing all our racing games (a topic for another time) I ended up leading the development of the porting framework we used to allow us to write high quality titles across the then diverse range of Java J2ME handsets.

At the time we were actually starting to expanding beyond J2ME devices to platforms requiring C++ or C# and one of our sister companies supplied a Java->C cross compiler. There were no mature cross-platform rendering layers suitable for the performance constraints of mobile at the time. Typically therefore, there was significant difference in how each game rendered for ultimate performance.

To accommodate this, the “platform” abstraction I designed was actually split into 2 levels:

Core functions shared by all games which we called “The Framework” e.g. app life-cycle, resource handling, localisation.

Game specific functions, usually dominated by rendering but covering some other areas.

The game logic was then cross compiled from Java to C++ using the platform specific underlying layers.

The Adventure Pals was no different to our Java games of the time in that the majority of the game was implemented on top of a core set of game specific functions, which if converted would allow us to port the game with a fraction of the effort.

Given we’d demonstrated the cross compiler of Haxe, all we had to do was resurrect this approach and port the game specific rendering functions to Unity and we’d hopefully start to see some output!

 
diag.png
 

Proof of Concept

The initial rough proof of concept was created as:

class Game {
  private var sprite:Sprite;
  private var movie:Movie;
  private var speed:Int;
        
  public function new() {
    speed = 5;
    sprite = new Sprite();
    movie = new Movie();
  }

  public function Update():Void {
    sprite.x += speed;
    if(sprite.x > 160.0 || sprite.x < -160.0) {
      speed = 0-speed;
    }
    sprite.Update();
  }
}

The extremely basic Haxe app above, bounced a single sprite around the screen with a movie as a background.

The Haxe Cross Compiler then did most of the hard work with the main “game logic” file being output in C# very easily.

In a Unity scene a C# script re-implemented the sprite plotting function, this time actually updating a simple 2D Sprite in the Scene. A further C# script served to bootstrap everything and call the apps Update() function each frame.

 
Original screenshot of the proof of concept running in Unity, notice that the sprites are actually present in the retained scene graph, something we’ll discuss further in a later article.

Original screenshot of the proof of concept running in Unity, notice that the sprites are actually present in the retained scene graph, something we’ll discuss further in a later article.

 

This was all very straightforward, the cross-compiled Haxe could access functions defined directly in Unity C# by using an extern class defined with the @native annotation.

You could either create these classes to expose existing UnityEngine methods to Haxe or to expose custom written C# and we ended up using a mix of both for various reasons. The example above shows a Haxe interface to the Unity Spine implementation. Notice that Haxe defaults to double-precision floats so the Single type is equivalent to C# Float.

@:native("UnitySpine") extern class UnitySpine extends Object {
    function new(dataAssetName:String, x:Single, y:Single, parent:GameObject):Void;
    function SetAnimation(mix:Int, anim:String, loop:Bool ):Void;
    function AddAnimation (mix:Int, anim:String, loop:Bool, delay:Int):Void;
    function setBonePosition(_name:String, x:Single, y:Single):Void;
    function setPosition(x:Single, y:Single, sx:Single, sy:Single):Void;
    function setLayerOrder(layerOrder:Int, z:Single):Void;
    ...
    ...
}

The First Run

The next stage was to get the entire set of source code for the game to compile and “run” in Unity. This was a laborious task, implementing stubs for all the elements which would need to be ported to Unity and slowly fixing bugs until the code would compile completely. A few debug trace calls were added to the main Haxe game loop to allow a measure of “success”:

Main.hx: The Adventure Pals - Unity Game - Version 0.0.1
Title.hx: start() called

All of the initial port to Unity was done on an Mac OS X machine. The Haxe code itself did require some modifications to make it more suitable to port. An editor with good refactoring support is a must here. At the time the only real option for OS X was an old version of IntelliJ IDEA and an outdated plugin. However now it seems that the Haxe plugin has good support for current versions.

Esoteric Spine

The final thing to fall into place was that The Adventure Pals was written using Esoteric’s Spine library for the majority of the animations. As luck would have it there was a Unity port with an almost identical API. Within a day spent at the Massive Monster offices we managed to go from the first run to an almost completely rendered and animated title screen. Here’s the very first video that we took of this to show the other members of Massive Monster situated around the globe:

From then on time was spent replacing the stubs with real implementations in Unity. Getting all of the various elements to line up correctly on screen and within each frame was a time consuming process given some were rendered in fundamentally differently ways. However soon, we had broken the back of it.

Next time we’ll start to talk about some of the challenges getting to a fully playable state while allowing Massive Monster to finish development in Haxe, but output in Unity and utilise effects and items from the Asset Store...

-MT

Console Porting: Tips for Unreal Engine Development

Continuing our work with publisher Digerati Games, the 1st February 2019 will see the release of Animal Super Squad for Xbox One. Originally developed by DoubleMoose Games using the Unreal Engine and ported by us to the Microsoft console.

In this new tech post and video, Steve Longhurst offers up 4 good practice design and coding elements that will help any console port in the long run. They are things to inform your design and code from the very start of development, but will make the job of console porting go smoothly when the time comes.

The 4 things to bear in mind are:

  1. Initial Interaction Screen (IIS) or User Engagement

  2. Gamepad Menu Navigation

  3. Player and PlayerController Usage

  4. Game Pause and Other External Interruptions

Watch the full video (just over 10 minutes) which includes blueprint and code examples. Some additional information and links also follow below. In game clips for illustration from Animal Super Squad by DoubleMoose Games.

Initial Interaction Screen (IIS) or User Engagement

This is the classic “Press any key to start” screen that appears at the beginning of a game.

Gamepad Menu Navigation

I have followed the design of a simple game front end, from the Unreal Engine UMG UI Designer Quick Start Guide. This guide is a good place to start when you are looking at fleshing out a framework for a game, it provides the structure for a front end menu interface and a pause menu. It builds upon the blueprint 1st person template, but can easily be used with the 3rd person template (as I have done), or your own prototype game code.

Part 1, offers advice on implementing keyboard navigation for your UI, which can be extended to include gamepad. The main takeaway is that you cannot just focus on using a mouse for navigation, console’s don’t usually feature these! Some additional help can also be found at the following links:

Player and PlayerController Usage

Where the first two items talked about would generally apply to any engine, this item is very specific to Unreal. The Player and PlayerController objects are used within C++ and Blueprint code, that I describe a way to manage access to them which can be extended when it comes to console porting.

Game Pause and Other External Interruptions

Console development actually has something in common with mobile development in this regard. Modern consoles dictate the game life-cycle from startup to shutdown, and you need to write some methods that can be called when external events occur. The tips presented here will help you add these in early, and be able to test them when developing on a PC.

— SL

Steve Longhurst
Eastshade Announcement

It's a new year, and that means new projects! We are extremely proud to announce that we'll be bringing Eastshade to consoles.

Eastshade is a visually arresting first-person, open world exploration game where you are a traveling painter, exploring the island and capturing the world on canvas using your artist’s easel. Talk to the inhabitants to learn about their lives. Make friends and help those in need. Discover mysteries and uncover secrets about the land. Surmount natural impasses to reach forgotten places. Experience how your actions impact the world around you.

We are really looking forward to ensuring that Eastshade looks and plays wonderfully on console. It will certainly be a challenging project given the stunning visuals but at DO we pride ourselves on realising all titles in their best light.

One of the most anticipated games of 2019. From the minute I played the first Eastshade demo I was captivated. Beautiful visuals with a truly engaging story reminiscent of the classic adventure games of yore.
— Matthew Tighe - DO Games, Founder and CTO

The PC version launches on February 13th 2019, check back soon for more details on the console releases.

In the meantime take in the breathtaking visuals in the trailer below.

 
 
Matthew Tighe
Party Panic Announcement

We’re happy to announce our next project working with Everglow Interactive to release the crazy multiplayer mini game romp - Party Panic onto Xbox One and PS4 next year.

Party Panic is an online or local multiplayer party game with over 30 rapid fire mini games, multiple games modes - and even a built in drinking game!

We’ve been on the lookout for a party game to work on for the last year so Party Panic really fits the bill. With the combination of game modes, the selection of mini-games and not to mention online multiplayer; it’s a challenging title where DO games can bring our experience and attention to detail to the project to stand out from the crowd.

Looking for a party game, they don’t come much better than Party Panic. So we are really excited to work with Dylan Meville and bring our quality of service to another game genre.
— Matthew Tighe - DO Games, Founder and CTO

Stay tuned for more news on Party Panic, some more exciting announcements and a few new tech blogs as we lead up to Christmas…

For now checkout the trailer below.

 
 
Matthew Tighe
Xeno Crisis Announcement!

As mentioned in their latest KickStarter update, we are pleased to announce that we’ll be handling the console porting for Bitmap Bureau’s retro arena shooter “Xeno Crisis”.

We’ve known Mike and Matt for many years, working together previously at the UK’s number one independent mobile games studio IOMO just a few short years ago :-).

While the Bitmap Bureau guys focus on the tweaking the final gameplay and creating the amazing selection of retro versions (Megadrive, Neo Geo AND Dreamcast!) we’ll be putting the contemporary console builds through certification - Nintendo Switch, Sony PlayStation 4 and Microsoft XboxOne.

The modern console versions are in safe hands with DO Games, and we have every confidence that Matt Tighe and his team will do a superb job.
— Matt Cope - Bitmap Bureau, Technical Director

Xeno Crisis is best described as Smash TV meets Aliens with amazing pixel artwork, perfectly tuned gameplay and a pumping YM2612 inspired sound track.’

It’s great to be working with Bitmap Bureau team again. We love their attention to detail and the sheer platform reach and retro appeal of this project!
— Matthew Tighe - DO Games, Founder & CTO

We’d highly recommend you checkout the KickStarter and pre-order for your platform of choice!

https://www.kickstarter.com/projects/1676714319/xeno-crisis-a-new-game-for-the-sega-genesis-mega-d







Matthew Tighe
Compiling Unreal Engine 4.20

In our next tech post, Steve Longhurst will show you how to compile your own version of Unreal. Not only are we at DO-Games experienced with Unity, but also Epic Games’ Unreal Engine. This video will show you how to compile your own version of the Unreal Engine, an essential starting point for console development and porting.

If you are using the Epic Games Unreal Engine v4 to make a game, and you plan to port to a console such as Xbox One, Switch or PS4, you will need to compile your own custom version of the engine, from source. Epic Games only supply the console support plugins as source code once you have signed up with the console manufacturers development program and signed their Non Disclosure Agreements.

Building the engine yourself might sound daunting, but it's really not hard, and Epic Games provides several useful documentation pages on how to go about it. Good places to start are:

Compiling your own Engine is also a very good way to dig into the inner workings of Unreal, using the Visual Studio debugger. If you write C++ code (not just Blueprint logic), you can debug step through your code and right into the Engine source, allowing you to learn what’s really going on under the covers. Lastly, it also means you can fix bugs or add missing features in the Unreal Engine, which we think is one of the big selling points of Unreal for anyone with coding experience.

This video is my experience in compiling the Unreal Engine, on Windows 10, using Microsoft Visual Studio 2017. Reading the documentation will give you all the theory, but it often helps to see someone actually perform the steps. Just ten minutes of watching this video and you will have first hand experience for when you come to do it yourself.

- SL

Shikhondo Announcement, partnership with Digerati

Today we are happy to announce our next release, "Shikhondo - Soul Eater" for Nintendo Switch, developed by Deer Farm and published by Digerati Distribution!

This marks the first title in our new multi-game deal with Digerati.

Shikhondo is a Korean bullet-hell shoot 'em up set within the beautiful and bizarre world of Asian mythology.  DO handled the Nintendo Switch version including the setup of Nintendo Switch Online Leaderboards.  Getting the game running smoothly with thousands of bullets on screen required bespoke optimisaton work, seeing DO completely re-implement the bullet handling system. The result - solid 60fps whether handheld or docked.

This is exactly the kind of challenge we love; where we can use our experience to ensure a title not only gets released, but with the high quality that consumers demand and that does the title justice.  At DO we love games and bringing them to console is a labour of love, not just a process.

DO founder and CTO Matthew Tighe said of the deal:

"The quality of Digerati's catalogue is obvious and shows they know how to support developers and make a success of their titles.  Shikhondo has a great aesthetic and posed an interesting challenge for us, so was a perfect fit.  I'm looking forward to more of the same working with Nick and the team who have been great.”.

Shikhondo will be released later this summer for digital download and with a limited edition physical run, so it's our first boxed game.

 

Stay tuned for details of the second game, fruity pie!

Source control with Git Large File Storage

This is the first of hopefully many technical posts, we'll be talking about Unity, game dev and hopefully some retro game dev very soon.  So, to kick things off - let's start at the beginning with some source control...

Recently we have started a few projects which were complete PC games with a large footprint.  The first thing we do is put the project under source control and we've always used GitHub and GitLFS which have worked well.  However with large projects it is can be fairly common to see the dreaded:

"fatal: The remote end hung up unexpectedly"

I've seen various posts on the net about using SSH over HTTPS and setting the HTTP post buffer parameter to a large value:

git config http.postBuffer 524288000

However for me none of these work reliably in all cases.  As you know GitHub has a file size limit of 100MB, but you often encounter problems well before you get to files of that size.

So here are some basic things we do when first pushing a project to GitHub.

Start with a small initial commit

Just setup the README, .gitignore, and your lfs tracking (see below).The reason for this is simple, if you end up needing to rebase in future and everything is in the initial commit, you will soon discover you are stuck.  To rebase you need a parent and so you can't rebase the original commit.

At this point, don't forget to install GitLFS.

git lfs install

Split up large commits and push one at a time

Even if it does work a single giant commit is fairly unwieldy. There is nothing more frustrating that firing off a 'git push' only to have it fail 3% from the end after 20 minutes.  From my experience it's fairly intolerant of transferring large files, so if you missed off a 'git lfs track' or the connection hangs it's much easier if you don't have to start again from scratch and when there is a problem this helps you narrow it down.

So add files to commits in groups, either those that occur naturally or try alphabetically.  You don't need to go too mad on the granularity but a few reasonable sized chunks is all that is needed.

Find all the large files in your project and lfs track them first!

It's quite important to track all the large files in your project to begin with.  If you don't and you commit any large files by accident they'll get added to your git object database and bloat it.  Secondly if you miss large files your pushes will fail either when they get rejected or often when the transfer times out.

While You can rebase and track the files at a later date, that is painful and you end up having to clean out the blobs manually usually anyway.  Much better to get things under control from the start.

Using find and ls in bash, you can list all assets above a certain size: 

find .-type f -size +50M -exec ls {} \;

I'm sure with some sed and grep wizardry you can just list the extensions to track, but this is a good enough start.

You'll often notice examples or demos from Asset Store items in here if using Unity.  Might be time to prune the project unless you really need them eating up your storage space.

GitHub allows files up to 100MB but gives a warning over 50MB.  I usually look at anything over 10 or 20MB.  As well as any obvious binary only assets.

If you do make mistakes and attempt to clean up afterwards I have heard good things about the BFG Repo Cleaner. But have never used it myself.

Until next time...

Hopefully this will help you out when commiting an existing project to any Git based system with file size limits requiring you to use LFS.  As mentioned there will be more technical posts coming up soon including a post-mortem of sorts for The Adventure Pals, and some retro IBM EGA appreciation... so stay tuned.

- MT

 

The First Tree Announcement

As mentioned on Twitter a few months back we are happy to announce our partnership with David Wehle to release The First Tree on Xbox One, PlayStation 4 and Nintendo Switch!

The First Tree is a beautiful, 3rd-person exploration game centered around two parallel stories: a fox trying to find her missing family, and a young couple dealing with a tragedy in their own. Through out the game you uncover artifacts from the young couple’s life as they too become intertwined in the fox’s journey towards The First Tree.

At DO, we are really excited to be working on such a stunning game with a storyline that is close to our heart.  This further cements our culture of working with talented and creative Indie developers to deliver their titles to console with the best experience on each platform.

Matthew Tighe
Invisigun Heroes Announcement!

We are excited to announce that DO Games has partnered with Shadi Muklashy and Sombr Studio to work on the title Invisigun Heroes!

Invisigun Heroes is a single-screen stealth battle arena with a twist... everyone is invisible!  Play local or network multiplayer across a variety of worlds with your chosen character.

At DO we pride ourselves on working with indie developers producing high quality, original titles with heart and soul; Invisigun Heroes exemplifies this.

From the variety of the arenas, the unique set of characters each with their own abilities, to the beautiful pixel art and pumping sound track - you can see why the title has seen a successful Kickstarter and Steam launch.

Stay tuned for more updates on this and some more future titles very shortly!

gamesMatthew Tighe
Pinstripe Release for Xbox One and PS4

We are proud to announce that Pinstripe from Atmos Games will be our first successful port to be released, in only a few weeks time!

DO took the original source code, creating the console ports and delivering them on time and within budget.  We worked closely with the publisher, Armor Games Studios to deliver this as their first console title.

Pinstripe will be available on both the Xbox One and PS4 as follows, check out the links below for more information and pre-orders.

Xbox One, February 7th

https://www.microsoft.com/en-us/store/p/pinstripe/bp8wvvcmf11b

PS4, February 13th

https://www.playstation.com/en-us/games/pinstripe-ps4/

 

More successful ports on the way soon, to even more platforms... if you have a game and you'd like help moving to console, mobile or any other platform we'd love to hear more.

Playstation Experience 2017

We've been at PSX 2017 this weekend in Anaheim, California to support the first 2 titles to be ported by DO to all major consoles; The Adventure Pals and Pinstripe.  Both published by Armor Games Studios.

Anaheim convention centre for Playstation PSX
PlayStation PSX expo

The Adventure Pals by Massive Monster, to be released spring 2018 is a quirky side-scrolling platformer with local co-op play, giraffes and a hot-dog obsessed villian.

This title uses our Haxe/OpenFL to Unity re-targeting technology - which provides a great way to super-charge a game you already have in development.  Massive Monster were able to take the existing code, run it in Unity with no changes and continue to update it in Haxe, while DO focused on the porting to Playstation 4, Xbox One and Nintendo Switch via Unity.

Our system keeps all graphic elements as true GameObjects in the scene graph.  Because of this, Jay and the team at Massive Monster were able to add in selection of great Unity based shaders and effects to create a really polished visual style.

There will be a tech post on this blog soon going in depth on how this all works, what the challenges were and how they were addressed.

Pinstripe by Atmosgames, to be released Q1 2018 is a critically acclaimed, atmospheric adventure game with a breathtaking audio/visual style.  This was already a Unity title released on Steam when we got involved.  We took the existing game code and handled the porting to Playstation 4 and Xbox One.

The Adventure Pals ported to PlayStation 4 by DO Games
Pinstripe ported to PS4 by DO Games

Both games are published by Armor Games Studios which is the new, multi-platform arm of the well established Flash games site we all know and love.

Huge thanks to Jay from Massive Monster who was crazy enough to believe Matt when we said an existing OpenFL game could be ported to console via Unity with no code changes.  Without this and Jay's subsequent introduction to Thomas and Armor, none of this would have been possible.

Secondly, to Thomas Brush for trusting us to handle a game he's poured 5 years into and for being a generally great person to work with.

Finally to the Ryan, Justin, Sean, Dan and the whole Armor Games team for inviting Matt out to PSX and for all their support though the process.

It's been a blast and we are looking forward to seeing both these titles in store and more to come.  If you are and indie dev with a title you'd like our help on please get in touch!

events, gamesMatthew Tighe