HoloLens sharing platform

Collaboration is a big value proposition that comes with HoloLens. Not only can you enable truly immersive experiences and enable scenarios never-before possible but you can do this in a collaborative environment letting multiple people see and create at the same time.

One of the easiest way to enable this is to use an component of the open-source toolkit for HoloLens called HoloToolkit (you can find it on GitHub ). The sharing component in that framework is an extensible multi-platform, multi-architecture network communication engine that is nicely encapsulated and exposed to Unity to make sharing as easy as possible.

Before we dive in to the components of it I want to explain exactly how this works. Probably the most important thing to mention is that a separate instance of the app runs on all devices that share the experience. The actual functionality and 3D objects are not streamed over the network but rendered by each device and only the interactions are streamed.

The Holotoolkit sharing infrastructure is a network based (WiFi) infrastructure with a Message-Bus lie architecture. Looking at the illustration above we can see the components that make up the framework: Server, Client, Manager, Messages Hub

Server

The Server component acts as a relay between all devices and is responsible, among others to distribute the messages around between the connected devices in a timely and reliable manner.

Some of the server features worth mentioning are Lobby and Session management, Message Sync and reliability and Auto Discovery.

Important to note that all of the communication happens on UDP as is the case for most real-time communication frameworks but the server is responsible for ensuring delivery or retries if needed.

The server itself presents as a Win32 command line application that you can run on any machine in the local network. There are efforts to deploy it on Azure and people have had success with it but at the moment the easiest scenario is to be deployed in the local network.

Manager

The Manager component is an optional piece of the framework that one can use to manage a current server or session. It comes as a Win32 or UWP app and it acts as another client that connects to the server and allows you to get more information about the lobby, users or even messages being sent.

Client

The 2 manager apps can easily behave like clients if you need to drive a collaboration session from the tablet or PC but it also shows how one can enable collaboration not just for people within the experience wearing a HoloLens but from outside as well.

The best part though is the Unity scripts and prefabs that make it extremely easy to start implementing collaboration. Everything is encapsulated in a prefab called Sharing that you need to drag in your scene:

The SharingStage is the main and most important script in the prefab. The only parameter you should need to change there is the Server Address where you need to enter the IP address of your server. You can get that by running the server and it will show its own IP address that you need to enter here.

The Session Tracker script allows you to hook into events from the server when other people join or leave the session. Since the server supports multiple ‘rooms’ or ‘sessions’ the Auto Join Session joins the session with the given name on the server automatically so the user wearing the HoloLens does not have to execute any extra task when starting the experience.

The last script, Custom Messages we’ll discuss in more detail next…

Message hub

I’ve mentioned the Message bus architecture of the Sharing framework so at this point we will talk about the messages that are being passed on the network in more detail.

The Custom Messages script that comes with the Sharing prefab in Unity is a great starter point for developing a more complex message structure for your application.

The message itself is an array of bytes with a specific format to enable easy decoding om the receiving end. There are a couple of helper methods to help read/write some common structures such as a Vector3 or Quaternion which are then used to encode more complex structures such as position, scale, rotation, movement, etc.

Here’s the basic structure of a message:

• Sender (1B): each client connected to the server gets an unique Client ID that can be used to identify the senders of a message
• MessageType (1B): defining a message type is important so one can know what message it received and know how to decode it.
• Payload (xB): the actual message content.

Consider keeping the payload as simple as possible since you need to send them over network at high frequency and they need to be processed by another HoloLens at the other end.
Since all connected clients would run the same app, you don’t need to send any graphical information, just send commands that could be executed on the other end to yield the same results. For example, if you’re manipulating a model and adding new components to it, you don’t need to send the new component over the network, just send the type of component you added (cube) and it’s characteristics, transform, and colors or other properties so that the other device can replicate your command.

This is probably the most complex part of your task to enable sharing and collaboration in a HoloLens application.

Conclusion

So that’s pretty much all there is for a high level introduction to the Holotoolkit sharing platform.
It might look complicated, but in reality it’s really not. Most of the hard work has already been done and is available as part of the toolkit and all you have to do is determine the best ways you can enable it for your application and users.