MAK Legion Capabilities

Many traditional simulation engines are not able to take full advantage of today’s multi-core hardware. Such systems use a single simulation execution thread that executes one task at a time.

A more efficient approach is to process entity tasks in parallel, taking better advantage of multiple cores and allowing the application to represent many more entities for each frame.

A challenge with multi-threading is that when one thread may be writing the state of an entity, another thread may also be reading it. Using locking to prevent this problem creates bottlenecks in execution and reduces the overall performance of the execution – basically, reducing the performance gains that multi-threading was meant to achieve.

The solution is to implement a double-buffering scheme where simulations read from the current frame’s copy while writing to the next frame’s copy. Once the entities have been updated in the current frame, the next frame becomes the current frame. This ensures the framework is threadsafe.

Making the Data Store part of the framework, versus part of a federate’s implementation, allows any application using the framework to benefit from this multi-threading capability.

Legion provides a simple Object-Oriented API to the Data Store, allowing applications to interface with the framework while shielding the application from the details of double buffering. Large numbers of entities can be simulated without concern for locking or synchronization issues. Implementing the Legion API does not, however, impose a requirement that the application use multi-threading. Even applications that are not multi-threaded can use the framework. Single-threaded applications will just not get the full performance benefit of this part of the Legion architecture.

Legion’s Data-Oriented Design is a significant contributor to the overall efficiency of the Legion framework. Arranging the entity data into large, contiguous blocks of memory allows efficient use of the computer’s memory cache. Allowing the application to iterate over many entities without needing to move data in and out of the cache (“cache misses”) reduces the processing overhead, thus reducing the time needed to process each entity.

To summarize the Legion Data Store Library:

• The Data Store Library is a standard, optimized, multithread-safe way for applications to store and access state data, that also doesn’t require the applications to be multithreaded.
• The Data Store Library is not dependent on the Network Library. Because of this, the Legion Data Store Library can be used to improve the performance of high volume simulation engines that are non-networked or that use different distributed simulation standards, such as DIS and HLA.
• The built-in standard object model and the type-safe API are based on DIS-style entity types and the Real-time Platform Reference (RPR) Federation Object Model (FOM), and they support SISO enums, geocentric coordinates, dead-reckoning, etc.
• Code generation is used to extend the Data Store to support custom object models.

The focus here is not to reinvent the interoperability wheel, but to be more efficient in how we send the data we have always sent during distributed simulation exercises.

A significant contributor to Legion efficiency lies in the interest management scheme.

From the client simulation application’s perspective, the client expresses interest in entity and event data through the Network API. Then the Legion Network Library on the client receives large messages from the server that represent all remotely-simulated entities matching the client’s interest criteria. These “reflected entities” are placed directly into the client's Data Store, ready for use by the client simulation in the next frame.

The client’s simulation code uses the same abstract Data Store API to access data about the reflected objects that it uses to store and access the entities that it is simulating locally.

The Network API:

The server’s Data Store is coarsely organized spatially by simulation instance. The Network Library in the Entity Server takes care of the fine filtering against the interest criteria, then sends only the data required by the client using the same data representation as defined in the Data Store, thus no data marshalling is required.

To summarize the Legion Networking Library:

• Using TCP for the transport mechanism ensures simpler and more efficient code
• Bundling the state for thousands of entities into a single large packet makes more efficient use of the network
• Entity Server sends only the data needed by each client
• Networking Library offloads responsibility of sending to many clients to the Entity Server (vs the clients)
• This Networking Library and Entity Server framework is well suited for WAN and cloud implementations

With DIS, each application implements its own unique Data Store as well as its own unique networking code (though some use 3rd party applications for this like VR-Link). Really, the only “Framework” is the network itself, where the interface between the application and the framework is at the network protocol level.

In HLA, the Federate application implements its own Data Store, but the Framework provides a common networking implementation (in this case the Run-time Infrastructure). The interface between the Application and the HLA Framework is at the Network API level. This requires a lot of marshalling/copying to encode and decode each attribute from the internal Data Store for the federate into the FOM specific representation to be passed to the HLA API.

With Legion, there is a common, optimized Data Store Library with a common, optimized network implementation. The interface between applications and the framework is at the Data Store API and the Network API level. This allows the Network Library to be more efficient since it can reach directly into the Data Store implementation.

There are a number of key differences between HLA and the Legion Framework design which are summarized in the table below:

HLA has been designed to be stateless and is unaware of Object Model semantics. The HLA API requires federates to pass data, and not “pull” data from the federate’s Data Store. The HLA API was also built around passing attributes one at a time. The concept of a simulation frame, used by Legion for double buffering, does not exist in HLA.

Legion’s fundamental basis is on Data-Oriented Design with an approach that is geared more toward efficient data access and distribution to take advantage of multi-core hardware environments and efficient cache access. It has been optimized for scalability and ease of use by trading off some of the flexibility that is required by HLA. For example, there is no Time Management in Legion, which has greatly simplified its implementation. It also implements a simplified ownership transfer mechanism - Legion cannot transfer individual attributes but must transfer the entire entity. In Legion, a simplified interest management approach is based on knowledge of semantics (geography, etc.).

Legion is not intended to be a general-purpose replacement to HLA. It is presented as an alternative when very large entity counts are required or when the benefits of a client-server architecture are desirable.

Initial integrations of Legion with several simulation clients and game engines have already been developed, demonstrating Legion's ability to bring any application into million-entity simulation scenarios.

• MAK has demonstrated scenarios of up to 3.8 million moving entities for the US Army Synthetic Training Environment using MAK’s reference implementation which included:
  • 380 instances of VR-Forces CGF, simulating 10,000 entities each
  • State of the entire scenario managed by a single Legion Server
  • Sim Engines and Legion Server hosted on AWS cloud, with Legion Monitor, VR-Forces GUI and VBS4 clients connected from the STE facility in Orlando
• Initial integrations achieved with Legion include: VR-Forces, VR-Engage, VBS4, Unreal, Unity, OneSAF, and Exonaut.
• Legion has a well defined API with source code examples in several languages including: C++, C, Java, and C# APIs.
• The Built-In API covers the full breadth of DIS/RPR FOM semantics, supporting DIS type implementations out of the box.
• Development of a Legion/HLA/DIS Gateway and Legion Logger are nearly complete.

MAK has been a member of the simulation interoperability standards community since its founding nearly 30 years ago. We believe that open standards are best for the overall community. We are currently working with the Simulation Interoperability Standards Organization (SISO) to provide the Legion Interface Specification as the basis for a new Scalability/Interoperability Standard. As part of our participation in the 2021 Virtual Simulation Innovation Workshop, MAK presented the Legion Framework and hosted a kick-off of a Legion Study group to begin the process of Legion standardization.