We hear from game and simulation developers that they want more choices that allow for collaboration, customization, and creative control in their production pipelines. Building 3D tooling from scratch can be cost prohibitive, take years to develop, and require significant resources to maintain. These developers end up choosing either to spend critical dollars reinventing the wheel, or to use proprietary solutions that can be difficult to customize.
To solve some of these challenges, we introduced the Lumberyard game engine in 2016. Lumberyard provided a completely free (no royalties or seat licenses), source-available, real-time 3D development engine that made it possible to build, deploy, and scale quickly with cloud integrations. Over the past 5 years, we continued evolving the product for our customers. We built a powerful component entity system and a popular Script Canvas visual scripting engine. We also acquired the Emotion FX animation editor, created a modular Gems system, and implemented many Twitch and AWS integrations. But one thing that never changed was our mission: making free, world class 3D rendering tools accessible to everyone . Now, we’re taking it one step further.
The journey to open source
As we began building the successor to Lumberyard, we realized we could do more. If the ultimate goal is innovation, we want to enable the game and simulation developer community to work side by side with us, in an open community, so they could access the technology as well as contribute to it and grow it. That’s why the Linux Foundation has announced the Open 3D Foundation, with AWS seeding the foundation with the Open 3D Engine (O3DE). O3DE is a AAA-capable, cross-platform open source game engine. As part of an open source community, O3DE can expand 3D development for games and simulations by providing all the tools that developers need to bring their real-time 3D environments to life. This engine is available under an Apache 2.0 license, so anyone can build and retain their intellectual property, and choose to contribute back to the project.
We invested over a year to recruit partners with the right mix of resources, expertise, and above all, motivation to foster a self-sustaining community. We partnered with Linux Foundation as our trusted expert open-source organizational home because it stands as one of the best on the planet at managing large open-source projects. The Linux Foundation can provide the level of expert management demanded by such a large open-source effort. We are thrilled for the backing of a range of partners who feel just as strongly as we do about empowering choice for games and simulations developers. These partners include: Accelbyte, Adobe, Apocalypse Studios, Audiokinetic, Backtrace.io, Carbonated, Futurewei, GAMEPOCH, Genvid Technologies, Hadean, Huawei, HERE Technologies, Intel, International Game Developers Association, Kythera AI, Niantic, Open Robotics, PopcornFX, Red Hat, Rochester Institute of Technology, SideFX, Tafi, TLM Partners, and Wargaming. Partnership with an established open-source foundation also means creating a balanced ecosystem and drives mutual success. This flywheel consists of the open-source community’s direction and innovation, coupled with partners and project developers building the product. Ultimately, this collaboration drives newer innovations. This model continually focuses our efforts on the things customers expect from Amazon and AWS, including innovations like massive scale support, cloud rendering, a studio in the cloud, and many native features for the engine itself.
So what’s changed? Is this Lumberyard with an open-source license?
In short, a lot! Yes, it’s open source, under the permissive Apache 2.0 license. And no, O3DE is very different than the artist formerly known as Lumberyard. We leaned heavily on our Lumberyard experiences, iterated, and improved O3DE for eventual collaboration and creative control. We kept the parts that customers loved most about Lumberyard and significantly revamped the rest. We aimed to build an engine that could stand the test of time in an open source world. Because game engines tend to be monolithic, we leaned heavily toward becoming modular with extensibility, embracing open standards tooling from the onset. However, we remained unsatisfied, so we added a new prefab system, a new build system, an extensible UI, many new cloud capabilities, numerous math library optimizations, new networking capabilities, and far too many performance improvements to mention here. Also, for good measure, we even added a whole new PBR renderer capable of forward+ and deferred rendering with ray tracing and GI support!
Modularity
The engine has been redesigned from the ground up to be modular so that nearly every part of the engine is a library. Developers can replace the graphics renderer, audio subsystem, editor, language support, network stack, physics system, and other libraries entirely without affecting other libraries. The core engine modules and any add-ons or plugins are collectively known as “Gems”. Developers can slim down their project by bringing in any of the 30+ features and tools, instead of everything plus the kitchen sink. They can also implement these features and tools as independent libraries. If a developer already has an engine and wants to reduce technical debt or update their code, they can easily choose their desired features from the open-source project to implement into their own tech stacks. Or, they can create and incorporate their own. Adding new features is straightforward with O3DE’s modular architecture, allowing developers to innovate continually.
Open-source friendly build system
For developers, modularity must also be easily understood and allow for rapid development, so we decided to move the entire build system into the open-source CMake system. This choice opened up better support for CTest, plug-in libraries, profiling, edit and continue, fast code generation, and native project generation for various IDEs like Visual Studio, XCode, and others. This new approach helps save time with a proper dependency tree to build the target of choice and ensure that only actual dependencies are rebuilt, increasing the speed of iterative development. This modularity focus also allowed us to simplify project and Gem management by removing all binary formats and moving them to human-readable JSON files that can be scripted against with CMake. As a result, adding or removing Gems to a project can be done with a simple, single-line change to a JSON file. Read more about the future of the Lumberyard Build System in the blog.
Engine-as-an-SDK
Lumberyard developers will find the engine has changed considerably. It has been built like an SDK, in that it can be provided in a pre-compiled form for engine modules and Gems, in what could be considered an “engine-as-an-SDK” approach. Likewise, precompiled libraries or Gems do not need to be recompiled when the project changes. And, when new engine versions are released, developers can update the core Gems and will only need to resolve compile issues in the project code itself. Studios can now easily customize engine components and replace the prebuilt components with specific custom modules as needed.
The Atom Renderer
Another significant change is the Atom Renderer, which, as expected, is delivered as a Gem. This renderer supports multiple platforms by providing a modern physically based renderer (PBR) that is ACES colorspace-compliant and fully supports ray-tracing on Vulkan, Metal (in development), and DirectX 12. Atom ships with Global Illumination (GI) and forward+ rendering out of the box, with support for deferred rendering pipelines. Like O3DE itself, Atom is modular, scriptable, and data-driven. We even wrote an entirely new shading language, known as AZSL. This language utilizes Atom’s data-driven capabilities so developers can continually innovate as new hardware interfaces emerge. And yes, Atom is open source and free just like everything else within the O3DE. Learn more about the Atom Renderer in this blog post.
Networking
Included within O3DE is a new networking stack featuring a highly flexible data-driven model supporting TCP/UDP with a low latency transport layer abstracted behind a simple API. The stack supports encryption and compression with a built-in simulator for latency, jitter, reordering, and loss. To ensure the lowest latency and highest fidelity, it supports entity replication via unordered unreliable data replication and local prediction latency compensation with backward reconciliation for server authority, as well as detachable player behaviors supporting automated desync detection and correction. To ensure that developers can use different server models, the stack supports player-hosted and dedicated server models out of the box. Get more development details in this blog post.
Faster, lighter, more performant
O3DE is significantly lighter, faster, and more performant than its Lumberyard ancestor. Our engineers stripped away over 2 million lines of legacy code redundancies to prepare for open source. We created new, high performance math libraries that leverage modern CPUs and current SIMD instruction sets. Math stands at the heart of great games and simulations, and we’ve ensured our customers, contributors, and community have an extremely solid foundation to work from. The end result will be faster frame rates, improved platform support (especially on ARM devices), higher accuracy, and increased compute resources available for animation, special effects, and gameplay. After all, nearly everything in a 3D engine is math! Read more about our high-performance math work in this blog post.
Cloud-enabled
We’ve added additional cloud capabilities with new support for Amazon Kinesis, AWS Lambda, Amazon Cognito, Amazon CloudWatch, AWS CloudTrail, AWS IAM, Amazon GameLift, and an updated AWS C++ SDK. These deep integrations into cloud services allow developers to deliver games and simulations featuring customizable support, innovate rapidly, and keep both existing and new players coming back for more. O3DE’s cloud capabilities will grow even further over time as partners and the community continue adding more options beyond AWS. As well, AWS will continue building even more cloud features and support for additional file formats spanning a range of new use cases in the coming months.
Productivity and extensibility
Tooling and pipeline implementations are as crucial to the time needed to streamline production. To help accelerate the customization, we have extended the UI tooling with Python extensions that access code interfaces and Qt to create custom editor components. We’ve substantially updated asset creation with new pre- and post-step Python bindings to allow in-flight processing and custom behaviors for FBX and material processing. These bindings enable technical artists to write Python scripts that may split, assign, or rehome assets, as well as create custom materials from any input source.
Many other features have changed so that the community can extend them in new ways. Script Canvas now supports compiling to Lua with future native code support, with script binding via our extensible behavior context manager. We’ve stepped away from Lumberyard’s “slice” model for managing in-game assets to the well-understood model of prefabs. Our modular standard interface enables developers to conduct direct function calling against other Gem interfaces and provides IDE autocomplete support. Overall, one of our goals is enabling developers to discover and implement features — as a whole or in parts — in their projects.
What next?
Open 3D Engine is currently available in developer preview and our teams are working diligently to provide a production-ready version available later this year. We are excited to collaborate with the community and other Open 3D Foundation partners to continually add more to the engine in the months and years ahead. We will be working with the Linux Foundation, partners, and colleges to help drive more innovation and excitement about yet-to-be-announced new annual O3DE event along with industry shows and hackathons worldwide.
Our hope is that this open-source effort provides a game and simulation development platform that anyone can leverage to quickly build even more amazing games and real-time simulations. We believe this is an electrifying time to work in the 3D visualization and tools business, and we hope you will join us in this journey. As we like to say at Amazon, it is still very much Day 1.
Read more on the Open 3D Engine initiative at docs.o3de.org and watch the video.