Cross Language Soccer Framework: An Open Source Framework for the RoboCup 2D Soccer Simulation

Soccer, a sport universally celebrated for its dynamic nature and strategic gameplay[1, 2, 3], serves as a captivating domain for artificial intelligence research. The RoboCup organization was established to advance the abilities of autonomous agents to understand and play soccer. The ultimate objective of this organization is to create a fully autonomous team that can defeat the FIFA World Cup champion by the year 2050 [5, 4, 7, 6]. Soccer Simulation 2D League (SS2D) is a platform that promotes the development of artificial intelligence in multi-agent environments [8, 9, 10, 11, 12, 13, 14, 15, 16]. Figure 1 provides a detailed visualization of the Soccer Simulation 2D game environment.

Existing widely-used C++-based codes like Helios, Cyrus2D, and Gliders offer tools that allow researchers to focus on the decision challenges within the environment [17, 18, 23]. Nevertheless, due to the inherent complexity of C++ and its limited integration with machine-learning frameworks, incorporating machine-learning methods into these frameworks poses significant challenges. Addressing these challenges involves the development of new frameworks in alternative languages. However, creating frameworks in different languages is a time-consuming process, and the performance of these languages may not be sufficiently high for making timely decisions. For example, Python, a widely used and straightforward programming language, lacks the necessary performance capabilities in the context of SS2D[29].

Recognizing the advantages of a high-performance base code, such as Helios, which completely supports SS2D server features, we propose a novel solution. This development paper introduces an innovative approach to overcome language barriers and enhance flexibility. By connecting the robust Helios base code to other languages using gRPC, we aim to provide researchers with the freedom to work in their preferred languages without compromising the performance benefits offered by C++. This approach empowers researchers to focus on the intelligence aspects of their agents, leveraging the high-quality and high-performance foundation laid by Helios.

As illustrated in Figure 3, the Cross Language Soccer framework architecture seamlessly integrates the Helios base code with a language-agnostic gRPC server. This connection facilitates efficient communication and collaboration between the high-performance Helios base code and multiple programming languages, empowering researchers in the RoboCup 2D Soccer Simulation domain.

In the following sections, we provide a detailed explanation of our approach, including the framework’s architecture, implementation details, and its implications for research in the SS2D environment.

The RoboCup Soccer Simulation 2D server (server) has two main jobs: it runs soccer games and interacts with coaches, players, trainers, and monitors. It communicates by sending observations to agents, receiving low-level actions from them, and applying them to the game. Various teams have contributed base codes to the RoboCup soccer simulation, for instance, the ”CMUnited” team from Carnegie Mellon University (USA)[19, 20], the ”UvA Trilearn” team from the University of Amsterdam (The Netherlands)[21], the ”MarliK” team from the University of Guilan (Iran)[25], the ”HELIOS” team from AIST Information Technology Research Institute (Japan), the ”Wrighteagle” team from the University of Science and Technology of China[22], the ”Gliders2d” team from the University of Sydney (Australia)[23, 24], and the ”Cyrus2d” and ”Pyrus” teams from Dalhousie University (Canada)[16, 18, 29].

The Helios base code provides in processing observations by denoising them and constructing a detailed world model that contains all of the information required for decision-making, such as an intercept table. Also, its richness in functionality, encompasses high-level actions (e.g., BodyGoToPoint, BodySmartKick), offensive planners (e.g. ChainActions), and extensive logging functionality, offering graphical logs through the SoccerWindow2 application.

In navigating the complexities of existing base codes like Cyrus and Gliders, which are built upon the powerful Helios base code, researchers often face challenges due to their intricacy. While these codes provide robust capabilities, applying new algorithms or working with them requires a deep understanding of both soccer simulation 2D and the base codes. Moreover, their reliance on C++ poses hurdles, as the language lacks abundant machine-learning frameworks and is relatively challenging for researchers less familiar with SS2D or C++. On the other hand, the Pyrus base code, developed in Python, offers a more accessible option but sacrifices performance.

Half Field Offense (HFO) is an environment tailored for reinforcement learning in soccer simulations, focusing exclusively on half-field scenarios[27]. This framework modifies the standard Soccer Simulation 2D server, limiting it to half-field play, which restricts its applicability to full-match scenarios and broader training objectives. Furthermore, HFO only supports C++ and Python, excluding other programming languages. Its specialized nature and lack of support for recent server updates or features also restrict its utility for researchers looking to utilize the latest developments in soccer simulation.

To address these challenges, we propose a novel solution by using Remote Procedure Call (RPC) technology, specifically gRPC (Google Remote Procedure Call). We implemented the gRPC client on the Helios base, so it facilitates efficient communication between the base and other languages. Therefore, we streamline the development process and enhance the flexibility of researchers. This approach allows developers to benefit from the high performance of C++ for computationally intensive tasks, such as denoising and world modeling, while enabling them to implement or improve Helios base decision-making logic in languages like C#, JavaScript, Python, or others.

The use of gRPC offers several advantages, including low latency, language-agnostic communication, and efficient serialization of data. This integration helps researchers work in their preferred languages without sacrificing the performance advantages of the Helios base code. In summary, our solution facilitates a bridge between the efficiency of C++ and the flexibility of other languages, empowering researchers in the SS2D domain.

We have introduced a new approach called Cross Language Soccer Framework to enhance the flexibility and interoperability of RoboCup Soccer Simulation 2D (SS2D). The Cross Language Soccer approach consists of two components: Soccer Simulation Proxy and Playmaker Server. The Soccer Simulation Proxy is an extended version of the Helios base that can send decision-making information to the Playmaker Server. It can receive high-level actions from the Playmaker Server and send them to RoboCup Soccer Simulation Server and/or SoccerWindow2. On the other hand, the Playmaker Server receives information from the client and selects the appropriate actions to be sent back to the client. We have implemented some sample servers in C#, Python, and JavaScript, but it can also be implemented in other languages to make use of their features.

This section highlights key development challenges encountered while building the framework and the innovative solutions implemented to address these challenges. These solutions are designed to simplify the setup and use of our framework, enhancing its accessibility for researchers and participants in the RoboCup competition.

In conclusion, the SS2D-Proxy approach presented in this paper addresses the challenges associated with existing C++-based codes in RoboCup Soccer Simulation 2D (SS2D) research. The complexities and learning difficulties posed by these codes, coupled with the time-consuming process of developing new base codes for each language, motivated the need for a versatile solution.

Our innovation involves connecting the widely adopted Helios base code to a language-agnostic framework using gRPC. This approach facilitates seamless communication between Helios and different languages, such as C#, Python, and JavaScript. By doing so, we empower researchers to work in their language of choice without compromising the high-quality, high-performance features offered by the C++ Helios base code.

The integration of gRPC not only streamlines communication but also introduces efficiency through low-latency data exchange and language-agnostic capabilities. Researchers can now focus on developing intelligent multi-agent systems in SS2D without the burdensome task of creating an entire base code.

Our solution enhances collaboration, allowing researchers to leverage the strengths of C++ for computationally intensive tasks, while developing decision-making logic in languages that better suit their preferences and expertise. The protocol buffer definitions and RPC functions provide a standardized and efficient means of communication between Helios and the gRPC server, ensuring a seamless exchange of game state information and actions.

In summary, the SS2D-Proxy approach opens new avenues for collaborative research in the RoboCup SS2D domain. It bridges the gap between the efficiency of C++ and the flexibility of multiple languages, paving the way for advancements in intelligent multi-agent systems within the dynamic context of RoboCup Soccer Simulation 2D.

We extend our heartfelt appreciation to Professor Stan Matwin and Professor Amilcar Soares for their exceptional support and guidance over the past three years. Their mentorship has been instrumental in shaping the trajectory of our research in the CYRUS team.

Special thanks are extended to Professor Hidehisa Akyama for his pivotal work on the Helios base and Soccer Simulation 2D server. His pioneering efforts have laid a solid foundation for researchers in the field, including ourselves.

These acknowledgments stand as a tribute to the collaborative spirit and mentorship that have played a crucial role in the development and realization of this project.