I saw the next generation Agent framework in Project89

Author: 0xhhh

Let me first state the conclusion.@project_89A completely new approach is used to design Agent Framework. This is a high-performance Agent Framework for game development. Compared with the currently used Agent Framework, it is more modular and has better performance.

This article took a long time to write, trying to make everyone understand what architectural upgrades this framework has made compared to traditional Agent frameworks. It took many revisions to get to this version, but some parts of the article are still too technical, and I haven't been able to further popularize them. If you have any suggestions for improving the article, please leave your comments.

Developer Background

This is a technical blog, so let’s first take a look at the technical strength of this founder:

Founder did this project before doing project89:https://github.com/Oneirocom/Magick, is also aAIShaw is the fourth-ranked developer of the programming software and this project can also be seen in Shaw's resume.

Top left: Founder of project89, Bottom right: Lalaune is Shaw of ai16z

Today we will mainly introduce the high-performance Agent Framework in project89:

https://github.com/project-89/argOS

1. Why use ECS to design Agent Framework?

From the application of the game field, the games currently using the ECS architecture include: BlockchainGames: Mud, Dojo Traditional games: Overwatch, Star Citizen, etc. And the current mainstream game engines are also evolving towards ECS, such as Unity.

What is ECS?

ECS (Entity-Component-System) is a commonly used architectural pattern in game development and simulation systems. It completely separates data from logic to efficiently manage various entities and their behaviors in large-scale scalable scenarios:

1. Entity • Just an ID (number or string) that does not contain any data or logic. • You can attach different components to it to give it various properties or capabilities as needed.

2. Component • Used to store specific data or status of an entity.

3. System • Responsible for executing logic related to certain components.

Let's understand this system with an example of a specific Agent action: In ArgOS, each Agent is regarded as an Entity, which can register different components. For example, in the figure below, our Agent has the following four components: Agent Component: mainly stores basic information such as Agent name, model name, etc. Perception Component: mainly used to store perceived external data Memory Component: mainly used to store the Agent Entity's Memory data, such as things done, etc. Action Component: mainly stores the Action data to be executed.

System workflow:

1. In this game, for example, if an agent senses a weapon in front of it, the execution function of the Perception System will be called to update the data in the Perception Component of the Agent Entity.

2. Then the Memory System is triggered, and the Perception Component and Memory Component are called at the same time, and the perceived data is persisted to the database through the Memory.

3. Then the Action System calls the Memory Component and the Action Component to obtain the surrounding environment information from the memory, and finally executes the corresponding action;

4. Then we get an Update Agent Entity where each Component data is updated. So we can see that the System is mainly responsible for defining which Components to execute the corresponding processing logic on.

And it is obvious that in project89 it is a world filled with various types of agents. For example, some agents not only have the above basic abilities but also the ability to make plans. Then it will be as shown in the following figure:

System operation process

However, the actual system execution process is not the traditional approach that we imagine, where the Perception System calls the Memory System after execution. There is no calling relationship between different systems. Each system will be executed once within a specified cycle, for example:

• The Perception System may execute once every 2 seconds to update the received external perceptions and update them to the Perception Component

• The Memory System may be executed once every 1 second, extracting data from the Perception Component and loading it into the Memory Component.

• The Plan System may be executed every 1000 seconds. Based on the information received, it determines whether it needs to make a reasonable plan based on the corresponding goal optimization, and then records this part of the update in the Plan Component.

• The Action System may also be executed every 2 seconds, so that it can respond in time according to external information. At the same time, if the Plan Component is updated, it needs to update its own Action Component based on this data, which in turn affects the initial Action.

The article so far is based on my understanding of ArgOS and greatly simplifies its architecture for easier understanding. Next, let’s take a look at the real ArgOS.

2. ArgOS System Architecture

In ArgOS, many components and systems are designed to enable Agents to think more deeply and perform more complex tasks.

And ArgOS divides the System into three levels (ConsciousnessLevel):

1) Conscious System

• Contains RoomSystem, PerceptionSystem, ExperienceSystem, ThinkingSystem, ActionSystem, CleanupSystem)

• The update frequency is usually high (eg every 10 seconds).

• Processing that is closer to the "real-time" or "conscious" level, such as environmental perception, real-time thinking, and action execution.

2) Subconscious System

• GoalPlanningSystem, PlanningSystem

• The update frequency is relatively low (eg every 25 seconds).

• Handles "thinking" logic, such as periodically checking/generating goals and plans.

3) UNCONSCIOUS SYSTEM

• Not yet enabled

• Update frequency is slower (e.g. more than 50 seconds),

Therefore, in ArgOS, different systems are divided according to the Consciousness Level to determine how often the system should be executed.

Why is it designed this way? Because the relationship between the various systems in ArgOS is extremely complex, as shown in the following figure:

1. The PerceptionSystem is responsible for collecting "stimuli" from the outside world or other entities and updating them to the Perception component of the agent. It determines whether the stimulus has changed significantly, and determines whether the stimulus has changed significantly based on the stability and processing mode (ACTIVE/REFLECTIVE/WAITING) and so on. Finally, it provides “current perception” information for subsequent ExperienceSystem, ThinkingSystem, etc.

2. ExperienceSystem converts Stimuli collected by PerceptionSystem into a more abstract "Experience". It calls LLM or rule logic (extractExperiences) to identify new experiences and store them in the Memory component. It removes duplicates, filters, and verifies experiences, and triggers "experience" events to other systems or external listeners through eventBus.

3. ThinkingSystem is the agent's own "thinking" system. It extracts the current state from components such as Memory and Perception, and generates "thinking results" (ThoughtResult) through generateThought(...) and LLM/rule logic. Based on the thinking results, it may: • Update the thoughts (thinking history) in Memory. • Trigger a new Action (put it in Action.pendingAction[eid]). • Change the agent's external Appearance (expression, posture, etc.), and generate related Stimulus to let other entities "see" the changes.

4. ActionSystem If the Action.pendingAction of an Agent is not empty, the action is actually executed through runtime.getActionManager().executeAction(…). After execution, the result is written back to Action.lastActionResult, and the room or other entity is notified. CognitiveStimulus is also generated so that the subsequent system "knows" that the action has been completed or can be included in memory.

5. GoalPlanningSystem periodically evaluates the progress of the goals in the Goal.current[eid] list, or checks whether there are significant changes in external/self-memory (detectSignificantChanges). When a new goal is needed or a goal adjustment is made, it is generated and written to Goal.current[eid] through generateGoals(…). At the same time, the goals in progress (in_progress) are updated. If they meet the completion or failure conditions, the state is changed, and a completion/failure signal is sent to the corresponding Plan.

6. PlanningSystem generates or updates the Plan (execution plan) for the "existing goal" (Goal.current[eid]). If it is detected that some goals do not have corresponding active plans, an execution roadmap containing several steps is generated through generatePlan(...) and written to Plan.plans[eid]. It also updates the Plan status associated with it when the goal is completed or failed, and generates corresponding cognitive stimulation.

7. RoomSystem handles updates related to the room: • Get the list of occupants in the room, generate "appearance" stimuli for each agent, so that other entities can "see" his appearance or actions. • Create and associate room environment Stimulus (such as appropriate "room atmosphere" information). Ensure that when the agent is in a certain spatial environment, other entities that are perceiving the space can perceive his appearance changes.

8. CleanupSystem periodically searches for and removes entities marked with Cleanup components, which is used to recycle Stimulus or other objects that are no longer needed, and prevent a large number of invalid entities from being left in the ECS.

• Example: A cycle from "seeing an object" to "performing an action"

The following scene example shows how each system cooperates to complete a complete process in one round (or several frames).

1. Scene preparation: There is an Agent (EID=1) in the world (World), which is in the "Active" state and is in a Room (EID=100). A new item "MagicSword" appears in the Room, and the corresponding Stimulus is generated.

2. The PerceptionSystem detects the appearance of "MagicSword", generates a Stimulus (type="item_appearance") for Agent(1) and adds it to Perception.currentStimuli[1]. Comparing it with the previous Stimului Hash, it is determined that "there is a significant change" and "reactivates" the Agent's ProcessingState (ACTIVE mode).

3. The ExperienceSystem sees that the Perception.currentStimuli of Agent(1) is not null, so it extracts the information such as "Sword appears" into one or more new Experiences (type: "observation"), stores them in Memory.experiences[1], and emits an "experience" event.

4. ThinkingSystem reads the status information of Memory, Perception, etc., and calls generateThought: "I saw the MagicSword, maybe I can pick it up and see what it can do..." The result of this thought contains an Action to be executed: { tool: "pickUpItem", parameters: { itemName: "MagicSword" } } ThinkingSystem writes this Action to Action.pendingAction[1]. If there is a change in appearance (for example, "with a curious expression"), the Appearance is updated and visual stimulation is generated.

5. ActionSystem sees Action.pendingAction[1] = { tool: "pickUpItem", parameters: ... }. Execute the "pickUp" action logic through runtime.getActionManager().executeAction("pickUpItem", 1, { itemName: "MagicSword" }, runtime). Get the result: { success: true, message: "You picked up the magic sword" }, update to Action.lastActionResult[1], and trigger the "action" event broadcast to the room (100). At the same time, generate cognitive stimulation (type="action_result"), write to Memory or let ThinkingSystem capture it in the next round.

6. GoalPlanningSystem (if the agent has goals) periodically evaluates the agent's goals, and if one of the agent's goals is "get a powerful weapon" and detects that the MagicSword has been obtained, it may mark that goal as completed. If new changes occur (e.g. "a new object appeared in the room" that affects the agent's goal?), a new goal is generated or the old goal is abandoned according to detectSignificantChanges.

7. PlanningSystem (if there are related goals) checks if a new plan is needed or an existing plan is updated for a completed or newly generated goal such as "obtain a powerful weapon". If it is completed, the associated plan [status] is set to "completed"; or if the goal is to expand the subsequent process ("research the magic sword"), more steps are generated.

8. RoomSystem updates the Occupants list and visible entities in room (100) (every frame or every round). If the appearance of agent (1) changes (e.g. Appearance.currentAction = "holding sword"), a new "appearance" visual stimulus is created to let other Agent2 and Agent3 in the same room know that "agent1 has picked up a sword".

9. CleanupSystem removes marked (Cleanup) entities or stimuli. If you no longer need to keep the "MagicSword" Stimulus after picking it up, you can delete the corresponding Stimulus entity in CleanupSystem.

Through the connection of these systems,AI The Agent achieves: • Perceive environmental changes (Perception) → Record or convert into internal experience (Experience) → Self-thinking and decision-making (Thinking) → Take action (Action) → Dynamically adjust goals and plans (GoalPlanning + Planning) → Synchronize environment (Room) → Timely recycle useless entities (Cleanup).

3. Analysis of ArgOS overall architecture

1. Core architecture layering

2. Component classification

In ECS, each entity can have several components. Based on their nature and life cycle in the system, components can be roughly divided into the following categories:

1. Core Identity Classes (Identity-Level Components) • Agent / PlayerProfile / NPCProfile, etc. • Used to uniquely identify entities, store core character or unit information, and generally need to be persisted to the database.

2. Behavior & State Components • Action, Goal, Plan, ProcessingState, etc. • Represents the entity's current task or goal, as well as the response status to external commands and internal thoughts. • Contains pendingAction, goalsInProgress, plans, and thoughts or tasks in the queue, etc. • Usually medium/short-term states, many of which will change dynamically with game rounds or business cycles. • Whether to store data depends on the situation. If you want to resume running from a breakpoint, you may write to the database regularly.

3. Perception & Memory Components • Perception, Memory, Stimulus, Experience, etc. • Records the external information perceived by the entity (Stimuli) and the experience (Experiences) extracted after perception. • Memory can often accumulate a large amount of data, such as conversation records, event history, etc.; it often needs to be persisted. • Perception may be real-time or temporary information, mostly valid for a short period of time, and can be decided based on needs.Xiaobai NavigationWrite to a database (for example, only store important perception events).

4. Environment and Space Classes (Room, OccupiesRoom, Spatial, Environment, Inventory, etc.) • Represents information such as rooms, environments, locations, and item containers.Room.id, OccupiesRoom, Environment and other fields often need to be persisted, such as room homepage descriptions, map structures, etc. • Components that are constantly changing (such as Entity moving between different rooms) can be written in an event-based or periodic manner.

5. Appearance and interaction classes (Appearance, UIState, Relationship, etc.) • Record the “visible” or “interactive” parts of the entity, such as Avatar, pose, facialExpression, social relationship network with other entities, etc. • Some parts can be processed only in memory (real-time performance), while others (such as key social relationships) may need to be persisted.

6. Auxiliary or operation and maintenance classes (Cleanup, DebugInfo, ProfilingData, etc.) • Used to mark which entities need to be recycled (Cleanup) or record debugging information (DebugInfo) for use in monitoring and analysis. • Generally only exists in memory and is rarely synchronized to the database unless required for logging or auditing.

3. System architecture

As mentioned above

4.Manager Architecture

In addition to Component and System, we actually lack a resource manager, such as how to access the database, how to deal with conflicts in status updates, etc.

Systems on the left (PerceptionSystem, ExperienceSystem, ThinkingSystem, etc.):

• Each system is scheduled to execute by SimulationRuntime in the ECS loop, querying and processing the entities it is concerned about (through component conditions).

• When executing logic, it is necessary to interact with Managers, for example:

• Call RoomManager (RM) to query/update room information.

• Use StateManager (SM) to get or save world/agent states, such as Memory, Goal, Plan, etc.

• Broadcast or listen to events via EventBus (EB).

• When some natural language processing or prompts are needed, the PromptManager (PM) is called.

—————-

Managers on the right (EventBus, RoomManager, StateManager, EventManager, ActionManager, PromptManager, etc.):

• Provides system-level functions, basically does not actively “drive” logic, but is called by Systems or Runtime.

• Typical examples:

• ActionManager specifically manages the registration and execution of actions.

• EventManager / EventBus is used for event publishing and subscription mechanism.

• RoomManager manages rooms, layouts, and occupants.

• StateManager is responsible for the synchronization between ECS and database or storage.

• PromptManager provides extensions such as LLM Prompt templates and context management.

• The middle SimulationRuntime (R):

• Is the “scheduler” of all Systems, starting or stopping system cycles at different levels (Conscious/Subconscious, etc.);

• Managers are also created during the construction phase and passed to each System for use.

• CleanupSystem:

• Note in particular that it also interacts with ComponentSync (CS) to synchronize the removal of components or event subscriptions when entities are recycled.

Conclusion: Each System will read and write data or call services through the corresponding Manager when needed, while the Runtime will uniformly schedule the life cycle and behavior of all Systems and Managers at a higher level.

5. How to interact with the database

In ECS, Systems is where the logic is actually executed, and database reading and writing can be done through a "Persistence Manager (PersistenceManager / DatabaseManager)" or "State Manager (StateManager)". The general process is as follows:

1. Initial Load • StateManager / PersistenceManager loads data of core persistent components such as Agents, Rooms, Goals, etc. from the database, creates corresponding entities, and initializes related component fields. • For example, read a batch of agent records and insert them into the ECS world, and initialize Agent, Memory, Goal, etc. for them.

2. ECS runtime (Systems Update Loop) • The system does things in each frame (or round): PerceptionSystem collects "perceptions" and writes them to the Perception component (mostly short-term and not stored in the database). ExperienceSystem writes new "cognitive experiences" to Memory.experiences. If it is a key experience, it may also call StateManager for immediate storage, or mark it with "needsPersistence" for subsequent batch writing. ThinkingSystem/ActionSystem/GoalPlanningSystem, etc. make decisions based on component content and update fields in ECS. If some components (such as Goal.current) undergo major changes and need to be persisted (such as new long-term goals), notify StateManager to write the field to the database through component listening or system events.

3. Periodic or Event-Driven • You can call interfaces such as PersistenceManager.storeComponentData(eid, "Goal") to store data at certain key points in the system (such as when a goal plan is updated or an important event occurs in the Agent). • You can also let StateManager scan components or entities with the "needsPersistence" tag in CleanupSystem or a timer and write them back to the database at one time. • In addition, logs or audit data (such as action history, thinking logs) can also be archived and stored here.

4. Manual or Shutdown Save • When the server or process is shutting down, StateManager.saveAll() is used to write all the data that has not been written to the database to ensure that the ECS state can be restored the next time it is loaded. • For some stand-alone/offline scenarios, you can also trigger the archive manually.

• Complete example process

Here is a simple scenario that demonstrates possible ways for components to interact with the database:

1. At startup: StateManager.queryDB("SELECT * FROM agents") → Get a batch of agent records, create Entity (EID=x) for each record in turn, and initialize the Agent, Memory, Goal and other component fields. At the same time, load room information from the "rooms" table and create a Room entity.

2. Running phase: PerceptionSystem detects the event "MagicSword appears" in a room and writes it to Perception.currentStimuli[eid]. ExperienceSystem converts Stimuli into Experience and assigns it to Memory.experiences[eid]. ThinkingSystem decides the next action based on information such as Memory and Goal, and generates Action.pendingAction[eid]. After ActionSystem executes the action, it writes the result to Memory or Action.lastActionResult. If this is a major plot event, the latest part of Memory.experiences[eid] will be marked as needsPersistence. After a period of time, StateManager finds that Memory.experiences[eid] has "needsPersistence" and writes it to the database (INSERT INTO memory_experiences ...).

3. Stop or breakpoint: Based on ECS or system scheduling, call StateManager.saveAll() when the server is shut down to write the latest states of key component fields (Agent, Memory, Goal, etc.) still in memory into the database. The next time you restart, you can load and restore the ECS world state from the database.

• Categorizing components not only facilitates clear management of entity data in the project, but also helps us control the boundaries of data that "needs to be persisted" and "exists only in memory". • Interaction with the database is usually handled by a dedicated Manager (such as StateManager). Systems use it to read and write databases, avoiding writing SQL or similar low-level statements directly in the system. • In this way, you can enjoy the efficiency and flexibility of ECS logic, as well as the persistence, breakpoint resumption and data statistical analysis advantages brought by the database.

5. Architecture Innovation

• The highlights of the entire architecture are:

• Each system runs independently and has no calling relationship with other systems. Therefore, even if we want to realize the Agent's ability to "perceive environmental changes (Perception) → record or convert into internal experience (Experience) → self-thinking and decision-making (Thinking) → take action (Action) → dynamically adjust goals and plans (GoalPlanning + Planning) → synchronize the environment (Room) → timely recycle useless entities (Cleanup)", each system will have many interdependent points in terms of function, but we can still use the ECS architecture to structure the whole into various unrelated systems. Each system can still run independently and will not have human or coupling relationships with other systems. I think this is also the main reason why Unity has increasingly migrated to the ECS architecture in recent years.

• And for example, if I just want an Agent to have some basic capabilities, I can easily achieve it by reducing the registration of some Components and the registration of Systems when defining Entities, without changing a few lines of code.

As shown below:

• At the same time, if you want to add new functions during the development process, it will not affect other systems, and you can easily load the functions you want. In addition, the performance of this current architecture is actually much stronger than that of the traditional object-oriented architecture. This is a recognized fact in the gaming industry, because the design of ECS is more suitable for concurrency. So when we are in a complex Defai scenario, using the ECS architecture may also be more advantageous, especially in the scenario of using Agent for quantitative trading, ECS will also be useful (not just in the Agent game scenario).

• Dividing the system into conscious, subconscious and unconscious to distinguish how long different types of systems should be executed is an extremely clever design, and it is already a very concrete human ability.

From my personal point of view, this is an extremely modular framework with excellent performance. The code quality is also very high and it contains good design documents. Unfortunately, the $project89 project has been lacking in publicity for this framework. Therefore, I spent a long time (4 days) writing this article. I think good things are still worth being discovered. An English version should be released tomorrow. I hope that more game teams or Defai teams will use this framework and provide everyone with a new potential architecture choice!

The article comes from the Internet:I saw the next generation Agent framework in Project89

Related recommendations: AICC event follow-up: When VCs get involved in AI Agent Token

Good news: One-day fast trading again; bad news: market value fell nearly 80% from its high point. Written by: Wenser, Odaily Planet Daily In just 3 days, the market value ofcryptocurrencyFusion with artificial intelligence DAO The Aiccelerate dao, which is called "Organization", has completed the reversal from being the center of attention to being attacked by everyone, which makes people feel...